> ## Documentation Index
> Fetch the complete documentation index at: https://docs.liquid.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# LFM2.5-24B-A2B
> 24B parameter Mixture-of-Experts reasoning model with 2B active parameters and 131K context, built for fast, private agents on-device
export const TextSglang = ({modelId, toolCallParser, samplingParams}) =>
Install:
{`uv pip install "sglang>=0.5.10"`}
Launch server:
{toolCallParser ? `sglang serve \\
--model-path ${modelId} \\
--host 0.0.0.0 \\
--port 30000 \\
--tool-call-parser ${toolCallParser}` : `sglang serve \\
--model-path ${modelId} \\
--host 0.0.0.0 \\
--port 30000`}
Query (OpenAI-compatible):
{`from openai import OpenAI
client = OpenAI(base_url="http://localhost:30000/v1", api_key="None")
response = client.chat.completions.create(
model="${modelId}",
messages=[{"role": "user", "content": "What is machine learning?"}],
${samplingParams || "temperature=0.3,"}
)
print(response.choices[0].message.content)`}
;
export const TextVllm = ({modelId, samplingParams, maxTokens}) =>
Install:
{`pip install vllm==0.14`}
Run:
{`from vllm import LLM, SamplingParams
llm = LLM(model="${modelId}")
sampling_params = SamplingParams(${samplingParams}max_tokens=${maxTokens || 512})
output = llm.chat("What is machine learning?", sampling_params)
print(output[0].outputs[0].text)`}
;
export const TextLlamacpp = ({ggufRepo, samplingFlags}) =>
Install:
{`brew install llama.cpp`}
Run:
{`llama-cli -hf ${ggufRepo} --conversation \\
${samplingFlags}`}
The -hf flag downloads the model directly from Hugging Face. For other installation methods and advanced usage, see the llama.cpp guide.
;
← Back to Text Models
LFM2.5-24B-A2B is Liquid AI's largest Mixture-of-Experts model, combining 24B total parameters with only 2B active parameters per token and a 131K context window. It is a reasoning model that always thinks before it answers, post-trained for tool calling and multi-step agentic tasks on-device.
## Specifications
| Property | Value |
| - | - |
| Parameters | 24B (2B active) |
| Context Length | 131K tokens |
| Architecture | LFM2.5 (MoE), 40 layers (30 gated short convolution + 10 GQA), 64 experts per MoE block with top-4 routing |
| Vocabulary Size | 128K |
| Languages | English, Arabic, Chinese, French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, Thai, Indonesian, Hindi, Russian, Polish |
| Generation Parameters | `temperature=0.1`, `top_k=50`, `repetition_penalty=1.05` |
Post-trained inside agent harnesses for reliable tool calling
Thinks before it answers, at 2B active cost per token
Long agent trajectories, documents, and conversations
## Quick Start
Quick start with Transformers (compatible with `transformers>=5.0.0`):
```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
model_id = "LiquidAI/LFM2.5-24B-A2B"
model = AutoModelForCausalLM.from_pretrained(
model_id,
device_map="auto",
dtype="bfloat16",
# attn_implementation="flash_attention_2" <- uncomment on compatible GPU
)
tokenizer = AutoTokenizer.from_pretrained(model_id)
streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
prompt = "What is C. elegans?"
input_ids = tokenizer.apply_chat_template(
[{"role": "user", "content": prompt}],
add_generation_prompt=True,
return_tensors="pt",
tokenize=True,
)["input_ids"].to(model.device)
output = model.generate(
input_ids,
do_sample=True,
temperature=0.1,
top_k=50,
repetition_penalty=1.05,
max_new_tokens=512,
streamer=streamer,
)
```
**Install** (requires `mlx-lm>=0.32.0`, earlier versions route the experts incorrectly):
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
pip install -U "mlx-lm>=0.32.0"
```
**Run:**
```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler, make_logits_processors
model, tokenizer = load("LiquidAI/LFM2.5-24B-A2B-MLX-4bit")
prompt = "What is C. elegans?"
if tokenizer.chat_template is not None:
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
messages, add_generation_prompt=True
)
sampler = make_sampler(temp=0.1, top_k=50)
logits_processors = make_logits_processors(repetition_penalty=1.05)
response = generate(
model, tokenizer, prompt=prompt, max_tokens=4096,
sampler=sampler, logits_processors=logits_processors,
verbose=True,
)
```
5-bit, 6-bit, and 8-bit builds are also available: [MLX-5bit](https://huggingface.co/LiquidAI/LFM2.5-24B-A2B-MLX-5bit), [MLX-6bit](https://huggingface.co/LiquidAI/LFM2.5-24B-A2B-MLX-6bit), [MLX-8bit](https://huggingface.co/LiquidAI/LFM2.5-24B-A2B-MLX-8bit).
## Reasoning
Unlike LFM2-24B-A2B, LFM2.5-24B-A2B always thinks before it answers. The chat template opens every assistant turn with a `` tag, so responses contain an explicit chain of thought before the final answer. Leave enough room in your max token budget for the reasoning trace. See [Chat Template](/lfm/key-concepts/chat-template) and [Tool Use](/lfm/key-concepts/tool-use) for the message format.
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.