> ## Documentation Index
> Fetch the complete documentation index at: https://docs.liquid.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# LFM2.5-8B-A1B
> 8B parameter Mixture-of-Experts model with 1.5B active parameters for fast, reliable tool calling and 128K context
export const TextLlamacpp = ({ggufRepo, samplingFlags}) =>
Install:
{`brew install llama.cpp`}
Run:
{`llama-cli -hf ${ggufRepo} -c 4096 --color -i \\
${samplingFlags}`}
The -hf flag downloads the model directly from Hugging Face. For other installation methods and advanced usage, see the llama.cpp guide.
;
export const TextSglang = ({modelId, toolCallParser, samplingParams}) =>
Install:
{`uv pip install "sglang>=0.5.10"`}
Launch server:
{toolCallParser ? `sglang serve \\
--model-path ${modelId} \\
--host 0.0.0.0 \\
--port 30000 \\
--tool-call-parser ${toolCallParser}` : `sglang serve \\
--model-path ${modelId} \\
--host 0.0.0.0 \\
--port 30000`}
Query (OpenAI-compatible):
{`from openai import OpenAI
client = OpenAI(base_url="http://localhost:30000/v1", api_key="None")
response = client.chat.completions.create(
model="${modelId}",
messages=[{"role": "user", "content": "What is machine learning?"}],
${samplingParams || "temperature=0.3,"}
)
print(response.choices[0].message.content)`}
;
export const TextVllm = ({modelId, samplingParams, maxTokens}) =>
Install:
{`pip install vllm==0.14`}
Run:
{`from vllm import LLM, SamplingParams
llm = LLM(model="${modelId}")
sampling_params = SamplingParams(${samplingParams}max_tokens=${maxTokens || 512})
output = llm.chat("What is machine learning?", sampling_params)
print(output[0].outputs[0].text)`}
;
export const TextTransformers = ({modelId, samplingParams}) =>
Install:
{`pip install "transformers>=5.2.0" torch accelerate`}
Download & Run:
{`from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "${modelId}"
model = AutoModelForCausalLM.from_pretrained(
model_id,
device_map="auto",
dtype="bfloat16",
)
tokenizer = AutoTokenizer.from_pretrained(model_id)
inputs = tokenizer.apply_chat_template(
[{"role": "user", "content": "What is machine learning?"}],
add_generation_prompt=True,
return_tensors="pt",
tokenize=True,
return_dict=True,
).to(model.device)
output = model.generate(**inputs, ${samplingParams}max_new_tokens=512)
input_length = inputs["input_ids"].shape[1]
response = tokenizer.decode(output[0][input_length:], skip_special_tokens=True)
print(response)`}
;
← Back to Text Models
LFM2.5-8B-A1B is Liquid AI's Mixture-of-Experts model, combining 8B total parameters with only 1.5B active parameters per forward pass with a 128K context window and chain of thought reasoning. This model delivers exceptional performance in tool calling and agentic tasks while running on-device.
## Specifications
| Property | Value |
| -------------- | ---------------- |
| Parameters | 8B (1.5B active) |
| Context Length | 128K tokens |
| Architecture | LFM2.5 (MoE) |
Extended context window for long documents and conversations
8B quality, 1.5B inference cost
Native function calling for agentic workflows
## Quick Start
Quick start with Transformers (compatible with `transformers>=5.0.0`):
```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
model_id = "LiquidAI/LFM2.5-8B-A1B"
model = AutoModelForCausalLM.from_pretrained(
model_id,
device_map="auto",
dtype="bfloat16",
# attn_implementation="flash_attention_2" <- uncomment on compatible GPU
)
tokenizer = AutoTokenizer.from_pretrained(model_id)
streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
prompt = "What is C. elegans?"
input_ids = tokenizer.apply_chat_template(
[{"role": "user", "content": prompt}],
add_generation_prompt=True,
return_tensors="pt",
tokenize=True,
).to(model.device)
output = model.generate(
input_ids,
do_sample=True,
temperature=0.2,
top_k=80,
repetition_penalty=1.05,
max_new_tokens=8192,
streamer=streamer,
)
```