MLX - Liquid AI Docs

Use MLX for running models on Apple Silicon Macs with Metal GPU acceleration.

MLX leverages unified memory architecture on Apple Silicon, allowing seamless data sharing between CPU and GPU. The mlx-lm package provides a simple interface for loading and serving LLMs.

Installation

Install the MLX language model package:

pip install mlx-lm

LFM3 and d1 models use MLX-VLM instead: see LFM3 and d1 Models.

Basic Usage

The mlx-lm package provides a simple interface for text generation with MLX models. See the Models page for all available MLX models, or browse MLX community models at mlx-community LFM2 models.

from mlx_lm import load, generate

# Load model and tokenizer
model, tokenizer = load("mlx-community/LFM2-1.2B-8bit")

# Generate text
prompt = "What is machine learning?"

# Apply chat template
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, tokenizer=False, add_generation_prompt=True
)

response = generate(model, tokenizer, prompt=prompt, verbose=True)
print(response)

Generation Parameters

Control text generation behavior using parameters in the generate() function. Key parameters:

temperature (float, default 1.0): Controls randomness (0.0 = deterministic, higher = more random). Typical range: 0.1-2.0
top_p (float, default 1.0): Nucleus sampling - limits to tokens with cumulative probability ≤ top_p. Typical range: 0.1-1.0
top_k (int, default 50): Limits to top-k most probable tokens. Typical range: 1-100
max_tokens (int): Maximum number of tokens to generate
repetition_penalty (float, default 1.0): Penalty for repeating tokens (>1.0 = discourage repetition). Typical range: 1.0-1.5

Example with custom parameters:

response = generate(
    model,
    tokenizer,
    prompt=prompt,
    temperature=0.3,
    min_p=0.15,
    repetition_penalty=1.05,
    max_tokens=512
)

Streaming Generation

Stream responses with stream_generate():

from mlx_lm import load, stream_generate

model, tokenizer = load("mlx-community/LFM2-1.2B-8bit")

messages = [{"role": "user", "content": "Tell me a story about space exploration."}]
prompt = tokenizer.apply_chat_template(
    messages, tokenizer=False, add_generation_prompt=True
)

for token in stream_generate(model, tokenizer, prompt=prompt, max_tokens=512):
    print(token, end="", flush=True)

Serving with mlx-lm

MLX can serve models through an OpenAI-compatible API. Start a server with:

mlx_lm.server --model mlx-community/LFM2-1.2B-8bit --port 8080

Using the Server

Once running, use the OpenAI Python client:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8080/v1",
    api_key="not-needed"
)

response = client.chat.completions.create(
    model="mlx-community/LFM2-1.2B-8bit",
    messages=[
        {"role": "user", "content": "Explain quantum computing."}
    ],
    temperature=0.3,
    max_tokens=512,
    extra_body={"min_p": 0.15, "repetition_penalty": 1.05},
)
print(response.choices[0].message.content)

You can also use curl to interact with the server:

Curl request example

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mlx-community/LFM2-1.2B-8bit",
    "messages": [{"role": "user", "content": "Hello!"}],
    "temperature": 0.3,
    "min_p": 0.15,
    "repetition_penalty": 1.05
  }'

Vision Models

LFM2-VL models support both text and image inputs for multimodal inference. Use mlx_vlm to load and generate with vision models:

Single Image Example

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_image_processor
from PIL import Image

# Load vision model
model, processor = load("mlx-community/LFM2-VL-1.6B-8bit")

# Load image
image = Image.open("path/to/image.jpg")

# Create prompt
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image"},
            {"type": "text", "text": "What's in this image?"}
        ]
    }
]

# Apply chat template
prompt = apply_chat_template(processor, messages)

# Generate
output = generate(model, processor, image, prompt, verbose=False)
print(output)

Multiple Images Example

images = [
    Image.open("path/to/first.jpg"),
    Image.open("path/to/second.jpg")
]

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image"},
            {"type": "image"},
            {"type": "text", "text": "What are the differences between these images?"}
        ]
    }
]

prompt = apply_chat_template(processor, messages)
output = generate(model, processor, images, prompt, verbose=False)
print(output)

LFM3 and d1 Models

The LFM3 models and the d1 decision models run with MLX-VLM 0.7.11 or later:

pip install -U "mlx-vlm>=0.7.11"

Model	MLX repositories
LFM3-1.7B	4bit, 5bit, 6bit, 8bit, bf16
LFM3-VL-2B	4bit, 5bit, 6bit, 8bit, bf16
LFM3-Audio-2B	4bit, 8bit, bf16
LFM3-Omni-2B	4bit, 8bit, bf16
d1-2B	4bit, 8bit, fp16
d1-omni-2B	8bit, fp16
d1-3B (deprecated)	4bit, 8bit, fp16
d1-omni-600M	8bit, fp16

The LFM3 models run with mlx_vlm.generate:

# LFM3-1.7B: text
python -m mlx_vlm.generate --model LiquidAI/LFM3-1.7B-MLX-4bit \
  --prompt "What is the capital of Australia?" --enable-thinking \
  --temperature 0.1 --top-k 50 --repetition-penalty 1.1 --max-tokens 16384

# LFM3-VL-2B: image
python -m mlx_vlm.generate --model LiquidAI/LFM3-VL-2B-MLX-4bit \
  --image /path/to/image.jpg --prompt "Describe this image." --enable-thinking \
  --temperature 0.1 --top-p 1.0 --top-k 50 --min-p 0.0 --repetition-penalty 1.1 \
  --max-tokens 16384

# LFM3-Audio-2B: speech recognition
python -m mlx_vlm.generate --model LiquidAI/LFM3-Audio-2B-MLX-4bit \
  --system "Perform ASR in English." --prompt "" --audio /path/to/speech.wav \
  --temperature 0 --max-tokens 2048

# LFM3-Omni-2B: text to speech, saved as a 24 kHz WAV file
python -m mlx_vlm.generate --model LiquidAI/LFM3-Omni-2B-MLX-4bit \
  --system "Perform TTS in American English using a female voice." \
  --prompt "The quick brown fox jumps over the lazy dog." \
  --output-modality audio --output tts.wav

mlx_vlm.generate turns reasoning off unless you pass --enable-thinking. mlx-lm cannot load LFM3-1.7B yet. Support is pending in ml-explore/mlx-lm#2016. The d1 models answer questions instead of generating text, and d1-2B and d1-omni-2B refuse mlx_vlm.generate and chat apps. Ask them with python -m mlx_vlm decide, which prints the answers as JSON:

python -m mlx_vlm decide --model LiquidAI/d1-2B-MLX-4bit \
  --state "I was charged twice this month. Please refund one of the charges." \
  --questions '{"refund": {"type": "noul", "instructions": "Is the customer asking for a refund?"}}'

Swap the model id to ask d1-omni-2B, d1-3B, or d1-omni-600M. d1-3B and d1-omni-600M return yes/no answers as value and probability. The d1 model cards show the Python API.

Speculative Decoding

Some LFM2.5 vision models have DSpark draft models that make generation faster without changing the output. Pass the drafter to MLX-VLM (v0.7.2 or newer) with --draft-model:

python -m mlx_vlm.generate \
  --model LiquidAI/LFM2.5-VL-3B \
  --draft-model LiquidAI/LFM2.5-VL-3B-DSpark \
  --draft-block-size 8 \
  --image /path/to/image.jpg \
  --prompt "Describe this image." \
  --max-tokens 256 \
  --temperature 0

DSpark in MLX-VLM currently supports greedy sampling only, so keep --temperature 0. See Speculative Decoding for which models have drafters.

​Installation

​Basic Usage

​Generation Parameters

​Streaming Generation

​Serving with mlx-lm

​Using the Server

​Vision Models

​LFM3 and d1 Models

​Speculative Decoding

Installation

Basic Usage

Generation Parameters

Streaming Generation

Serving with mlx-lm

Using the Server

Vision Models

LFM3 and d1 Models

Speculative Decoding