Skip to main content
Use vLLM for GPU serving, batching, and an OpenAI-compatible API. Start with LFM3-1.7B for text or LFM3-VL-2B for images. The examples below use a CUDA GPU and vLLM 0.34.0, which adds native LFM3 support.

Installation

Use an isolated Python 3.12 environment and a CUDA setup compatible with the vLLM installation guide:
LFM3 requires 0.34.0’s native implementation. Follow the release notes when upgrading an existing deployment; defaults and dependencies changed too.

Serve LFM3 text

Review the checkpoint’s configuration code before enabling --trust-remote-code. --enforce-eager uses eager execution for this quickstart rather than CUDA graph capture. The 4,096-token limit is a small quickstart budget, not the checkpoint’s maximum context. Increase it to fit your task and GPU memory. Install the client with uv pip install openai, then send a request:
Leave room for reasoning as well as the final answer. The reasoning parser separates reasoning from content; it does not disable thinking. See Thinking. Set stream=True for streaming responses, and pass OpenAI-style tools for parsed tool calls. Use response_format with a JSON schema for constrained JSON output.

Vision, audio, and omni

For images, use the launch configuration on the LFM3-VL model page, then send OpenAI image_url content parts. Multimodal input limits and memory needs differ from text-only serving. Upstream vLLM lists additional LFM3 multimodal architectures. Architecture registration alone does not establish speech output, video handling, or a particular API contract. Follow a tested example for the exact checkpoint and modality; the Audio and Omni pages retain their documented runtime paths.

LFM2.5 and LFM2

Older supported checkpoints keep their native architectures and tool-call formats:
The client request has the same shape. Change model to match the server and use that checkpoint’s sampling settings. For LFM2.5-1.2B-Instruct, use temperature=0.1, top_k=50, and repetition_penalty=1.05. Original LFM2 models and thinking/MoE variants have their own settings; check the model’s quickstart rather than inheriting LFM3’s parser or generation preset.

LFM2.5 vLLM Cookbook

Keep the older family-specific recipes for MoE, Japanese, DSpark, vision, hardware tuning, and sampling. Their model/version scope is separate from the LFM3 quickstart above.