Skip to main content

LFM3 documentation validation

Checked on 2026-11-17. These are integration smoke tests, not quality benchmarks or certification of every checkpoint, platform, or modality.

Executed checks

Text checks ask for France’s capital and require Paris in the answer. Image checks use a synthetic solid red image and require red in the answer. Serving tests inspect structured tool_calls and JSON responses rather than treating an HTTP 200 as sufficient. The reusable verifier additionally checks completion reasons and tool arguments.

Reproduce serving checks

Start the server using the relevant documentation page, then run from the repository root:
Use --repetition-penalty 1.05 for the older representative checkpoints and change the server’s model/parser settings as documented. The verifier expects a local endpoint without authentication. For vision checks, start a server with the appropriate image encoder/projector. The SGLang matrix used the Dockerfile dependencies in the guide and --context-length 4096 --mem-fraction-static 0.8. Text LFM3 used qwen3 / qwen3_coder; LFM3-VL used auto parsers; older checkpoints used lfm2. Tool arguments and completion reasons were also inspected for the successful non-streaming requests. The CUDA-image recipe was tested with normal kernel compilation and graph capture enabled. First startup can spend several minutes compiling kernels. The VL model page additionally retains its whitespace constraint flag from the earlier capability verification. The vLLM matrix used --max-model-len 4096 --gpu-memory-utilization 0.8 --enforce-eager. LFM3 used qwen3 reasoning and qwen3_coder tool parsing; older models used lfm2 tool parsing. Keep vLLM’s own Transformers dependency; do not override it with the direct-Transformers guide’s pin.

Findings retained in the docs

  • LFM3 on MLX uses MLX-VLM, not the older MLX-LM path.
  • LFM3 uses different serving parsers from LFM2/LFM2.5. A parser separates reasoning from final output; it does not switch thinking off.
  • ONNX runtime families are different: plain ONNX Runtime for LFM3 and GenAI for older LiquidONNX exports. Published bundles can come from other export pipelines.
  • ONNX external data must resolve within the graph folder. Materialize Hugging Face cache symlinks before loading these graphs.
  • Original LFM2 tool behavior differs across the tested imports. A missed tool call in llama.cpp is a model response, not proof that its parser lacks support. Ollama’s HTTP 400 is a separate imported-model capability restriction.
  • Older native bindings, provider recipes, browser demos, training notebooks, and DSpark target/drafter pairs retain their actual checkpoints. They are not made LFM3-compatible by renaming a model ID.

Validation limits

This change does not freshly certify iOS/Android native builds, Python/Node/.NET in-process bindings, LM Studio/Atomic Chat GUIs, provider deployments, every quantization, MoEs, d1, DSpark, video, or LFM3 audio/omni paths in every runtime. Historical notebooks remain scoped to their original checkpoints. Repository CI exercises their configured runnable cells, respecting existing skip directives and shortened training settings; passing CI is not a full training-quality evaluation. The new training example verifies two optimizer steps; full SFT, DPO, GRPO, and multimodal training require separate evaluation.

Documentation checks

  • Mintlify strict validation and internal broken-link check.
  • Public URL snapshot (250 active URLs, including all 13 API endpoint pages).
  • Generated snippets match their notebook/generator sources.
  • Notebook Ruff and mypy checks.
  • Python syntax checks (54 MDX Python blocks) and git diff --check.
  • Three notebook-preprocessing regression tests, including %pip extraction before script execution.