> ## Documentation Index
> Fetch the complete documentation index at: https://docs.liquid.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Speculative Decoding

> Speed up LFM2.5 text and vision models with DSpark draft models in SGLang, llama.cpp, and MLX-VLM, with no change to output.

export const DsparkMlxVlm = ({modelId, draftModelId}) => <div>
<p><strong>Install:</strong></p>
<CodeBlock language="bash">
{`pip install "mlx-vlm>=0.7.2"`}
</CodeBlock>
<p><strong>Run with the drafter:</strong></p>
<CodeBlock language="bash">
{`python -m mlx_vlm.generate \\
    --model ${modelId} \\
    --draft-model ${draftModelId} \\
    --draft-block-size 8 \\
    --image /path/to/image.jpg \\
    --prompt "Describe this image." \\
    --max-tokens 256 \\
    --temperature 0`}
</CodeBlock>
<p><strong>Flags:</strong></p>
<ul>
<li><code>--draft-model</code>: the drafter repo on Hugging Face. MLX-VLM detects that it is a DSpark drafter.</li>
<li><code>--draft-block-size</code>: how many tokens the drafter proposes per step. Use 8 on Apple silicon.</li>
<li><code>--temperature 0</code>: DSpark in MLX-VLM currently supports greedy sampling only.</li>
</ul>
</div>;

export const DsparkLlamacpp = ({ggufRepo, draftRepo, vision = false, samplingFlags}) => <div>
<p><strong>Install:</strong></p>
<CodeBlock language="bash">
{`brew install llama.cpp`}
</CodeBlock>
<p>DSpark needs a recent build. Check that <code>llama-server --help</code> lists <code>draft-dspark</code> under <code>--spec-type</code>.</p>
<p><strong>Run the target with the draft sidecar:</strong></p>
<CodeBlock language="bash">
{`llama-server \\
    -hf ${ggufRepo}:F16 \\
    -hfd ${draftRepo}:F16 \\
    --spec-type draft-dspark --spec-draft-n-max ${vision ? "8" : "10"} --spec-draft-n-min 0 \\
    -ngl 99 -ngld 99 -fa on \\
    ${samplingFlags}`}
</CodeBlock>
<p><strong>Flags:</strong></p>
<ul>
<li><code>-hf</code>: the target GGUF on Hugging Face, with its quantization tag. Use <code>-m</code> for a local file.</li>
<li><code>-hfd</code>: the draft sidecar GGUF. Use <code>-md</code> for a local file. The sidecar has no embeddings or LM head, so it cannot run on its own.</li>
<li><code>--spec-type draft-dspark</code>: turns on DSpark speculative decoding.</li>
{vision ? <li><code>--spec-draft-n-max</code>: the most tokens the drafter proposes per step. The drafter was trained with block size 9. Use 8 on Apple silicon.</li> : <li><code>--spec-draft-n-max</code>: the most tokens the drafter proposes per step. llama.cpp clamps it to the drafter's block size.</li>}
<li><code>--spec-draft-n-min</code>: the fewest tokens the drafter proposes per step.</li>
<li><code>-ngl 99</code> / <code>-ngld 99</code>: offload all target and drafter layers to the GPU.</li>
<li><code>-fa on</code>: turns on flash attention.</li>
</ul>
</div>;

export const DsparkSglang = ({modelId, draftModelId, vision = false, samplingParams}) => <div>
<p><strong>Install:</strong></p>
{vision ? <p>Requires <a href="https://github.com/sgl-project/sglang/releases/tag/v0.5.19">SGLang v0.5.19</a> or newer.</p> : <p>Requires a build of SGLang with DSpark support for LFM2 and LFM2-MoE targets (<a href="https://github.com/sgl-project/sglang/pull/31041">PR #31041</a>).</p>}
<p><strong>Launch server with the drafter attached:</strong></p>
<CodeBlock language="bash">
{`python -m sglang.launch_server \\
    --model-path ${modelId} \\
    --speculative-algorithm DSPARK \\
    --speculative-draft-model-path ${draftModelId} \\
    --speculative-draft-attention-backend flashinfer \\${vision ? `
    --speculative-dspark-block-size 9 \\` : ""}
    --disable-radix-cache --mem-fraction-static ${vision ? "0.8" : "0.75"} --port 30000`}
</CodeBlock>
<p><strong>Query (OpenAI-compatible):</strong></p>
<CodeBlock language="python">
{`from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="None")

response = client.chat.completions.create(
    model="${modelId}",
    messages=[${vision ? `{
        "role": "user",
        "content": [
            {"type": "image_url", "image_url": {"url": "http://images.cocodataset.org/val2017/000000039769.jpg"}},
            {"type": "text", "text": "Describe what you see in this image."},
        ],
    }` : `{"role": "user", "content": "What is machine learning?"}`}],
    ${samplingParams}
)

print(response.choices[0].message.content)`}
</CodeBlock>
<p><strong>Flags:</strong></p>
<ul>
<li><code>--speculative-algorithm DSPARK</code>: turns on DSpark speculative decoding.</li>
<li><code>--speculative-draft-model-path</code>: the drafter repo on Hugging Face, or a local path.</li>
<li><code>--speculative-draft-attention-backend flashinfer</code>: the attention backend the drafter runs on.</li>
{vision ? <li><code>--speculative-dspark-block-size</code>: how many tokens the drafter proposes per step. Use 9, the block size the drafter was trained with.</li> : <li>Block size: read from the drafter's <code>config.json</code>, so you don't need to set it.</li>}
<li><code>--disable-radix-cache</code>: turns off SGLang's prefix cache, as in the drafter's model card.</li>
<li><code>--mem-fraction-static</code>: the share of GPU memory SGLang reserves for weights and the KV cache.</li>
</ul>
</div>;

Speculative decoding makes generation faster without changing what the model outputs. A small **draft model** proposes several tokens ahead, and the **target model** checks all of them in a single forward pass. The target keeps every token it agrees with and corrects the first one it does not.

## Available draft models

| Target model | DSpark Drafter (HF) | DSpark Drafter (GGUF) |
| - | - | - |
| **Text Models** | | |
| [LFM2.5-1.2B-Instruct](/lfm/models/lfm25-1.2b-instruct) | [✓](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct-DSpark) | [✓](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct-DSpark-GGUF) |
| [LFM2.5-2.6B](/lfm/models/lfm25-2.6b) | [✓](https://huggingface.co/LiquidAI/LFM2.5-2.6B-DSpark) | [✓](https://huggingface.co/LiquidAI/LFM2.5-2.6B-DSpark-GGUF) |
| [LFM2.5-8B-A1B](/lfm/models/lfm25-8b-a1b) | [✓](https://huggingface.co/LiquidAI/LFM2.5-8B-A1B-DSpark) | [✓](https://huggingface.co/LiquidAI/LFM2.5-8B-A1B-DSpark-GGUF) |
| **Vision Language Models** | | |
| [LFM2.5-VL-3B](/lfm/models/lfm25-vl-3b) | [✓](https://huggingface.co/LiquidAI/LFM2.5-VL-3B-DSpark) | [✓](https://huggingface.co/LiquidAI/LFM2.5-VL-3B-DSpark-GGUF) |

## Supported inference engines

| Engine | Text drafters | Vision drafters |
| - | - | - |
| [SGLang](/deployment/gpu-inference/sglang) | ✓ | ✓ |
| [llama.cpp](/deployment/on-device/llama-cpp) | ✓ | ✓ |
| [MLX-VLM](/deployment/on-device/mlx) | ✗ | ✓ |

## Run with text models

<Tabs>
  <Tab title="SGLang">
    <DsparkSglang modelId="LiquidAI/LFM2.5-2.6B" draftModelId="LiquidAI/LFM2.5-2.6B-DSpark" samplingParams={'temperature=0.1,\n    extra_body={"top_k": 50, "repetition_penalty": 1.1},'} />
  </Tab>

  <Tab title="llama.cpp">
    <DsparkLlamacpp ggufRepo="LiquidAI/LFM2.5-2.6B-GGUF" draftRepo="LiquidAI/LFM2.5-2.6B-DSpark-GGUF" samplingFlags="--temp 0.1 --top-k 50 --repeat-penalty 1.1" />
  </Tab>
</Tabs>

## Run with vision models

<Tabs>
  <Tab title="SGLang">
    <DsparkSglang modelId="LiquidAI/LFM2.5-VL-3B" draftModelId="LiquidAI/LFM2.5-VL-3B-DSpark" vision={true} samplingParams={'temperature=0.2,\n    extra_body={"top_k": 50, "repetition_penalty": 1.0},'} />
  </Tab>

  <Tab title="llama.cpp">
    <DsparkLlamacpp ggufRepo="LiquidAI/LFM2.5-VL-3B-GGUF" draftRepo="LiquidAI/LFM2.5-VL-3B-DSpark-GGUF" vision={true} samplingFlags="--temp 0.2 --top-k 50 --repeat-penalty 1.0" />
  </Tab>

  <Tab title="MLX-VLM">
    <DsparkMlxVlm modelId="LiquidAI/LFM2.5-VL-3B" draftModelId="LiquidAI/LFM2.5-VL-3B-DSpark" />
  </Tab>
</Tabs>

## FAQ

<AccordionGroup>
  <Accordion title="Does speculative decoding change the model's output?">
    No. The target model checks every drafted token and keeps only the ones it would have generated itself. With greedy decoding (`temperature=0`), the output is identical to running the target model alone. With sampling, the output follows the same distribution as the target model, as long as you use the same sampling settings.
  </Accordion>

  <Accordion title="What speculative decoding methods are supported for LFMs?">
    Only DSpark. Liquid AI publishes DSpark draft models for the models listed in [Available draft models](#available-draft-models). Other methods, such as DFlash, DFlash2, EAGLE, and multi-token prediction (MTP), are not supported. LFM models do not include MTP heads, so engine options such as vLLM's `--speculative-config` with `"method": "mtp"` do not work with them.
  </Accordion>

  <Accordion title="Why is the speedup smaller with quantized models?">
    DSpark gives the largest speedup with 16-bit (F16 or BF16) target models. With a quantized target, such as Q4, each verification step costs more, so the speedup is much smaller. On CPUs and Apple silicon, a quantized target may give little or no speedup.

    Quantizing the drafter has little effect on speed. For the best results, run a 16-bit target. If you need a quantized target to fit in memory, measure the speedup on your own hardware first.
  </Accordion>
</AccordionGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.