Skip to main content
Speculative decoding makes generation faster without changing what the model outputs. A small draft model proposes several tokens ahead, and the target model checks all of them in a single forward pass. The target keeps every token it agrees with and corrects the first one it does not.

​
Available draft models

​
Supported inference engines

​
Run with text models

​
Run with vision models

​
FAQ

No. The target model checks every drafted token and keeps only the ones it would have generated itself. With greedy decoding (temperature=0), the output is identical to running the target model alone. With sampling, the output follows the same distribution as the target model, as long as you use the same sampling settings.
Only DSpark. Liquid AI publishes DSpark draft models for the models listed in Available draft models. Other methods, such as DFlash, DFlash2, EAGLE, and multi-token prediction (MTP), are not supported. LFM models do not include MTP heads, so engine options such as vLLM’s --speculative-config with "method": "mtp" do not work with them.
DSpark gives the largest speedup with 16-bit (F16 or BF16) target models. With a quantized target, such as Q4, each verification step costs more, so the speedup is much smaller. On CPUs and Apple silicon, a quantized target may give little or no speedup.Quantizing the drafter has little effect on speed. For the best results, run a 16-bit target. If you need a quantized target to fit in memory, measure the speedup on your own hardware first.