Skip to main content

llama.cpp

Embed models in your app or run a local API across CPUs and GPUs.

Ollama

Get a local model API running with a few commands.

MLX

Build Python applications with GPU acceleration on Apple Silicon.

vLLM

Handle concurrent requests with continuous batching.

SGLang

Serve agents with tool calling and constrained JSON output.

Modal

Run GPU services that scale without managing servers.