llama.cpp
Embed models in your app or run a local API across CPUs and GPUs.
Ollama
Get a local model API running with a few commands.
MLX
Build Python applications with GPU acceleration on Apple Silicon.
vLLM
Handle concurrent requests with continuous batching.
SGLang
Serve agents with tool calling and constrained JSON output.
Modal
Run GPU services that scale without managing servers.