The runtimes catalog on this site grew from 7 to 21 engines this week, every row re-verified against its own repository: stars, license, platforms, OpenAI-API surface, and whether it ships a UI. The catalog is the backbone of every “run locally” block on the model pages, and it was quietly missing most of the field: nothing for the GPU-serving pair that runs most multi-user labs, nothing for the NPU tier that Ryzen AI and Snapdragon laptops now ship, and nothing for the engines whose entire pitch is one file.
The count comes first: for all the “which local stack” threads, only two engines hold most of the reach. Ollama sits at 183,730 stars and llama.cpp at 131,617 - and llama.cpp is older, faster to new-model support, and underneath most of the list, including Ollama itself. Engines consolidate while models proliferate: 21 rows is most of the working field, where the model catalog crossed 110 this month.
Pick by hardware, not by hype
Three numbers decide which engine suits a machine: the memory the model needs, the bandwidth moving it each token (decode is bandwidth-bound), and whether OpenAI-compatible serving matters for the client code. The field sorts cleanly once those three are on the table.
-
GGUF on CPU plus small GPU stays the zero-friction tier: llama.cpp underneath, Ollama in front, LM Studio and Jan as desktops, llamafile and KoboldCpp when one file is the goal. Slower per token than everything to the right; runs on anything from a mini PC to a repurposed console board.
-
Apple unified memory decodes from shared LPDDR5X, and MLX leads it on tok/s. This is the tier the site’s Mac build rows and the Spark-cluster math come from.
-
Discrete-GPU serving is vLLM, SGLang, TensorRT-LLM and TGI: continuous batching makes eight GPUs pay off, radix caching makes agent loops cheap, and this is what most API endpoints actually run on.
-
NPU and hybrid is the newest tier: Lemonade serves Ryzen AI NPUs, Nexa SDK targets Qualcomm silicon, IPEX-LLM and OpenVINO carry Intel, and KTransformers splits a huge MoE between pinned experts in system RAM and hot ones on the GPU - the shape that runs frontier-sized MoEs on one workstation.
The honest notes
GPT4All is the field’s caution row: 77,347 stars, and no push since May 2025 - the biggest star count on the board belongs to a project that went quiet. text-generation-webui’s last push was August. Stars are adoption history, not maintenance status; the catalog’s repo-verified dates are on every row for exactly this reason. The other honest line: most of the NPU tier serves small-to-mid models well and 70B-class poorly today; the tier’s point is always-on efficiency, not frontier fits.
The field consolidates because decode speed is physics. Every engine reads the same weights the same way; what differentiates is the batching strategy, the cache, and the hardware targets. That is why the catalog works as a fixed 21-row table instead of a feed.
Sources: GitHub repos per engine (full repo links on each runtime row) - Ollama - llama.cpp - vLLM - SGLang - LocalAI - Lemonade - KTransformers
Related on this site: The budget rig under $3,000 is a repurposed console - Hermes Agent ships on NVIDIA’s RTX Spark launch - DeepSeek V4.1 Flash (model page)