Strata: a 125B model on a 12GB card, measured

Strata LLM is an open source inference engine, MIT licensed, built in C++ by a developer posting as Niko1221. It runs a 125B mixture-of-experts model on a 12GB graphics card and 64GB of system memory. The project went up on September 24 and now sits at 40,100 stars, 4,400 forks, 349 contributors and 5,637 commits, with a release every couple of days and the current build at 0.2.10. That is not the profile of a demo repository.

The claim is real. The speed number printed next to it is where the honesty gets interesting.

*Caption: The project’s landing page: a 125B MoE model on a 12GB card, 25 to 38 tok/s claimed in the same row. Screenshot: stratallm.org.*

How it gets there

A mixture-of-experts model does not use all of itself to produce one token. Qwen3.8-Flash-Next fires somewhere between two and eight experts out of 64 per token. Strata works from that fact: attention heads and the most frequently triggered experts stay resident in video memory, and the dormant majority of the model sits in system RAM, pulled across PCIe only when the router asks for it. The project describes predicting which experts the next one or two tokens will need so the transfer is already in flight, and moving weights with pinned memory and asynchronous CUDA streams so the copy does not route through CPU compute.

*Caption: The split in one picture: what stays on the card, what waits in RAM, and the PCIe link that every offloaded token pays for. Figure: Tokenstead.*

The speed table, read carefully

The README carries measured numbers, taken at 2.00 bits per weight on engine versions 0.2.5 to 0.2.7. They are specific and they are modest:

RTX 5070, 12GB, 64GB RAM: 4 to 6 tokens/s output, about 110 tokens/s reading a 4K prompt
RTX 3060, 12GB, 46GB RAM: 9 to 11 tokens/s output, about 155 tokens/s prompt
One Radeon AI PRO R9700, 251GB RAM: 21 to 25 tokens/s output, 320 to 360 tokens/s prompt
Four R9700s: 74 to 77 tokens/s output
Ryzen AI Max+ 395 Strix Halo, 128GB unified: 19 to 21 tokens/s on code, 11 in chat

*Caption: The headline pairs the smallest supported card with the speed measured on larger ones. On an actual 12GB card the engine produces 4 to 11 tokens per second. Figure: Tokenstead.*

Set the landing page’s “25 to 38 tok/s” against that table and the numbers only coexist if you assume the faster card. The engine does what it says: a 125B model runs on 12GB of VRAM. It runs at four to eleven tokens per second when you do that, which is fine for an agent working through a queue overnight and painful in a chat window. The README is the honest document here, and it is the one to trust.

The DeepSeek claim that spread this week

A widely shared post said DeepSeek V4.1 Flash Coder now runs locally in Strata on 64GB of RAM plus a GPU. Strata does support the model, and its own table shows what that costs: 239GB at 1.75 bits per weight, 257GB at 2.00, 274GB at 2.25, or 191GB for the coder set with half the experts removed. The README states plainly that what does not fit in memory is read from the SSD, which is why the disk requirement sits just above the download size.

*Caption: DeepSeek V4.1 Flash in Strata, by quantization set, from the project’s own table. This is not a 64GB model; it is a 200GB model that can lean on an SSD. Figure: Tokenstead.*

So the claim is directionally true and materially misleading. The 64GB machine runs the model; it does not hold it. Every answer streams hundreds of gigabytes from storage, and the tokens-per-second figures for that path are not the ones in the table.

What it changes for the builds on this site

Nothing about the hardware tiers moves, and the reason is useful. Expert offloading turns memory capacity into time, so the tier ladder becomes a speed ladder for large models rather than a pass-or-fail line. A used 3090 with 64GB of DDR5, the value pick at $1,499, will now hold a 125B model at single-digit tokens per second. The $5k desk build with 96GB of unified memory does better on the models that fit it and worse on PCIe-bound streaming, because unified memory has no link to cross. The GPU rig with 96GB of video memory plus cache cards is the build where offloading stops mattering for the 100B class.

*Caption: 40,100 stars, 349 contributors, 64 releases, and a directory per platform including a DGX Spark build. Screenshot: GitHub.*

The second consequence is the one that matters most for the direction this site has been following. DeepSeek V4.1 Flash landed as the model teams route into subagents, and it needs hardware most people do not own. Strata is the first runtime that puts it on a desktop, even if the desktop is reading most of it off an SSD. A slow local copy of the model that does the bulk of your agent work is still a copy you own, and the cost of that ownership is measured in tokens per second, which is a number you can plan around.

The project is three weeks old, at version 0.2.10, with 710 open issues and another 617 open pull requests. Treat it as a fast-moving tool with real engineering behind it and a rough edge list to match, and check the README numbers rather than the headline before you buy a card for it.