GLM-5.3-Flash, the 320B-A18B MoE under the MIT license, has been local-able since launch but slow at the context lengths people actually run agents at. On September 4, Unsloth’s GLM-5.3-Flash guide shipped an update that closes a chunk of that gap: multi-token prediction (MTP) support in llama.cpp, with measured decode throughput up to 3.3x faster at long context. The work builds on the day-zero llama.cpp support PR (ggml-org/llama.cpp#27754) and lands through Unsloth’s fork branch glm5next/upstream.
Unsloth’s measured decode throughput (tg32) for GLM-5.3-Flash GGUF UD-IQ1_S on one B200. The optimized decoding path closes most of the gap by itself; MTP n=2 adds more on top where published.
What the numbers say
Unsloth’s table, one B200, GGUF UD-IQ1_S (93.09 GB), tg32 decode:
-
4,096-token context: 53.5 tok/s baseline, 59.5 with the optimized decoding path, 86.5 with MTP at n=2 draft tokens. That is 1.48x over baseline.
-
16,384-token context: 41.0 baseline, 58.0 optimized. 1.41x. MTP numbers were not published at this context length.
-
65,536-token context: 20.7 baseline, 49.0 optimized. 2.37x, without MTP counted. The headline 3.3x sits at the longest contexts Unsloth measured.
Two things stand out. First, the optimized decode path alone carries most of the long-context gain, so the update is worth taking even before you touch MTP. Second, the gain grows with context: at 4K you get roughly 1.5x, at 64K more than 2x. That matches how MoE decode works - long context is where the per-token cost concentrates, so the same kernel work buys more.
This also lines up with what NVIDIA published two days earlier about speculative decoding generally: draft length is not free, and “higher AL does not equal higher speedup.” Unsloth found the same wall empirically. Their guidance: stop at around n=2 draft tokens, because more drafts make inference slower. GLM-5.3-Flash’s MTP module was trained into the checkpoint at pretraining time, which per NVIDIA’s benchmarking is exactly the mechanism that suits large models on big GPUs.
What you need to run it
The quant ladder is the gate. GLM-5.3-Flash is a 320B-parameter model, so RAM plus VRAM needs per Unsloth’s GGUF release:
- 1-bit (UD-IQ1_S and friends): roughly 100 GB
- 2-bit: roughly 115 GB
- 3-bit: 128 to 150 GB
- 4-bit: 162 to 210 GB
Unsloth’s demos used 3-bit UD-IQ3_XXS (120.37 GB) to fit 128 GB machines, which is also the memory class of a DGX Spark or a 128 GB Mac. The community numbers on exactly that class are real but smaller than the B200 table: the Weschera/glm53-flash-one-spark project measured one DGX Spark going from 17.9 tok/s to 25 to 34 tok/s after adopting the same work, roughly 1.4x to 1.9x. A 128 GB machine is running a 1-bit or low-3-bit quant, and the speedup compresses as the quant thins out and more time goes to dequantizing.
Setup paths, in order of friction:
-
Unsloth Desktop: update to the latest version. MTP works out of the box with no extra files.
-
Ollama / LM Studio / stock llama.cpp: the underlying llama.cpp PR is merged day-zero; Unsloth’s fork branch
glm5next/upstream (built with -DGGML_CUDA=ON) carries the optimized MTP path if stock builds trail it.
-
Build the fork yourself: clone, checkout the branch, build, point it at the Unsloth GGUF.
One early wrinkle showed up in the Hugging Face discussion: some builds hit an MTP load error (“nextm isn’t implemented”) before the fixes landed in the PR thread. If you pulled a build in the first days, update before concluding MTP does not work on your rig.