Skip to content

Thinking Machines releases Inkling 2 Small weights

Published Nov 16, 2026

On 13 November at 21:12 UTC the repository for Inkling 2 Small changed status from promised to published. Thinking Machines Lab uploaded 24 safetensors shards to Hugging Face, 394GB in total, under Apache 2.0. The model card lists 594B total parameters with 17B active per token, and the download counter had reached 3,647 with 292 likes by the following morning.

Three consecutive weekly sweeps had listed this model as announced but not available. The difference now is not the benchmark table, it is that the files exist.

Vendor mark from the Inkling 2 Small model card. Image: Thinking Machines Lab.

Vendor mark from the Inkling 2 Small model card. Image: Thinking Machines Lab.

What it does differently

Each token is routed to 8 of 384 experts in a sparse mixture-of-experts backbone across 48 decoder layers. That routing is why a 594B parameter model needs 17B worth of computation per token: the weights are the memory problem, not the arithmetic.

Inputs are text and images, projected into a shared space and processed by the same decoder. The card gives 1M tokens of context, which puts it in the class where a repository fits in one request.

The precision decision is the part worth noticing. The model was pretrained, midtrained and post-trained in NVFP4, and Thinking Machines calls that the native precision rather than a compression step applied afterwards. They publish a second checkpoint in MXFP8 for people who want it, and describe its benchmark gain over NVFP4 as one to two points that they do not consider meaningful.

The parameters and numerics section of the model card. Screenshot: Hugging Face.

The parameters and numerics section of the model card. Screenshot: Hugging Face.

The hardware bill

The card is direct about this. The NVFP4 checkpoint needs at least 500GB of aggregated VRAM, which they illustrate as two NVIDIA B300 GPUs or four H200s. The MXFP8 checkpoint needs 800GB, illustrated as four B300s or eight H200s.

A machine with 512GB of unified memory is the only single box on our hardware list that comes close, and 500GB of card memory inside a 512GB machine leaves very little for the operating system and the KV cache. This is a datacenter model with an open licence, not a desktop one.

The repository listing shows 394GB across the shard set. Screenshot: Hugging Face.

The repository listing shows 394GB across the shard set. Screenshot: Hugging Face.

Recipes shipped the same day

Four inference stacks published instructions on release: SGLang, vLLM, TokenSpeed and Unsloth. The vLLM recipe quotes about 1,100 output tokens per second per GPU at 100 tokens per second per user, measured on a TP8 configuration with DSpark, which is a vendor benchmark rather than a number we reproduced.

The vLLM recipe page for Inkling 2 Small, including its throughput claim. Screenshot: vLLM recipes.

The vLLM recipe page for Inkling 2 Small, including its throughput claim. Screenshot: vLLM recipes.

The Unsloth guide is the more useful document for anyone with less than a cluster, because it is written around llama.cpp and GGUF rather than tensor parallelism. It repeats the 594B and 17B figures and points at quantised builds.

API access is available through NVIDIA, Fireworks, Together, Baseten, Nebius, Modal and Cloudflare for people who want the model without the memory.

Unsloth’s local inference guide, published the same day as the weights. Screenshot: Unsloth.

Unsloth’s local inference guide, published the same day as the weights. Screenshot: Unsloth.

The numbers on the card

Vendor-measured, with the best score in each row belonging to a different model in most cases:

  • Vals Index v2.2: 56.1% against 61.3% for the best model listed
  • Artificial Analysis Index v5.0: 30.6
  • Humanity’s Last Exam, text only: 44.1% against 49.4% best
  • AA-LCR v1.1: 83.3% against 88.7% best
  • DeepSWE v1.1: 72.1%
  • FrontierCode v2.0: 25.4%
  • Terminal-Bench Science v0.1: 5.7%

That last figure is the one to sit with. On terminal-based scientific work the model scores 5.7% where the best model in the table scores 4.8%, so both are close to the floor. A 594B open weight model is not automatically a strong agent.

What is not established

No independent reproduction of any benchmark has been published. The throughput figure is the vendor’s own. The download counts move hourly and will be stale by the time you read this. Nothing here speaks to whether the mixture-of-experts routing holds up after fine-tuning, which is the claim most buyers of an Apache 2.0 model actually care about.

What is established is the part that kept changing: the weights are downloadable today, in two precisions, with four inference recipes, at a licence that permits commercial use.

Sources: Hugging Face: Inkling 2 Small - vLLM recipe - Unsloth guide - SGLang recipe

Related on this site: Open weights after Huang

Tokenstead community

Discuss this analysis with people running it

The Discord is where readers compare notes on the models, rigs, and prices in this piece - and where the next guide often starts.

Something wrong or stale? Send a correction.

Discussion

Be the first to comment

Start a discussion

Got a take on this, a rig to show off, or a benchmark that says otherwise? Sign up and start the thread - your comment publishes instantly once you're in.