Amazon replaced Sonnet 4.6 with a 4B decision model

TLDR: Two Amazon engineers swapped Claude Sonnet 4.6 for Strands Decider, an open 4B decision model, in one of the company’s largest internal ticketing systems, and report the same labelling quality at 98 percent less cost. It enables a category of work that has been paying frontier-model prices for a job that never needed sentence generation. The difference between this and a cheaper language model is that a decision model cannot write anything at all: it returns a probability for each option in a fixed list, plus a confidence score.

*Caption: The write-up from the two engineers who run the system, published November 9, 2026. Their words: “performed just as well at 98% less cost”. Screenshot: Strands Agents.*

What was actually running

The system labelled millions of internal tickets into 20 category types: customer escalations, capacity requests, automated alarms, security findings. An agent sent every ticket through Sonnet 4.6 with a long system prompt and got back JSON with a label and reasoning.

That is a classification problem wearing a chat model’s clothes. The output was always one of twenty known labels, but the model was re-reading a prompt longer than the ticket itself and generating an explanation nobody stored.

The swap, and the numbers

They started with the 2B Decider and moved to the E4B Gemma 4 build, which did better on a held-out set of tickets. Then they re-tuned the confidence threshold at which a label gets applied.

Caption: The measured comparison on the same tickets. Time per call falls 78 percent, total tokens per ticket fall 70 percent, and the output is a single choice instead of a JSON object. Screenshot: Strands Agents.

*Caption: The three numbers that moved. Cost per million tickets drops from about $22,200 to about $500. Figure: Tokenstead, from the published tables.*

Caption: The cost table, at published us-east-1 on-demand prices. One A10G at about 640 ms per request handles roughly 5,600 tickets an hour, which is how the per-ticket number gets to half a tenth of a cent. Screenshot: Strands Agents.

The arithmetic is worth sitting with. At $3 per million input tokens and $15 per million output, a 6,724-token ticket costs 2.2 cents. At $1.408 for an ml.g5.xlarge instance-hour, the same ticket costs about $0.0005. That is 44 times cheaper.

How they de-risked it

They did not flip a switch. The labelling Lambda reads a three-way setting from AWS AppConfig on every message:

BEDROCK: Sonnet labels every ticket, Decider is never called.
DUAL: both run in parallel, Sonnet’s answer is applied, and every disagreement is logged.
DECIDER: Decider labels everything, Bedrock is not called.

Rolling back was a 45-second config change. Running both on live traffic, then reading every disagreement, told them more than any offline evaluation would have.

The deployment itself is ordinary: ticket events land in SQS, a Lambda reads the ticket and calls a SageMaker endpoint, CloudWatch holds the logs, the stack is defined in CDK with Beta, Gamma and Prod stages. The endpoint’s network is isolated with no outbound access, and weights are copied in from S3 at start-up.

The part that will bite anyone copying this

Two constraints, both from the write-up.

Label descriptions have to fit a 4,096-token window. Production’s label descriptions alone were longer than that. The engineers rewrote all 20 labels as one- or two-sentence criteria, sent them as the options of a single choice question, and added a “none” option. On long tickets, the model drops the oldest comments to fit.

Key order changes the answer. “Sending the same content with keys in a different order moved the output probabilities by up to 0.19.” They now sort keys on every request. If you serve a decision model behind an API and compare runs without sorting, you are measuring your serializer.

Caption: The October launch post that introduced the class: a 2B model that answers from a fixed set of options and returns a confidence score, released with weights, training data and scripts. Screenshot: Strands Agents.

When this does not work

The engineers name the boundaries themselves, and they are narrow:

The task must be a closed choice from a known catalogue.
The volume must be high.
The latency of the old path must matter.
The outputs must not need explanation text.
The catalogue must not change weekly.
The useful context must fit into a few thousand tokens.

Break any of those and you are back to a language model, and the likely explanation is that you were paying for reasoning you were not using.

Why this matters beyond Amazon

Decision models became a recognised class only this autumn, after TypeSafe AI’s Jev, and the pitch has been mostly about latency. This is the first production account with token counts, instance hours, a rollback plan and a failure mode that includes the words “keys in a different order”.

The lesson for anyone running a classification agent on a frontier API is not that a 4B model is magic. It is that the frontier model was doing two jobs, and only one of them was ever necessary. Strip out the generation, keep the decision, and the bill for a million tickets goes from a salary to a rounding error.

*Caption: The weights are open: amazon/strands-decider-E4B-gemma4-v1-2610 on Hugging Face, a Gemma 4 E4B fine-tune. Anyone can run the same experiment on one 24GB card. Screenshot: Hugging Face.*

*Caption: The thread where most people found it, 336 points on the day. The top comments argue about whether this is a model story or a “we stopped over-engineering” story. Screenshot: Hacker News.*

If you have a phone, a 24GB card, or a spare A10G, the experiment is a weekend: export a thousand labelled examples, build a choice question with the labels as options, and see whether a 4B decision model with a confidence threshold beats the prompt you are paying for.