Microsoft-Decision-1: a 9B scoring model that returns probabilities, not prose

Victoria Shinder11 October 2026, 09:392 min read

Microsoft introduced Microsoft-Decision-1 on 10 October, a model built for one narrow job: given a fixed set of options, return a calibrated probability for each in a single pass. It is available in Microsoft Foundry and through OpenRouter, and Microsoft prices it at $0.042 per million input tokens, with output tokens free — which makes sense for a model whose output is a handful of numbers.

What it is underneath. The post says the model is Qwen3.5-9B, post-trained for fast single-pass decision scoring, and that Microsoft plans to rebase it on other models, including its own MAI models and OpenAI's. Supported formats are yes/no, multiple choice, ratings, and rubric-based grading of AI responses and agent actions, all through a structured API call.

Why a separate model. Microsoft's argument is latency arithmetic in agent workflows: if each of 20 sequential decisions adds 100 milliseconds, the workflow gains two seconds. Routing a request, deciding whether an agent should continue, retry or hand off, labelling feedback, judging an answer against a rubric — these produce a choice, not text, and a generative model spends time writing what a classifier could score.

The claims, and whose they are. Every figure below is from Microsoft's own evaluation, published with the launch:

• Highest accuracy in a 36-benchmark comparison covering nearly 150,000 questions held out from training, against public models from the JevBench leaderboard.
• The fastest model Microsoft measured: 2.5× quicker than H2O-Lightning-4B v1.1, the runner-up, with median latency about 35× lower than GPT-6 Sol.
• On consistency, eight kinds of perturbation of the same request (paraphrase, reordering, renamed keys, formatting noise) flip the decision 1.3% of the time on average, with no flips when option descriptions are paraphrased or options are reversed or shuffled.
• On safety, 5,250 requests across 11 benchmarks of harmful content, jailbreaks and prompt injection, with no single score given in the post.

Internal uses quoted: Xbox Research sorted more than 10,000 pieces of player feedback into fixed themes at quality Microsoft calls competitive with GPT-6 Sol, over 14× faster and 200× cheaper; the Copilot team found it competitive with GPT-5.6 Luna for grading responses at 100× the speed.

How to read it. Calibration is the property that matters most here, because applications use the probability to decide when to act and when to ask a human — Microsoft's own example is that a 90% score should be right about nine times in ten. The post shows calibration charts, but no independent evaluation exists yet. Anyone wiring this into an agent's stop-or-continue step should measure calibration on their own labelled cases before trusting the threshold. The post also lists six competing decision models by name, which is a useful starting set for that comparison.

Source: Microsoft Command Line, Introducing Microsoft-Decision-1, our model for fast decision-making, 10 October 2026 — https://commandline.microsoft.com/microsoft-decision-1-model-foundry/