Routing a request by its stakes only works if the router can see the stakes

Victoria Shinder26 October 2026, 10:203 min read

Cloudflare put a model router into public beta this month: set the model to cloudflare/auto and a classifier picks one per request, scoring it one to five on complexity, ambiguity, stakes and dependence on earlier context, then combining that with benchmark numbers under a plain formula — utility is expected quality minus an adaptive cost penalty. The most disarming thing in the announcement is that Cloudflare published a benchmark in which its own router loses to a strong fixed model. That honesty is worth the price of admission, and it points straight at the design's real weakness.

The classifier reads only the newest turns, capped at eight kilobytes. A tech lead who has built banking and healthcare platforms named the problem in one sentence: in a long session a user says "this is a customer refund" early and, many turns later, just says "go ahead." The instruction that commits the money is the shortest, blandest turn in the conversation — and on a recent-turns view it looks cheap, low-stakes, safe to send to the small model. The turn that does the damage is precisely the one the router is least equipped to recognise.

His proposed fix is the right shape, and it is not Cloudflare's to make: carry an application-owned risk label across the whole session, set when the refund is first named, and use it to decide whether a request may touch cloudflare/auto at all. A short final instruction must not silently lower the risk class of the action underneath it. That is a statement about where responsibility sits. The router can estimate stakes from text; only the application knows that this conversation is authorised to move money, and that knowledge cannot be reconstructed from the last eight kilobytes.

There are smaller traps worth naming. The routing weights come from benchmark results, so a new model does not route well until someone adds it to the scoring matrix by hand — the classifier is only as current as its last calibration, which is an uncomfortable property in a market where models ship monthly. Without a session ID, requests in one conversation can land on different models and lose the prompt cache, and Cloudflare adds a switching penalty that grows with context because a new model may redo reasoning at output prices. And the default pool is not the pool the headline benchmark used; the two baselines that beat the router sit in an opt-in list, so a team reproducing the numbers should confirm what its gateway actually selects.

None of this makes automatic routing a bad idea. The unit that matters is the task, not the request — a cheaper model is no saving if someone spends an hour fixing its output — and a router that scores difficulty and prices it is a reasonable way to chase that. But the honest reading of this launch is that routing by stakes moves the hard problem rather than solving it. You still have to decide what is at stake, and the one place that decision can be made correctly is above the router, in the application that knows what the conversation is for.