OpenAI widens prompt caching for GPT-6 and adds miss diagnostics

Serguey Asael Shinder23 September 2026, 08:062 min read

OpenAI has published changes to prompt caching for the GPT-6 family, dated 22 September 2026. Cached input tokens carry discounts of up to 90%, and the company says the new system delivers higher cache hit rates by default, with discounts applying to eligible shared prefixes reused within a 30-minute window.

The reason given is agents. A persistent agent working for hours makes a series of API requests that build on one another, carrying forward the same instructions, tool definitions and earlier context; caching that shared prefix reuses the computation rather than repeating it.

Two developer-facing tools accompany the change. A Prompt Caching Dashboard shows how much of an application's input is served from cache, tracks hit rates over time and compares cached against uncached tokens. A diagnostics tool explains an unexpected miss by comparing a request with a recent response and naming what changed — the payload includes a reason field with values such as tools_changed, and an estimate of the affected token count.

GitHub's chief product officer Mario Rodriguez is quoted saying that over recent months the share of prompt tokens requiring fresh processing across billions of requests to OpenAI models fell by more than 50% relative to their previous baseline.

What it means

The diagnostics tool is the more interesting half, because the failure mode of prompt caching has always been silent. A cache miss costs money and latency and looks exactly like a normal request; teams discover the problem in a bill rather than in a trace. Naming the cause — a changed tool definition, a modified setting — turns a mysterious cost into a code review comment.

tools_changed as a named reason is worth dwelling on. It says that editing a tool schema invalidates the prefix for everything after it, which makes tool definitions a part of the prompt with cost consequences, not just an interface. Applications that build tool lists dynamically per request may be paying for that pattern without knowing it.

One number to hold carefully: the 50% figure is a customer's account of their own workload over months, quoted by the vendor. It is a plausible order of magnitude, not a benchmark, and the only figure that describes your application is the one on the new dashboard.