# Prompt caching

A provider that caches a prompt bills the repeated start of your next request at its cached-input price, a fraction of its input price. Most providers cache on their own. Claude caches only where your request marks it, and llm.buzz never adds a mark for you.

## Which providers cache

- **Claude models** (`anthropic/...`) cache a prompt only where the request carries a cache marker, on every provider. A request without one, such as a plain OpenAI SDK or Codex call, pays the full input price on every turn.
- **OpenAI models, and most other models**, cache a repeated prefix on their own, without a marker. Some providers do not cache at all.

`GET https://api.llm.buzz/api/v1/models/{author}/{slug}/endpoints` says what each provider of a model does. `pricing.input_cache_read` is its cached-input price, and `supports_implicit_caching` is `true` when it caches a request without markers at a lower price. The [Models page](/models) marks the models that cache.

## Mark a prompt for Claude

From the OpenAI SDK, send Anthropic's `cache_control` at the top level of the request. The provider puts the cache breakpoint on the last block it can cache, so the breakpoint moves with the conversation and each turn reads what the turn before it wrote. On Chat Completions, a marker on the last function tool also caches the tool definitions.

```python tab="OpenAI SDK"
reply = client.chat.completions.create(
    model="anthropic/claude-sonnet-5.5",
    messages=messages,
    extra_body={"cache_control": {"type": "ephemeral"}},
)
```

```python tab="Anthropic SDK"
message = client.messages.create(
    model="anthropic/claude-sonnet-5.5",
    max_tokens=1024,
    system=[{"type": "text", "text": instructions, "cache_control": {"type": "ephemeral"}}],
    messages=messages,
)
```

A marker is `{"type": "ephemeral"}`, with an optional `ttl` of `"5m"`, the default, or `"1h"`. `null` carries nothing, and any other value is a 400 that names where it was, such as `cache_control` or `tools.1.cache_control`.

| Surface | Where a marker goes |
|---|---|
| Chat Completions | The top level, function tools, messages, text content parts and assistant tool calls. A marker on a server tool, such as `openrouter:tool_search`, is a 400. |
| Responses | The top level only. A marker on a tool is a 400. |
| Messages | Every place the Anthropic Messages API takes one, as the Anthropic SDK sends it. |

A marker reaches every provider of the model that takes Claude's markers. A provider that caches on its own, such as OpenAI, receives none. `x-sgr-ignored-parameters` names a marker that stayed behind, as `cache_control->not_forwarded(provider_decides_caching; ...)`: a top-level or message marker when no provider of the model can take it, and a tool marker when any provider cannot.

Claude takes at most 4 breakpoints in a request, and the top-level marker counts as one unless the last block of the last message already carries a marker. A one-hour breakpoint must come before every five-minute one, in the order the provider reads them: tools, then system, then messages, then the top-level marker. A provider that refuses such a layout is skipped and the model's other providers serve the request; when every provider refuses it, the answer is a 400 `invalid_parameter` with `param: cache_control` (Anthropic's `invalid_request_error` on Messages), and its message says what to remove or move.

On Claude Sonnet 5.5, whose cached input costs a twentieth of its input, a 40-turn session that grows from 20,000 to 120,000 tokens pays about $5.60 for its input without a marker, and under $1 with one top-level marker.

## What a cached read costs

Each provider lists its cache prices beside its input and output prices, in US dollars per token on the endpoints route and per million tokens on the model pages:

| Price | On the endpoints route | What it bills |
|---|---|---|
| Cached input | `pricing.input_cache_read` | Input read from the cache. |
| Cache write | `pricing.input_cache_write` | Input written to the cache for five minutes. |
| One-hour cache write | `pricing.input_cache_write_1h` | Input written to the cache with `ttl: "1h"`. |

A request bills its cached input at the cached price, the input written to the cache at the write price, and the rest of its input at the input price, and `usage.cost` includes all three. Claude Sonnet 5.5, for example, lists $2 per million input tokens, $0.10 cached, $2.50 for a five-minute write and $4 for a one-hour write.

## Read the cache in usage

| Surface | Fields |
|---|---|
| Chat Completions | `usage.prompt_tokens_details.cached_tokens` (read from the cache), `cache_write_tokens` (written to it) and `cache_write_1h_tokens` (the one-hour part of the writes) |
| Responses | `usage.input_tokens_details.cached_tokens`, `cache_write_tokens` and `cache_write_1h_tokens` |
| Messages | Anthropic's `cache_read_input_tokens`, `cache_creation_input_tokens` and `cache_creation`, its split of the writes into five-minute and one-hour; `input_tokens` counts only the input that was not cached |

On Chat Completions and Responses, the reads and the writes are separate parts of the prompt's token count. `cache_write_1h_tokens` appears only when the provider reports the split. Chat Completions shows a field only when the provider reported it; Responses always shows `cached_tokens` and `cache_write_tokens`, with `0` when the provider reported none.

## Keep a conversation on its cache

A cache lives at one provider. Name the conversation with `session_id` in the body or the `x-session-id` header, on every turn, and its turns stay on the provider that holds its warm cache, ahead of a cheaper or faster provider. On Chat Completions and Responses, `prompt_cache_key` names it too. A client that sends none of these still gets a session from the conversation's opening: its first system text and first message.

The conversation moves when that provider fails or your request can no longer use it, such as a `provider.only` that leaves it out, and a `provider.order` in the request turns this off.

Providers keep their caches per account, and llm.buzz calls each provider from its own account for every organization. So two organizations that send the same prompt prefix to the same provider can share its cache entry, and how fast an answer comes back can show that someone sent that prefix recently.

## OpenAI's explicit caching

OpenAI's GPT-5.6 and later models, such as `openai/gpt-5.6-luna` and `openai/gpt-6-sol`, take OpenAI's explicit caching controls on Chat Completions and Responses, as the OpenAI SDK sends them:

```python
reply = client.responses.create(
    model="openai/gpt-5.6-luna",
    input=[{"role": "user", "content": [
        {"type": "input_text", "text": long_context, "prompt_cache_breakpoint": {"mode": "explicit"}},
        {"type": "input_text", "text": question},
    ]}],
    extra_body={"prompt_cache_options": {"mode": "explicit"}},
)
```

- `prompt_cache_breakpoint: {"mode": "explicit"}` marks the end of a prefix to reuse, on a text part: on Chat Completions a `text` part of a system, developer, user or tool message, and on Responses an `input_text` part of a system, developer or user message or of a `function_call_output`. On any other part, or on an assistant message, it is a 400 that names where it was.
- `prompt_cache_options` takes `mode`, `implicit` (OpenAI's default) or `explicit` (OpenAI then writes the cache only at breakpoints), and `ttl` `30m`.
- `prompt_cache_retention` is `in_memory` or `24h`. `24h` keeps prefixes for up to 24 hours, so on a request with `provider.zdr: true`, `provider.data_collection: "deny"` or your organization's no-retention policy it is a 400 `invalid_parameter` on `prompt_cache_retention`. OpenAI may refuse `in_memory` on GPT-5.5 and later, so on such a request to those models send no retention at all.

OpenAI receives these controls as you send them. Sent to a Claude model, a breakpoint becomes a five-minute Claude marker, named in `x-sgr-ignored-parameters` as `prompt_cache_breakpoint->translated(cache_control)`, and the options and retention are not sent. A provider that cannot take them receives none of them, and when no provider of the model can, `x-sgr-ignored-parameters` names them. Your `cache_control` never becomes an OpenAI breakpoint, so you never pay for an OpenAI cache write you did not ask for. OpenAI reports its writes in `cache_write_tokens`, billed at the five-minute write price.