# Responses

Responses is OpenAI's API of items in and items out, with the conversation's state kept by llm.buzz, so a follow-up sends only its own input. llm.buzz serves it at `POST https://api.llm.buzz/api/v1/responses` for every text model, and the OpenAI SDK and Codex call it with the base URL changed.

## Send a request

`input` is a string or a list of items, and `instructions` is the system instruction for this request. The OpenAI SDKs give the reply text as `output_text`.

```python tab="Python"
from openai import OpenAI
import os

client = OpenAI(
    base_url="https://api.llm.buzz/api/v1",
    api_key=os.environ["LLM_BUZZ_KEY"],
)
response = client.responses.create(
    model="openai/gpt-5.6-luna",
    instructions="You answer in one short sentence.",
    input="Why do HTTP clients retry with backoff?",
)
print(response.output_text)
```

```ts tab="TypeScript"
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llm.buzz/api/v1",
  apiKey: process.env.LLM_BUZZ_KEY,
});
const response = await client.responses.create({
  model: "openai/gpt-5.6-luna",
  instructions: "You answer in one short sentence.",
  input: "Why do HTTP clients retry with backoff?",
});
console.log(response.output_text);
```

```bash tab="curl"
curl https://api.llm.buzz/api/v1/responses \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.6-luna",
    "instructions": "You answer in one short sentence.",
    "input": "Why do HTTP clients retry with backoff?"
  }'
```

The answer is OpenAI's response object with the additions [below](#the-answer). The model name, and the models you can call, come from `GET https://api.llm.buzz/api/v1/models` ([Choosing an API](choosing-an-api.md#find-a-model-for-an-api)).

## Continue a conversation

llm.buzz keeps each finished turn. Send `previous_response_id` with the `id` of the last answer and only this turn's `input`, and llm.buzz adds the turns before it.

```python tab="Python"
first = client.responses.create(
    model="openai/gpt-5.6-luna",
    instructions="You answer in one short sentence.",
    input="Which port does HTTPS use?",
)
second = client.responses.create(
    model="openai/gpt-5.6-luna",
    instructions="You answer in one short sentence.",
    previous_response_id=first.id,
    input="And plain HTTP?",
)
print(second.output_text)
```

```ts tab="TypeScript"
const first = await client.responses.create({
  model: "openai/gpt-5.6-luna",
  instructions: "You answer in one short sentence.",
  input: "Which port does HTTPS use?",
});
const second = await client.responses.create({
  model: "openai/gpt-5.6-luna",
  instructions: "You answer in one short sentence.",
  previous_response_id: first.id,
  input: "And plain HTTP?",
});
console.log(second.output_text);
```

```bash tab="curl"
# RESPONSE_ID is the "id" of the first answer
curl https://api.llm.buzz/api/v1/responses \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.6-luna",
    "instructions": "You answer in one short sentence.",
    "previous_response_id": "'"$RESPONSE_ID"'",
    "input": "And plain HTTP?"
  }'
```

- A turn is kept for up to 24 hours. It is not a prompt log: the console does not show it, and it is kept even when your organization turned prompt logging off or the request demanded zero retention ([Models and routing](models-and-routing.md#data-policy)). Send `store: false` to keep a turn out.
- A request that continues a turn llm.buzz no longer has answers 400 `previous_response_not_found` with `param: previous_response_id`, and its message asks you to resend the whole conversation in `input`. A turn is gone after 24 hours, when it was sent with `store: false`, and sooner if llm.buzz's store of turns restarts or fills up. The Codex client resends by itself.
- When llm.buzz cannot reach its store of turns, a continuation answers 503 `continuation_store_unavailable`. Wait `Retry-After` seconds and retry. A turn that finishes while the store is unreachable is answered but not kept, so it cannot be continued later.
- Each turn follows only its own `instructions`. Send them again on every turn that should follow them.
- A kept turn belongs to the organization, key identity and model that made it. A continuation that names another model answers 400 `previous_response_not_found`, so continue with the same model.
- A continuation can stay on the provider that served the turn it continues. While that provider can still be used, `provider.order` and the routing preset do not move it, and `only` and `ignore` do not narrow it, so send `provider` preferences with the first turn. A continuation also keeps the caching session its first turn resolved, unless it sends its own `session_id`, `x-session-id` or `prompt_cache_key` ([Prompt caching](prompt-caching.md#keep-a-conversation-on-its-cache)).

## Input items

`input` is a string, which counts as one user message, or a list of items:

- Messages, `{"role": "user", "content": ...}`, with the roles `user`, `assistant`, `system` and `developer`. A `developer` or `system` item is accepted for every model. A leading one is the model's instruction. A later one stays in place when the provider takes one there, and otherwise moves into the user text at the same place, named in the body's `x-sgr-ignored-parameters`, for example `messages.system->folded(system_messages_leading_only)`.
- `function_call` and `function_call_output` items carry a tool loop. Tool call IDs are opaque strings of 1 to 65,536 characters: send each one back complete and unchanged, in the call and in its output. A `function_call_output`'s `output` is a string or a list of text and image parts, and any other part kind is a 400 that names it. [Tools and structured output](tools-and-structured-output.md) covers the loop.
- `reasoning` items from an earlier answer, sent back as they came. Ask for `include: ["reasoning.encrypted_content"]` and a provider that supports it returns the model's reasoning encrypted in its reasoning items. Send those items back with the next request to keep the model's thinking across tool calls in a conversation sent with `store: false`. An encrypted item is tied to the provider account that produced it. When a provider refuses one you sent back, llm.buzz retries without the encrypted reasoning items, keeps the rest of the conversation, and sets the response header `x-gateway-replay-repair: encrypted_reasoning_stripped`.

## Request fields

A field llm.buzz deliberately refuses is a 400 before any provider is called, and a value that fails validation is a 400 `invalid_parameter` with `metadata.param` naming the field. A field llm.buzz drops, such as `messages` or `stop`, which Responses does not know, is named in the body's `x-sgr-ignored-parameters`.

| Field | What llm.buzz does |
|---|---|
| `model` | The model's name, `author/slug`. Four suffixes change one request: `:floor` tries the cheapest provider first, `:nitro` the fastest, `:exacto` is accepted and ignored, and `:online` is accepted and searches nothing (see the `web` plugin below). [Models and routing](models-and-routing.md#suffixes). |
| `input`, `instructions` | `input` is required: a request without it is 400 `invalid_parameter`. [Input items](#input-items). `instructions` is the system instruction of this request alone; an empty or whitespace-only value counts as absent. |
| `previous_response_id`, `store` | `previous_response_id` continues a kept turn: [Continue a conversation](#continue-a-conversation). `store: false` keeps this turn out, so nothing can continue it. |
| `max_output_tokens` | The most output tokens, reasoning included. Without one, llm.buzz does not choose a smaller limit for you. A provider that needs a larger minimum than your limit is skipped, and when none is left the request is refused. |
| `temperature`, `top_p` | Sent to the provider; an out-of-range `temperature` is 400 `invalid_parameter`. A model that takes none, or takes them only at reasoning effort `none`, has the value dropped and named: [Sampling settings a model does not take](api-reference.md#sampling-settings-a-model-does-not-take). |
| `top_k` | Sent to providers that take it. Dropped and named as `top_k->dropped(unsupported_by_provider)` when none does. |
| `seed`, `logit_bias` | Dropped and named. Chat Completions refuses them; Responses does not. |
| `stop`, `n`, `frequency_penalty`, `presence_penalty`, `response_format`, `stream_options`, `audio`, `modalities`, `functions`, `function_call`, `prediction` | Not fields of the Responses API, so dropped and named. Chat Completions refuses `audio`, `modalities`, `functions`, `function_call` and `prediction`; Responses does not. |
| `stream` | `true` streams the answer as server-sent events: [Stream](#stream). |
| `tools`, `tool_choice` | Sent, in OpenAI's shape. Some models cannot be forced to call a tool, and some providers take a tool schema only in part. [Tools and structured output](tools-and-structured-output.md) says what happens to each. |
| `parallel_tool_calls` | Honored on every provider. Where a provider has no such control, `true` is its default behavior and is named `parallel_tool_calls->dropped(provider_default)`, and `false` is enforced by llm.buzz, which returns only the first tool call of a turn and names it `parallel_tool_calls->emulated(serialized_by_gateway)`. |
| `text` | `text.format` with `type: "json_schema"` asks for JSON that matches your schema: [Tools and structured output](tools-and-structured-output.md). `text.format.type` accepts only `text` and `json_schema`; any other, `json_object` included, is 400 `invalid_parameter` naming `text.format.type`. `text.verbosity` is a hint, sent to providers that take it and dropped and named elsewhere. |
| `include` | `reasoning.encrypted_content` returns encrypted reasoning ([Input items](#input-items)). `message.output_text.logprobs`, with `top_logprobs`, asks for token probabilities: llm.buzz sends both to providers that serve the native Responses API, and the provider returns, ignores or refuses them. Any other selector is 400 `invalid_parameter` naming `include`, the six that would echo server-tool output or an input image among them: `code_interpreter_call.outputs`, `computer_call_output.output.image_url`, `file_search_call.results`, `message.input_image.image_url`, `web_search_call.action.sources` and `web_search_call.results`. |
| `reasoning` | `effort`, `summary`, `generate_summary` and `context` are accepted. If you send both `summary` and `generate_summary` they must match, or the request is a 400 `invalid_parameter` naming `reasoning`. `effort` is translated into each provider's own terms ([Reasoning](reasoning.md)). Without a `summary`, llm.buzz asks OpenAI for an automatic one unless the effort is `none`. `reasoning.mode` is 400 `invalid_parameter`. |
| `truncation` | Only `"disabled"`. Any other value is 400 `invalid_parameter`. |
| `client_metadata` | Sent to providers that serve the native Responses API and dropped and named elsewhere. |
| `metadata` | Echoed on the response object: [The answer](#the-answer). |
| `cache_control` | Anthropic's prompt-cache marker, `{"type": "ephemeral"}` with an optional `ttl` of `"5m"` or `"1h"`, at the top level only. A marker on a tool is a 400. Passed on or translated, never added by llm.buzz. [Prompt caching](prompt-caching.md). |
| `prompt_cache_breakpoint` | OpenAI's explicit cache mark, `{"mode": "explicit"}`, on an `input_text` part of a `system`, `developer` or `user` message item or of a `function_call_output`. Sent to OpenAI as given. For Claude it becomes a five-minute `cache_control` marker, named `prompt_cache_breakpoint->translated(cache_control)`. On any other part, or on an assistant message, it is a 400 that names where it was. [Prompt caching](prompt-caching.md#openais-explicit-caching). |
| `prompt_cache_options`, `prompt_cache_retention` | OpenAI's cache mode and retention. Sent to OpenAI only; no other provider receives them, and where no provider of the model takes them, `x-sgr-ignored-parameters` names them as not forwarded. `prompt_cache_retention: "24h"` on a request that demands zero retention is a 400 naming it. [Prompt caching](prompt-caching.md#openais-explicit-caching). |
| `prompt_cache_key` | Names the conversation for caching; `promptCacheKey` works too. It names the session when `session_id` and `x-session-id` are absent. |
| `session_id` | A name for the conversation, 1 to 256 characters, so its turns stay on the provider that holds its cache. An empty or whitespace-only value counts as absent; any other value that is not a string of 1 to 256 characters, such as one over 256 characters, is 400 `invalid_parameter` naming `session_id`. The `x-session-id` header names it too. |
| `provider` | Steers one request: `order`, `only`, `ignore`, `allow_fallbacks`, `sort`, `zdr` and `data_collection`. Other keys, such as `require_parameters`, `quantizations`, `max_price` and `sort.partition`, are accepted and do not change which provider serves. Official SDKs send it through `extra_body`. [The provider object](api-reference.md#the-provider-object). |
| `gateway` | llm.buzz's own per-request bounds, `retry` and `routing.allow_fallbacks`. Never sent to a provider. A warm-up frame on the WebSocket refuses it. [Retries and fallbacks per request](api-reference.md#retries-and-fallbacks-per-request). |
| `service_tier` | `auto` and `default` are served at the standard tier and not sent to the provider. `flex`, `priority` and `fast` are 400 `unsupported_capability` naming `service_tier`, before any provider is called: llm.buzz sells no priority tier, and no model has a price for one today. |
| The `web` plugin | Accepted, and nothing is searched: llm.buzz runs no web search. The request is served without one, nothing is billed for one, and `x-sgr-ignored-parameters` names `web_search->dropped(search_unavailable)`. For an organization whose requests llm.buzz keeps in the EU, the request is 400 `web_search_unavailable_in_region`. |
| other `plugins` entries | Dropped and named by id, as `plugins.response-healing`. |
| `models`, `route`, `transforms`, `usage`, `trace`, `debug` | Dropped and named. A model's own providers fail over through its routing preset, in place of `models`. Usage counts and `usage.cost` come back whether or not you send `usage`. |
| `background`, `context_management`, `conversation`, `max_tool_calls`, `moderation`, `prompt` | Refused: 400 `unsupported_parameter` naming the field. |
| `defer_loading` on a tool | Keeps the tool out of the prompt until a tool search finds it. [Tool search](api-reference.md#tool-search). |

## Headers

You send:

| Header | Use |
|---|---|
| `Authorization` | `Bearer` and your inference key. [Authentication](authentication.md). |
| `Idempotency-Key` | Makes a retry safe: [Retries](#retries). |
| `x-session-id` | Names the conversation, as `session_id` does. |
| `X-Title`, `HTTP-Referer` | Name your app. [App attribution](api-reference.md#app-attribution). |
| `x-client-request-id` | Your own correlation ID. llm.buzz echoes it on the answer. It also names the conversation for caching when you send no `session_id`, `x-session-id`, `prompt_cache_key` or `previous_response_id`; a value that changes on every request gives each request its own cache key, so send one stable value per conversation, or none. |

Every served answer carries:

| Header | Value |
|---|---|
| `x-request-id` | The request's ID. Its generation ID is `gen-` and the same 32 hex digits: [Cost of a finished request](api-reference.md#cost-of-a-finished-request). |
| `x-gateway-alias` | The model name that served. |
| `x-gateway-canonical-model` | The exact model that produced the answer. |
| `x-gateway-provider` | The display name of the provider that answered, `llm.buzz` for llm.buzz's own provider. The response object has no `provider` key, so read it here. |
| `x-gateway-route-depth`, `x-gateway-route-reason` | The answering provider's position in the order tried, and why the route was chosen. |

An answer that asks you to wait, a 429 and some 402 and 503 answers, carries `Retry-After`: wait that many seconds before you retry. An HTTP answer also carries `x-gateway-replay-repair: encrypted_reasoning_stripped` when llm.buzz retried without the encrypted reasoning items you sent back ([Input items](#input-items)).

## The answer

A non-streamed answer is OpenAI's response object. llm.buzz adds or fixes these parts:

- `output` is a list of items: `message` items with `output_text` parts, `function_call` items, and `reasoning` items. A reasoning item's `summary_text` holds the model's readable reasoning where the provider gives one, and its `encrypted_content` holds the encrypted reasoning when you asked for it. [Reasoning](reasoning.md).
- `status` is `completed` for a finished answer and `incomplete` when the output ceiling cut it.
- `instructions`, `metadata` and `tools` echo the request: `instructions` is `null` when you sent none, `metadata` is `{}`, and `tools` keeps your order, with each function tool as served.
- `frequency_penalty` and `presence_penalty` are present, as in OpenAI's response object.
- `usage` is described [below](#usage-and-cost).
- `x-sgr-ignored-parameters` is a key in the body, not a header. It lists every field llm.buzz dropped and every value it changed, with the reason, such as `temperature->dropped(unsupported_by_provider)`. Check it when a setting seems to have no effect.

## Stream

Set `stream: true` and the answer arrives as server-sent events, OpenAI's Responses stream events: `response.created`, `response.in_progress`, then the output events, such as `response.output_text.delta`, and exactly one terminal event. The response object in those events carries the echo of `instructions`, `metadata` and `tools`, and `x-sgr-ignored-parameters`, as a non-streamed answer's does. Closing a stream sent without an `Idempotency-Key` stops the request, and the prompt and the output produced up to then are billed. [Streaming](streaming.md) covers the event shapes of all three text APIs.

## WebSocket

`GET https://api.llm.buzz/api/v1/responses` with a WebSocket upgrade is the transport Codex uses. Authenticate with the same `Authorization` header as an HTTP request.

- You send `response.create` frames. llm.buzz answers each with the response's stream events, one event as JSON in each text frame.
- A failure of the request itself arrives as an OpenAI error frame, `{"type": "error", ...}`, and not as an HTTP status: [Errors](#errors).
- A frame with `generate: false` is a warm-up: llm.buzz answers it without calling a provider.
- llm.buzz checks the key before it accepts the upgrade, and a management key is refused with 403 at that point. A `GET` that is not a well-formed upgrade answers 426 `upgrade_required`; the Codex client falls back to HTTP on it.
- The socket closes at the 600-second deadline ([Limits](#limits)). The requests a socket carries do not appear in prompt logs, and an `Idempotency-Key` on the upgrade request does not apply to the frames.

## Usage and cost

| Field | Value |
|---|---|
| `input_tokens`, `output_tokens` | The counts the provider reported for the attempt that answered. |
| `input_tokens_details.cached_tokens` | Input tokens read from the provider's cache. Always present, `0` when the provider reported none. |
| `input_tokens_details.cache_write_tokens` | Input tokens written to it. Always present, `0` when the provider reported none. |
| `input_tokens_details.cache_write_1h_tokens` | The one-hour part of the writes, only when the provider reports the split. |
| `cost` | The request's total cost in US dollars at your rate. |

Cache reads and writes are separate parts of `input_tokens`. `cost` is a JSON number that includes any billed tool-search rounds, so it can exceed the counts times the listed prices. A failed attempt that another provider replaced adds 0. It is left out when any part of the cost is unknown; a model with no price reports `0`. A replay with the same `Idempotency-Key` returns the original cost. [Usage and limits](usage-and-limits.md) covers the rest, and [Prompt caching](prompt-caching.md#read-the-cache-in-usage) the cache fields.

## Retries

Send an `Idempotency-Key` header on an HTTP request and a retry with the same key and the same body never runs the request twice: it returns the first answer with its original cost, or 409 `idempotency_replay_unavailable` when that answer is not stored where the retry arrived. That 409 means the first request ran or is still running, so retry with the same key after a short backoff. The exception is a first call that ended in an error answer: its key answers 409 for good, so send a new key. [What to send next](retries-and-idempotency.md#what-to-send-next) tells the cases apart.

- The same key with another body is 409 `idempotency_conflict`. A changed suffix or `provider` hint counts as another body.
- A retry without the key, or with a new one, runs the request again and bills it again.
- A provider that fails or times out before output reaches you is replaced by the next one, and the failed attempt costs nothing; a provider's own 400 is not tried on another provider. Once output has reached you, the request stays on the provider that sent it.
- The OpenAI SDK also retries failed requests on its own, and each retry is a separate request. Set `max_retries=0` when your application must bound the work of one operation.

[Retries and idempotency](retries-and-idempotency.md) explains the rest.

## Errors

Over HTTP, errors come in OpenRouter's envelope. `metadata.gateway_code` is the code to branch on, and `metadata.param` names the field at fault.

```json
{"error": {"code": 400, "message": "...", "metadata": {"error_type": "invalid_request", "gateway_code": "unsupported_parameter", "param": "background"}}}
```

On the WebSocket, a failure of the request arrives as OpenAI's error frame, with the status in the frame, for example when the key's spending limit is reached:

```json
{"type": "error", "status": 403, "error": {"message": "Key limit exceeded", "type": "permission_error", "param": null, "code": "key_limit_exceeded"}}
```

| Status | Code | When |
|---|---|---|
| 400 | `invalid_parameter` | A field fails validation and `metadata.param` names it, or `body` when the fault spans fields: a missing `input`, an out-of-range `temperature`, a `session_id` over 256 characters, a `truncation` other than `"disabled"`, `reasoning.mode`, an `include` selector other than `reasoning.encrypted_content` and `message.output_text.logprobs`, or a `text.format.type` other than `text` or `json_schema`. Fix the field. |
| 400 | `invalid_json` | The body is not JSON. |
| 400 | `unsupported_parameter` | You sent a refused field: `background`, `context_management`, `conversation`, `max_tool_calls`, `moderation` or `prompt`. Remove it. |
| 400 | `previous_response_not_found` | `previous_response_id` names a turn llm.buzz does not have: expired, sent with `store: false`, lost, or made by another model. Resend the whole conversation in `input`. |
| 400 | `unsupported_capability` | `service_tier` is `flex`, `priority` or `fast`, or no provider of the model offers what the request asks. `metadata.param` names it. |
| 400 | `invalid_request` | A refusal after llm.buzz read the body: the provider's own 400, whose sentence follows `provider rejected the request:`, or another admission refusal. The message says which. |
| 400 | `context_length_exceeded` | The prompt fits no provider's context window. |
| 400 | `web_search_unavailable_in_region` | The request asked for a web search and your organization's requests stay in the EU. |
| 409 | `idempotency_conflict` | The `Idempotency-Key` was used with another body. |
| 409 | `idempotency_replay_unavailable` | The request ran or is running, and its answer is not stored where this retry arrived. Retry with the same key after a short backoff. If the first call ended in an error answer, the key answers 409 for good: send a new key. |
| 413 | `request_too_large` | The body is over 64 MiB. |
| 426 | `upgrade_required` | A `GET` to `/responses` that is not a WebSocket upgrade. Open a WebSocket, or `POST` the request. |
| 503 | `continuation_store_unavailable` | llm.buzz cannot reach its store of turns. Retry after `Retry-After` seconds. |

[Errors](errors.md) lists every code, including the key, balance, model and provider errors this API shares with the others.

## Limits

- A model's context window is `context_length` on its entry in `GET https://api.llm.buzz/api/v1/models`, and `top_provider.max_completion_tokens` is its output ceiling.
- A request may run for 600 seconds. A stream still generating then ends with a timeout error and is billed for the prompt and the output you received, and a WebSocket closes. A request that is not streaming answers 504 `deadline_exceeded`.
- A request body over 64 MiB answers 413 `request_too_large`.
- A request makes at most 8 provider calls, every fallback included; `gateway.retry` lowers the ceiling.
- A kept turn can be continued for up to 24 hours.

[Usage and limits](usage-and-limits.md#limits) collects the limits of every API.

## Related

- [Tools and structured output](tools-and-structured-output.md), [Streaming](streaming.md), [Reasoning](reasoning.md), [Images in requests](images-in-requests.md) and [Prompt caching](prompt-caching.md) cover what the three text APIs share.
- [Chat Completions](chat-completions.md) and [Messages](messages.md) are the other two text APIs; [Choosing an API](choosing-an-api.md) compares them.
- [Coming from OpenRouter](from-openrouter.md) lists every difference from OpenRouter in one place.