Responses

Send a request

input is a string or a list of items, and instructions is the system instruction for this request. The OpenAI SDKs give the reply text as output_text.

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://api.llm.buzz/api/v1",
    api_key=os.environ["LLM_BUZZ_KEY"],
)
response = client.responses.create(
    model="openai/gpt-5.6-luna",
    instructions="You answer in one short sentence.",
    input="Why do HTTP clients retry with backoff?",
)
print(response.output_text)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llm.buzz/api/v1",
  apiKey: process.env.LLM_BUZZ_KEY,
});
const response = await client.responses.create({
  model: "openai/gpt-5.6-luna",
  instructions: "You answer in one short sentence.",
  input: "Why do HTTP clients retry with backoff?",
});
console.log(response.output_text);

curl https://api.llm.buzz/api/v1/responses \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.6-luna",
    "instructions": "You answer in one short sentence.",
    "input": "Why do HTTP clients retry with backoff?"
  }'

The answer is OpenAI's response object with the additions below. The model name, and the models you can call, come from GET https://api.llm.buzz/api/v1/models (Choosing an API).

Continue a conversation

llm.buzz keeps each finished turn. Send previous_response_id with the id of the last answer and only this turn's input, and llm.buzz adds the turns before it.

first = client.responses.create(
    model="openai/gpt-5.6-luna",
    instructions="You answer in one short sentence.",
    input="Which port does HTTPS use?",
)
second = client.responses.create(
    model="openai/gpt-5.6-luna",
    instructions="You answer in one short sentence.",
    previous_response_id=first.id,
    input="And plain HTTP?",
)
print(second.output_text)

const first = await client.responses.create({
  model: "openai/gpt-5.6-luna",
  instructions: "You answer in one short sentence.",
  input: "Which port does HTTPS use?",
});
const second = await client.responses.create({
  model: "openai/gpt-5.6-luna",
  instructions: "You answer in one short sentence.",
  previous_response_id: first.id,
  input: "And plain HTTP?",
});
console.log(second.output_text);

# RESPONSE_ID is the "id" of the first answer
curl https://api.llm.buzz/api/v1/responses \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.6-luna",
    "instructions": "You answer in one short sentence.",
    "previous_response_id": "'"$RESPONSE_ID"'",
    "input": "And plain HTTP?"
  }'

• A turn is kept for up to 24 hours. It is not a prompt log: the console does not show it, and it is kept even when your organization turned prompt logging off or the request demanded zero retention (Models and routing). Send store: false to keep a turn out.
• A request that continues a turn llm.buzz no longer has answers 400 previous_response_not_found with param: previous_response_id, and its message asks you to resend the whole conversation in input. A turn is gone after 24 hours, when it was sent with store: false, and sooner if llm.buzz's store of turns restarts or fills up. The Codex client resends by itself.
• When llm.buzz cannot reach its store of turns, a continuation answers 503 continuation_store_unavailable. Wait Retry-After seconds and retry. A turn that finishes while the store is unreachable is answered but not kept, so it cannot be continued later.
• Each turn follows only its own instructions. Send them again on every turn that should follow them.
• A kept turn belongs to the organization, key identity and model that made it. A continuation that names another model answers 400 previous_response_not_found, so continue with the same model.
• A continuation can stay on the provider that served the turn it continues. While that provider can still be used, provider.order and the routing preset do not move it, and only and ignore do not narrow it, so send provider preferences with the first turn. A continuation also keeps the caching session its first turn resolved, unless it sends its own session_id, x-session-id or prompt_cache_key (Prompt caching).

Input items

input is a string, which counts as one user message, or a list of items:

• Messages, {"role": "user", "content": ...}, with the roles user, assistant, system and developer. A developer or system item is accepted for every model. A leading one is the model's instruction. A later one stays in place when the provider takes one there, and otherwise moves into the user text at the same place, named in the body's x-sgr-ignored-parameters, for example messages.system->folded(system_messages_leading_only).
• function_call and function_call_output items carry a tool loop. Tool call IDs are opaque strings of 1 to 65,536 characters: send each one back complete and unchanged, in the call and in its output. A function_call_output's output is a string or a list of text and image parts, and any other part kind is a 400 that names it. Tools and structured output covers the loop.
• reasoning items from an earlier answer, sent back as they came. Ask for include: ["reasoning.encrypted_content"] and a provider that supports it returns the model's reasoning encrypted in its reasoning items. Send those items back with the next request to keep the model's thinking across tool calls in a conversation sent with store: false. An encrypted item is tied to the provider account that produced it. When a provider refuses one you sent back, llm.buzz retries without the encrypted reasoning items, keeps the rest of the conversation, and sets the response header x-gateway-replay-repair: encrypted_reasoning_stripped.

Request fields

A field llm.buzz deliberately refuses is a 400 before any provider is called, and a value that fails validation is a 400 invalid_parameter with metadata.param naming the field. A field llm.buzz drops, such as messages or stop, which Responses does not know, is named in the body's x-sgr-ignored-parameters.

Headers

You send:

Every served answer carries:

An answer that asks you to wait, a 429 and some 402 and 503 answers, carries Retry-After: wait that many seconds before you retry. An HTTP answer also carries x-gateway-replay-repair: encrypted_reasoning_stripped when llm.buzz retried without the encrypted reasoning items you sent back (Input items).

The answer

A non-streamed answer is OpenAI's response object. llm.buzz adds or fixes these parts:

• output is a list of items: message items with output_text parts, function_call items, and reasoning items. A reasoning item's summary_text holds the model's readable reasoning where the provider gives one, and its encrypted_content holds the encrypted reasoning when you asked for it. Reasoning.
• status is completed for a finished answer and incomplete when the output ceiling cut it.
• instructions, metadata and tools echo the request: instructions is null when you sent none, metadata is {}, and tools keeps your order, with each function tool as served.
• frequency_penalty and presence_penalty are present, as in OpenAI's response object.
• usage is described below.
• x-sgr-ignored-parameters is a key in the body, not a header. It lists every field llm.buzz dropped and every value it changed, with the reason, such as temperature->dropped(unsupported_by_provider). Check it when a setting seems to have no effect.

Stream

Set stream: true and the answer arrives as server-sent events, OpenAI's Responses stream events: response.created, response.in_progress, then the output events, such as response.output_text.delta, and exactly one terminal event. The response object in those events carries the echo of instructions, metadata and tools, and x-sgr-ignored-parameters, as a non-streamed answer's does. Closing a stream sent without an Idempotency-Key stops the request, and the prompt and the output produced up to then are billed. Streaming covers the event shapes of all three text APIs.

WebSocket

GET https://api.llm.buzz/api/v1/responses with a WebSocket upgrade is the transport Codex uses. Authenticate with the same Authorization header as an HTTP request.

• You send response.create frames. llm.buzz answers each with the response's stream events, one event as JSON in each text frame.
• A failure of the request itself arrives as an OpenAI error frame, {"type": "error", ...}, and not as an HTTP status: Errors.
• A frame with generate: false is a warm-up: llm.buzz answers it without calling a provider.
• llm.buzz checks the key before it accepts the upgrade, and a management key is refused with 403 at that point. A GET that is not a well-formed upgrade answers 426 upgrade_required; the Codex client falls back to HTTP on it.
• The socket closes at the 600-second deadline (Limits). The requests a socket carries do not appear in prompt logs, and an Idempotency-Key on the upgrade request does not apply to the frames.

Usage and cost

Cache reads and writes are separate parts of input_tokens. cost is a JSON number that includes any billed tool-search rounds, so it can exceed the counts times the listed prices. A failed attempt that another provider replaced adds 0. It is left out when any part of the cost is unknown; a model with no price reports 0. A replay with the same Idempotency-Key returns the original cost. Usage and limits covers the rest, and Prompt caching the cache fields.

Retries

Send an Idempotency-Key header on an HTTP request and a retry with the same key and the same body never runs the request twice: it returns the first answer with its original cost, or 409 idempotency_replay_unavailable when that answer is not stored where the retry arrived. That 409 means the first request ran or is still running, so retry with the same key after a short backoff. The exception is a first call that ended in an error answer: its key answers 409 for good, so send a new key. What to send next tells the cases apart.

• The same key with another body is 409 idempotency_conflict. A changed suffix or provider hint counts as another body.
• A retry without the key, or with a new one, runs the request again and bills it again.
• A provider that fails or times out before output reaches you is replaced by the next one, and the failed attempt costs nothing; a provider's own 400 is not tried on another provider. Once output has reached you, the request stays on the provider that sent it.
• The OpenAI SDK also retries failed requests on its own, and each retry is a separate request. Set max_retries=0 when your application must bound the work of one operation.

Retries and idempotency explains the rest.

Errors

Over HTTP, errors come in OpenRouter's envelope. metadata.gateway_code is the code to branch on, and metadata.param names the field at fault.

{"error": {"code": 400, "message": "...", "metadata": {"error_type": "invalid_request", "gateway_code": "unsupported_parameter", "param": "background"}}}

On the WebSocket, a failure of the request arrives as OpenAI's error frame, with the status in the frame, for example when the key's spending limit is reached:

{"type": "error", "status": 403, "error": {"message": "Key limit exceeded", "type": "permission_error", "param": null, "code": "key_limit_exceeded"}}

Errors lists every code, including the key, balance, model and provider errors this API shares with the others.

Limits

• A model's context window is context_length on its entry in GET https://api.llm.buzz/api/v1/models, and top_provider.max_completion_tokens is its output ceiling.
• A request may run for 600 seconds. A stream still generating then ends with a timeout error and is billed for the prompt and the output you received, and a WebSocket closes. A request that is not streaming answers 504 deadline_exceeded.
• A request body over 64 MiB answers 413 request_too_large.
• A request makes at most 8 provider calls, every fallback included; gateway.retry lowers the ceiling.
• A kept turn can be continued for up to 24 hours.

Usage and limits collects the limits of every API.

Related

• Tools and structured output, Streaming, Reasoning, Images in requests and Prompt caching cover what the three text APIs share.
• Chat Completions and Messages are the other two text APIs; Choosing an API compares them.
• Coming from OpenRouter lists every difference from OpenRouter in one place.