# Quickstart

Call any model on llm.buzz with the OpenAI or Anthropic SDK you already use.

If you use OpenRouter today, [Coming from OpenRouter](from-openrouter.md) lists what differs.

## Get a key

Create a key in the console under [Keys](https://app.llm.buzz/console/keys), and keep it in an environment variable.

```bash
export LLM_BUZZ_KEY="exp_vk_..."
```

## Make a request

Point your SDK at `https://api.llm.buzz/api/v1`. The Anthropic SDK works too, through `/api/v1/messages`.

```python tab="Python"
from openai import OpenAI
import os

client = OpenAI(
    base_url="https://api.llm.buzz/api/v1",
    api_key=os.environ["LLM_BUZZ_KEY"],
)
reply = client.chat.completions.create(
    model="deepseek/deepseek-v4.1-flash",
    messages=[{"role": "user", "content": "Hello"}],
)
print(reply.choices[0].message.content)
```

```ts tab="TypeScript"
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llm.buzz/api/v1",
  apiKey: process.env.LLM_BUZZ_KEY,
});
const reply = await client.chat.completions.create({
  model: "deepseek/deepseek-v4.1-flash",
  messages: [{ role: "user", content: "Hello" }],
});
console.log(reply.choices[0].message.content);
```

```bash tab="curl"
curl https://api.llm.buzz/api/v1/chat/completions \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "deepseek/deepseek-v4.1-flash", "messages": [{"role": "user", "content": "Hello"}]}'
```

## Stream the answer

Set `stream` to true to get the answer as it is generated. [Streaming](streaming.md) covers the events of each text API and what a failure looks like.

```python
stream = client.chat.completions.create(
    model="deepseek/deepseek-v4.1-flash",
    messages=[{"role": "user", "content": "Write a haiku"}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")
```

## Choose a model

Model names are `author/slug`, as on each model page. By default each request goes to the best provider on price and speed, and fails over to the next one. [Choosing an API](choosing-an-api.md) says which API each kind of model takes.

```python
# the cheapest provider first
model="deepseek/deepseek-v4.1-flash:floor"

# only providers that keep nothing
extra_body={"provider": {"zdr": True}}
```

## Handle errors

The error body's `metadata.gateway_code` says what went wrong; [Errors](errors.md) lists the codes.

| Status | Code | What to do |
|---|---|---|
| 401 | `invalid_key` | Check the key, and that it is not disabled or revoked. |
| 402 | `insufficient_credits` | Add credit, or turn on auto top-up. Free models still answer. |
| 404 | `model_not_found` | Check the model name on the [Models page](/models). |
| 404 | `no_allowed_providers` | Your provider preferences leave no provider. Relax them. |
| 429 | `unavailable_route` | Wait for `Retry-After` seconds, then retry. |
| 503 | `unavailable_route` | No provider could answer just now. Retry with backoff. |

# Choosing an API

llm.buzz serves each kind of model on the API made for it. A chat model answers Chat Completions, Responses and Messages alike; embeddings, image, audio and decision models have their own routes.

## The APIs

Routes are relative to `https://api.llm.buzz/api/v1`, except the decisions route that names its own base.

| API | Route | Models | Clients | Streams | Replays a keyed retry | Error format |
|---|---|---|---|---|---|---|
| [Chat Completions](chat-completions.md) | `POST /chat/completions` | Text models | OpenAI and OpenRouter SDKs, Cline, OpenCode, Aider | Server-sent events | Yes | OpenRouter's envelope |
| [Responses](responses.md) | `POST /responses`, and `GET /responses` as a WebSocket | Text models | OpenAI SDK, Codex | Server-sent events, or the WebSocket | Yes, over HTTP | OpenRouter's envelope; OpenAI's error frames on the WebSocket |
| [Messages](messages.md) | `POST /messages`, `POST /messages/count_tokens` | Text models | Anthropic SDK, Claude Code | Server-sent events | No | Anthropic's error body |
| [Embeddings](embeddings.md) | `POST /embeddings` | Embeddings models | OpenAI and OpenRouter SDKs | Never | No | OpenRouter's envelope |
| [Images](images.md) | `POST /images`, `POST /images/generations` | Image models | OpenRouter SDK on `/images`, OpenAI SDK on `/images/generations` | Never | No | OpenRouter's envelope |
| [Speech to text](audio.md#speech-to-text) | `POST /audio/transcriptions` | Speech-to-text models | OpenAI and OpenRouter SDKs | Never | No | OpenRouter's envelope |
| [Text to speech](audio.md#text-to-speech) | `POST /audio/speech` | Text-to-speech models | OpenAI and OpenRouter SDKs | Never | No | OpenRouter's envelope |
| [Decisions](decisions.md) | `POST /systemone`, `POST https://api.llm.buzz/api/alpha/decisions` | None today | OpenRouter's SDKs | Never | No | OpenRouter's envelope |

Only Chat Completions and Responses replay a request sent again with the same `Idempotency-Key`, and Responses does so over HTTP only: a key on the WebSocket does not apply. On every other API a retry runs as a request of its own: [Retries and idempotency](retries-and-idempotency.md). Every API takes your key as a bearer token, and Messages also takes it in `x-api-key`: [Authentication](authentication.md#send-the-key).

The SDKs take the base URL in their own setting. The OpenAI SDK takes `base_url="https://api.llm.buzz/api/v1"` (`baseURL` in TypeScript), the OpenRouter SDKs take `server_url="https://api.llm.buzz/api/v1"` (`serverURL` in TypeScript), and the Anthropic SDK takes `base_url="https://api.llm.buzz/api"` (`baseURL` in TypeScript), without `/v1`: [Base URL and SDKs](api-reference.md#base-url-and-sdks). The OpenRouter SDKs reach decisions through a server URL of their own: [Decisions](decisions.md#routes).

## Which text API

Chat Completions, Responses and Messages reach the same models. llm.buzz turns each into one request, so the providers a model has, how a request is routed, what it costs and the limits it runs under do not depend on the API you call. A text model answers all three, whoever made it: a Claude model answers Chat Completions, and a model that is not Claude answers Messages. A chat model that returns images is the exception ([below](#find-a-model-for-an-api)).

Pick the API your client speaks.

- **Chat Completions** for most SDKs and coding agents: the OpenAI and OpenRouter SDKs, Cline, OpenCode and Aider.
- **Responses** for the OpenAI SDK's newer API and for Codex, and when you want llm.buzz to keep the conversation, so a follow-up sends only its own input.
- **Messages** for the Anthropic SDK and Claude Code.

[Cheapest models for coding agents](/docs/guides/cheapest-models-for-coding-agents) sets up Claude Code, Codex, Cline, OpenCode and Aider. What differs between the three:

| | Chat Completions | Responses | Messages |
|---|---|---|---|
| Conversation | You send the whole history each time. | You send your next input and `previous_response_id`; llm.buzz keeps a turn for up to 24 hours, unless you send `store: false`. | You send the whole history each time. |
| System instruction | A `system` or `developer` message. | `instructions`, for this request only. | `system`. |
| Output limit | `max_tokens`, `max_completion_tokens` or `max_output_tokens`, one per request. | `max_output_tokens`. | `max_tokens`, required. |
| Structured output | `response_format`, with `json_schema` or `json_object`. | `text.format`, with `json_schema`; `json_object` is a 400. | Anthropic's `output_config.format`. An `output_config` that holds more than `effort` is sent as you wrote it when every provider that may serve the request takes Anthropic's API, and otherwise dropped and named; its `effort` still applies. `response_format` is dropped and named. |
| `Idempotency-Key` | Replays a retry. | Replays a retry sent over HTTP. | Not used. |
| The `provider` object in the request | Read. | Read. | Read. |
| `provider` in the answer | The name of the provider that answered, on the body and on every stream chunk. | Not in the answer. | Not in the answer. |
| Readable reasoning | `message.reasoning` and `delta.reasoning`. | `summary_text` of a reasoning item. | `thinking` blocks. |
| Token probabilities | `logprobs` and `top_logprobs`. | `include: ["message.output_text.logprobs"]` with `top_logprobs`. | Not offered. |
| Tool search declaration | `{"type": "openrouter:tool_search"}`. | `{"type": "tool_search"}`. | `tool_search_tool_bm25` or `tool_search_tool_regex`. |

[Tools and structured output](tools-and-structured-output.md), [Streaming](streaming.md), [Reasoning](reasoning.md) and [Images in requests](images-in-requests.md) explain what the three share, with each API's spelling.

## Find a model for an API

`GET https://api.llm.buzz/api/v1/models` lists the models your key can call, and the public models without a key. Its `output_modalities` parameter picks the kind: it defaults to `text`, `all` lists every kind, and a comma list such as `text,embeddings` lists models that produce any of them.

```bash
curl -s "https://api.llm.buzz/api/v1/models?output_modalities=embeddings"
```

| `output_modalities` | Lists | Call it on |
|---|---|---|
| `text` | Text models. | Chat Completions, Responses, Messages |
| `embeddings` | Embeddings models. | Embeddings |
| `image` | Image generation models, and chat models that return images. | Images; Chat Completions for a chat model |
| `transcription` | Speech-to-text models. | Speech to text |
| `speech` | Text-to-speech models. | Text to speech |
| `decisions` | Models that answer decisions: none today. | Decisions |

A model sent to an API that does not serve it answers 400 on `model`. The text APIs name the route an embeddings, image or audio model takes. A chat model that returns images serves Chat Completions only: Responses and Messages refuse it.

## Not served

- `POST /api/v1/completions`, OpenAI's older text-completions route, answers 404. Send the prompt as one user message to Chat Completions.
- Rerank and video generation answer 404 too.

[Coming from OpenRouter](from-openrouter.md) lists every route and feature of OpenRouter's that llm.buzz does not offer.

# Coming from OpenRouter

llm.buzz speaks OpenRouter's API, so most OpenRouter code works after you change the base URL and the key. This page lists what else differs, with links to the pages that explain more.

## Switch your client

Create a key on the console's [Keys](https://app.llm.buzz/console/keys) page. It starts `exp_vk_`. There is no OAuth key exchange: you create a key in the console, and every client sends it as its API key: as a bearer token, or on Messages in `x-api-key`, the header the Anthropic SDK sends ([Authentication](authentication.md#send-the-key)).

| Client | Setting | Value |
|---|---|---|
| OpenRouter Python | `server_url` | `https://api.llm.buzz/api/v1` |
| OpenRouter TypeScript | `serverURL` | `https://api.llm.buzz/api/v1` |
| OpenAI | `base_url` (`baseURL` in TypeScript) | `https://api.llm.buzz/api/v1` |
| Anthropic | `base_url` (`baseURL` in TypeScript) | `https://api.llm.buzz/api` |
| curl | the URL | `https://api.llm.buzz/api/v1/chat/completions` |

The Anthropic address has no `/v1`: the SDK adds `/v1/messages` itself ([Base URL and SDKs](api-reference.md#base-url-and-sdks)). The OpenRouter SDKs call decisions through a server URL of their own: [Decisions](decisions.md#routes). Any other client of these APIs needs only the base URL and the key.

## What works the same

The routes you call are OpenRouter's: Chat Completions, Responses, Messages, Embeddings, Images, speech to text and text to speech, `GET /models` and `GET /models/{author}/{slug}/endpoints` with the same shapes, `GET /generation`, and the key and credits routes `GET /key`, `GET /credits` and `/keys` ([Endpoints](api-reference.md#endpoints)). The `provider` object takes `order`, `only`, `ignore`, `allow_fallbacks`, `sort`, `zdr` and `data_collection` ([Provider preferences](models-and-routing.md#provider-preferences)), and `:nitro` and `:floor` change the order on a model's name ([Suffixes](models-and-routing.md#suffixes)). Errors come in OpenRouter's envelope with `metadata.error_type`, and add `metadata.gateway_code`, llm.buzz's own code to branch on; Messages answers Anthropic's error body ([The envelope](errors.md#the-envelope)). A page can call the API from a browser with its visitor's own key ([Browser access](api-reference.md#browser-access)), and `X-Title` and `HTTP-Referer` name your app ([App attribution](api-reference.md#app-attribution)).

## Request fields that differ

On Chat Completions, Responses and Messages, llm.buzz drops a field it does not know instead of refusing it, and names each dropped field in `x-sgr-ignored-parameters`. That is a key in the response body, never a header: in a Chat Completions body and in the content and finish chunks of its stream, in the Responses object, and in Messages in `message_start` and the final message. Embeddings and Images drop unknown fields without naming them, and Decisions refuse them. A field llm.buzz deliberately refuses is a 400 before any provider is called. On Chat Completions and Responses its `metadata.gateway_code` is `unsupported_parameter` (the field is refused), `invalid_parameter` (only some values are) or `unsupported_capability`, and `metadata.param` names it. Messages renders every field refusal as Anthropic's `invalid_request_error`, with `(param: NAME)` at the end of the message. The same field can be refused on one API and dropped on another:

| Field | What llm.buzz does |
|---|---|
| `models`, `route`, `transforms`, `usage`, `trace`, `debug` | Dropped and named on all three APIs. |
| `plugins` entries other than `web` | Chat Completions and Responses: dropped, named `plugins.<id>`. Messages: `plugins` dropped whole and named. |
| `:online`, `web_search_options`, the `web` plugin | Chat Completions: served without a search. Responses: `:online` and the `web` plugin are served without a search, and `web_search_options`, not a Responses field, is dropped and named. Messages: `:online` is named `model:online`, `plugins` and `web_search_options` are dropped and named, and nothing is searched. |
| `seed`, `logit_bias` | Chat Completions: 400 `unsupported_parameter`. Responses and Messages: dropped and named. |
| `audio`, `modalities`, `functions`, `function_call`, `prediction` | Chat Completions: 400 `unsupported_parameter`. Responses and Messages: dropped and named. |
| `moderation` | Chat Completions and Responses: 400 `unsupported_parameter`. Messages: dropped and named. |
| `stream_options` | Chat Completions: dropped and named when the request does not stream; on a stream `include_usage` changes nothing, because the last chunk carries usage whenever the counts are known. Responses and Messages: dropped and named. |
| `n` other than 1 | Chat Completions: 400 `invalid_parameter`. Responses and Messages: dropped and named. |
| `store` | Chat Completions: `true` is 400 `invalid_parameter`. Responses: used, and `store: false` keeps the turn out of continuation. Messages: dropped and named. |
| `previous_response_id` | Chat Completions: dropped and named. Responses: used, and it continues a kept turn. Messages: dropped and named. |
| `service_tier` | Chat Completions and Responses: `flex`, `priority` and `fast` are 400 `unsupported_capability`; `auto` and `default` are served at the standard tier and not sent on. Messages: 400, naming `service_tier`. |
| `background`, `conversation`, `max_tool_calls`, `prompt` | Chat Completions: dropped and named. Responses: 400 `unsupported_parameter`. Messages: dropped and named. |
| `context_management` | Chat Completions: dropped and named. Responses: 400 `unsupported_parameter`. Messages: sent where the provider takes it, dropped and named elsewhere. |
| `betas`, `container`, `fallbacks`, `fallback_credit_token`, `mcp_servers`, `thread`, `user_profile_id` | Chat Completions and Responses: dropped and named. Messages: 400, naming the field; send `betas` as the `anthropic-beta` header. |

Responses refuses a few more values with 400 `invalid_parameter`: a `truncation` other than `"disabled"`, any `reasoning.mode`, a `text.format.type` other than `text` or `json_schema` (`json_object` included: ask for JSON with a `json_schema` format), and any `include` selector other than `reasoning.encrypted_content` and `message.output_text.logprobs`. Chat Completions takes `response_format: {"type": "json_object"}`. [Chat Completions](chat-completions.md#request-fields), [Responses](responses.md#request-fields) and [Messages](messages.md#request-fields) list every field of their API. [Tools and structured output](tools-and-structured-output.md) covers the tool and format fields.

What happens in place of these fields:

- **Fallback models.** A model's own providers fail over through its routing preset in place of `models`: [How a provider is chosen](models-and-routing.md#how-a-provider-is-chosen). Usage counts and `usage.cost` come back whether or not you send `usage`.
- **Web search** is not available on llm.buzz today. A request that asks for one is still served, without the search; nothing is sent to a search service and nothing is billed for one. `x-sgr-ignored-parameters` names it `web_search->dropped(search_unavailable)`. For an organization whose requests llm.buzz keeps in the EU, such a request is 400 `web_search_unavailable_in_region`. [Suffixes](models-and-routing.md#suffixes).
- **Other `provider` keys.** `require_parameters`, `quantizations`, `max_price` and `sort.partition` are accepted on all three APIs and do not change which provider serves. `provider.sort` takes `price`, `latency`, `throughput` and `smart`; `throughput` ranks like `latency`, by recent time to first token. `exacto`, `:exacto` and any other sort string leave the model's routing preset in place. [The provider object](api-reference.md#the-provider-object).
- **Suffixes.** A suffix such as `:free`, `:extended` or `:thinking` is part of the model's name, which is matched exactly, and no model is listed under such a name today. A name that is not listed answers 404 `model_not_found`. [Model names](models-and-routing.md#model-names).
- **Service tiers.** llm.buzz does not sell a priority tier, and no model has a price for `flex`, `priority` or `fast` today, so on Chat Completions and Responses a request for one is refused with 400 `unsupported_capability`, naming `service_tier`, before any provider is called. `auto` and `default` are served at the standard tier.
- **Conversations.** Responses keeps each finished turn for up to 24 hours: [Continue a conversation](responses.md#continue-a-conversation).

## Routes llm.buzz does not serve

Apart from batches, files, decisions and the two model routes below, an OpenRouter route that is missing from the [Endpoints](api-reference.md#endpoints) table answers 404 `unknown_route` ("Unknown route"). Among them:

- `/completions`, `/rerank` and `/videos`;
- `/providers`, `/endpoints/zdr`, `/activity`, `/analytics/*`, `/generation/content` and `/generation/feedback`;
- `/presets`, `/guardrails`, `/workspaces`, `/organization/*`, `/byok`, `/auth/keys`, `/tools`, `/observability/*` and `/private-endpoints`.

llm.buzz does not enable batch jobs: `/batches` and `/files` answer 404 with a key, "batches are not enabled on this gateway": send a large job as single requests, as the [batch jobs guide](/docs/guides/batch-jobs) shows. `/models/count` and `/models/user` read their last segment as a model name and answer 404 `model_not_found`: read `total_count` on `GET /models` for a count, and list the models your key can call with `GET /models` and your key. Decisions are served, but no public model answers them today: [Decisions](decisions.md).

In their place: a model's endpoints route lists its providers (`/providers`), `provider.zdr` keeps a request on zero-retention providers, and your organization's data policy narrows the endpoints route the same way (`/endpoints/zdr`), the console's Activity page and `GET /generation` cover activity and analytics, and the console's Prompt logs page shows logged prompts (`/generation/content`).

## Features llm.buzz does not offer

- **Router models.** `openrouter/auto` and OpenRouter's other router models name no model on llm.buzz: a request for one answers 404 `model_not_found`. Name a model, and choose its provider order with `provider.sort`.
- **Saved presets.** There is no `/presets` route. llm.buzz's own routing presets (`smart`, `cheapest`, `fastest` and `order`) order one model's providers, and `provider.sort` or a `:floor` or `:nitro` suffix picks one of them for a single request: [How a provider is chosen](models-and-routing.md#how-a-provider-is-chosen).
- **Server tools.** Tool search is the one tool llm.buzz runs for you: [Tool search](api-reference.md#tool-search). Web search is not available, as above, and OpenRouter's other server tools are not offered.
- **Plugins.** No plugin runs. On Chat Completions and Responses an entry in `plugins` is dropped and named `plugins.<id>`, and the `web` plugin is the web search above; Messages drops `plugins` whole.
- **A response cache.** Replay is opt-in, on Chat Completions and Responses only: a retry that carries the same `Idempotency-Key` returns the first answer, and no other request is answered from a stored one. Messages reads no `Idempotency-Key`, so a Messages retry runs and is billed again ([Idempotency-Key](retries-and-idempotency.md#idempotency-key)).
- **Workspaces.** Your keys, the people you invite with their roles and one prepaid balance belong to an organization: [Organizations](authentication.md#organizations).
- **Guardrails.** There is no `/guardrails` route.
- **Bring your own key.** The hosted service has no bring-your-own-key: `is_byok` is `false` on every generation record.
- **Broadcast to observability tools.** There is no `/observability` route. Read your traffic on the console's [Activity](https://app.llm.buzz/console/activity) and [Logs](https://app.llm.buzz/console/logs) pages.
- **SAML single sign-on and SCIM.** Neither is offered. People sign in to the console with an email and a password, Google or GitHub.
- **Regional base URLs.** There is no regional base URL. To keep a request in the EU, list the EU tags in `provider.only`: [EU](models-and-routing.md#eu). `GET /models?region=eu` lists only the models with an EU provider: [The model list](api-reference.md#the-model-list).
- **Video generation.** `/videos` answers 404.
- **`openrouter_metadata`.** Read the `provider` key of a Chat Completions or Embeddings answer and the `x-gateway-*` headers of every answer: [Answer fields and headers](api-reference.md#responses).
- **Other attribution headers.** llm.buzz reads `X-Title` and `HTTP-Referer`. A header such as `X-OpenRouter-Title` is not read, and a browser preflight that asks for it does not allow it, so the browser refuses to send it.

## Answers that differ

- **Endpoint metrics.** `uptime_last_5m`, `uptime_last_30m`, `uptime_last_1d`, `latency_last_30m` and `throughput_last_30m` come from your organization's own traffic. A minute counts as observed with 1 attempt or more, where OpenRouter needs 10. `status` is always 0. Each window ends at the last computed 5-minute bucket, so the values run 2 to 8 minutes behind, and a field is `null` when its window has no sample. Without a key only `uptime_last_1d` can be set. [Endpoints](api-reference.md#endpoints).
- **Key objects.** `GET /key` and the key objects of `/keys` carry fixed values where llm.buzz has no counterpart: the `byok_usage` figures are `0`, `include_byok_in_limit` is `false` and `workspace_id` is `"default"`, and a `/keys` object has `external_user` `null`. `GET /key` also has `is_provisioning_key` `false`, `organization_id` `null`, a `free_model_daily_requests` object whose `limit`, `remaining` and `used` are `0`, and a deprecated `rate_limit`. [Check your key and credit](usage-and-limits.md#check-your-key-and-credit).
- **Creating keys.** `POST /keys` takes `name`, `limit`, `limit_reset` and `expires_at`, and accepts and ignores `include_byok_in_limit`, `workspace_id` and `external`. `external.api_key` is 400, because llm.buzz issues its own keys. Management keys are created only in the console.
- **Reasoning efforts.** A model object has no `reasoning`, `knowledge_cutoff` or `benchmarks` key. The model list names a model's efforts in `reasoning_efforts`, from `none` to `max`, when the model's lead provider's listing names them; the key is left out otherwise, including where llm.buzz knows a model's efforts only from its own rules for the model's family, and no public model names `reasoning` in `supported_parameters` today. A model without the key can still reason: [Reasoning](reasoning.md#which-models-reason).
- **Image models.** An image model's `supported_parameters.size` lists the sizes it takes, and `allowed_passthrough_parameters` is always empty, because llm.buzz checks every field and passes none through. The image model routes need a key: [Find an image model](images.md#find-an-image-model).
- **Audio endpoints.** A text-to-speech endpoint's `max_completion_tokens` is null, and OpenRouter's endpoint flags `supports_voice_cloning`, `supports_diarization`, `supports_image_reference` and `supports_multiple_audio_references` are left out, because no provider here offers those features: [Find an audio model](audio.md#find-an-audio-model).
- **The provider in the answer.** The request's `provider` object works on Chat Completions, Responses and Messages. The answer carries a `provider` key, the name of the provider that answered, only on Chat Completions (every body and stream chunk) and Embeddings. Every answer names its provider in `x-gateway-provider`.
- **Retries.** On Chat Completions and Responses, a retry with the same `Idempotency-Key` returns the first answer, or answers 409 `idempotency_replay_unavailable` when its answer is not stored where the retry arrived. Send the same key again only when the first call got no answer, or after an answer carrying `x-gateway-admission-refused: true`; after any other error answer send a new key. [What to send next](retries-and-idempotency.md#what-to-send-next) has the cases.

## Prompt logs

llm.buzz keeps the prompts and answers of your organization's Chat Completions, Responses and Messages requests in prompt logs, and logging is on by default. An admin turns it off, or deletes the logged prompts, on the console's Prompt logs page. `provider.zdr` and `provider.data_collection` work as on OpenRouter, and a request with either demand is never logged. [Prompt logs](models-and-routing.md#prompt-logs) explains the rest.

## Billing

- You pay from prepaid credit that your organization buys on the console's [Credits](https://app.llm.buzz/console/credits) page, with a service fee added when you buy: [Prepaid credit](pricing.md#prepaid-credit).
- A provider that fails or sends an empty answer and is replaced by another adds nothing to the cost; you pay the provider that answered. [Retries and idempotency](retries-and-idempotency.md#inside-one-request).
- A free offer is a provider discount with an allowance, not a `:free` model: [Free offers](pricing.md#free-offers).

# Chat Completions

Chat Completions is OpenAI's conversation API, the one most SDKs and coding agents speak. llm.buzz serves it at `POST /chat/completions` under `https://api.llm.buzz/api/v1` for every text model, with OpenRouter's extensions, so the OpenAI and OpenRouter SDKs call it with the base URL changed.

## Send a request

Send the whole conversation in `messages` on every turn: Chat Completions has no continuation, so a follow-up question carries the turns before it.

```python tab="Python"
from openai import OpenAI
import os

client = OpenAI(
    base_url="https://api.llm.buzz/api/v1",
    api_key=os.environ["LLM_BUZZ_KEY"],
)
reply = client.chat.completions.create(
    model="deepseek/deepseek-v4.1-flash",
    messages=[
        {"role": "system", "content": "You answer in one short sentence."},
        {"role": "user", "content": "Which port does HTTPS use?"},
        {"role": "assistant", "content": "HTTPS uses port 443."},
        {"role": "user", "content": "And plain HTTP?"},
    ],
)
print(reply.choices[0].message.content)
```

```ts tab="TypeScript"
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llm.buzz/api/v1",
  apiKey: process.env.LLM_BUZZ_KEY,
});
const reply = await client.chat.completions.create({
  model: "deepseek/deepseek-v4.1-flash",
  messages: [
    { role: "system", content: "You answer in one short sentence." },
    { role: "user", content: "Which port does HTTPS use?" },
    { role: "assistant", content: "HTTPS uses port 443." },
    { role: "user", content: "And plain HTTP?" },
  ],
});
console.log(reply.choices[0].message.content);
```

```bash tab="curl"
curl https://api.llm.buzz/api/v1/chat/completions \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek/deepseek-v4.1-flash",
    "messages": [
      {"role": "system", "content": "You answer in one short sentence."},
      {"role": "user", "content": "Which port does HTTPS use?"},
      {"role": "assistant", "content": "HTTPS uses port 443."},
      {"role": "user", "content": "And plain HTTP?"}
    ]
  }'
```

The answer is OpenAI's chat completion object with the additions [below](#the-answer). The model name, and the models you can call, come from `GET https://api.llm.buzz/api/v1/models` ([Choosing an API](choosing-an-api.md#find-a-model-for-an-api)).

## The messages array

`messages` is the conversation in order, with the roles `system`, `developer`, `user`, `assistant` and `tool`.

- A `developer` message is accepted for every model. A provider whose API has no developer role receives it as a system message, so the two roles read the same.
- A `system` or `developer` message that comes after the conversation has started stays in place when the provider takes one there. Otherwise its text moves into the user turn at the same place, and the body's `x-sgr-ignored-parameters` names the move, for example `messages.system->folded(system_messages_leading_only)`.
- Message content is a string or a list of parts. The parts are `text` and `image_url`; an image needs a model that lists `image` among its `architecture.input_modalities`. A `tool` message may carry `image_url` parts beside its text. [Images in requests](images-in-requests.md) has the rules.
- A tool call's `id` is an opaque string of 1 to 65,536 characters. Send each one back complete, unchanged, as the `tool_call_id` of its `tool` message. [Tools and structured output](tools-and-structured-output.md) covers the loop.

## Request fields

A field llm.buzz deliberately refuses is a 400 before any provider is called, and a value that fails validation is a 400 `invalid_parameter` with `metadata.param` naming the field. A field llm.buzz drops is named in the body's `x-sgr-ignored-parameters`.

| Field | What llm.buzz does |
|---|---|
| `model` | The model's name, `author/slug`. Four suffixes change one request: `:floor` tries the cheapest provider first, `:nitro` the fastest, `:exacto` is accepted and ignored, and `:online` is accepted and searches nothing (see `web_search_options`). [Models and routing](models-and-routing.md#suffixes). |
| `messages` | Required: a request without it is 400 `invalid_parameter`. [The messages array](#the-messages-array), above. |
| `max_tokens`, `max_completion_tokens`, `max_output_tokens` | The most output tokens, which may include reasoning. Send one of the three: two together are 400 `invalid_parameter`. Without one, llm.buzz does not choose a smaller limit for you. A provider that needs a larger minimum than your limit is skipped, and when none is left the request is refused. |
| `temperature`, `top_p` | Sent to the provider; an out-of-range `temperature` is 400 `invalid_parameter`. A model that takes none, or takes them only at reasoning effort `none`, has the value dropped and named: [Sampling settings a model does not take](api-reference.md#sampling-settings-a-model-does-not-take). |
| `top_k` | Sent to providers that take it. Dropped and named as `top_k->dropped(unsupported_by_provider)` when none does. |
| `frequency_penalty`, `presence_penalty` | Sent. Dropped and named, as `frequency_penalty->dropped(unsupported_by_provider)`, where the model takes none. |
| `seed`, `logit_bias` | Refused: 400 `unsupported_parameter` naming the field. |
| `stop` | Sent to providers that take it. Where a provider's API has no stop field, llm.buzz cuts the answer at the first match itself and ends it with `finish_reason: stop`. |
| `n` | Only `1`. Any other value is 400 `invalid_parameter`. |
| `store` | Only `false`. `true` is 400 `invalid_parameter`. |
| `stream` | `true` streams the answer as server-sent events: [Stream](#stream). |
| `stream_options` | Dropped and named when the request does not stream. On a stream, `include_usage` changes nothing: the last chunk carries usage whenever the counts are known. |
| `tools`, `tool_choice` | Sent, in OpenAI's shape. Some models cannot be forced to call a tool, and a provider that cannot enforce `strict` gets the tool without it. [Tools and structured output](tools-and-structured-output.md) says what happens to each. |
| `parallel_tool_calls` | Honored on every provider. Where a provider has no such control, `true` is its default behavior and is named `parallel_tool_calls->dropped(provider_default)`, and `false` is enforced by llm.buzz, which returns only the first tool call of a turn and names it `parallel_tool_calls->emulated(serialized_by_gateway)`. |
| `response_format` | `json_schema` asks for an answer in a JSON schema, `json_object` for any JSON object. Where a provider has no JSON mode, llm.buzz adds an instruction asking for JSON and names it `response_format->instruction(json_object)`. Use `json_schema` when the shape must be enforced. [Tools and structured output](tools-and-structured-output.md). |
| `verbosity` | `low`, `medium` or `high`, as a hint. Sent to providers that take it and omitted, named `verbosity`, elsewhere. Another value is 400 `invalid_parameter` naming `verbosity`. |
| `logprobs`, `top_logprobs` | `logprobs: true` returns each token's log probability in `choices[0].logprobs.content` (and `.refusal`); `top_logprobs`, 0 to 20, needs `logprobs: true`; a count above 20, or without `logprobs`, is 400 `invalid_parameter`. Sent to providers that return them; a request no provider of the model can serve is refused. A numeric `thinking_budget`, and a stop sequence llm.buzz applies itself, cannot be combined with them. |
| `reasoning`, `reasoning_effort`, `include_reasoning` | How hard the model thinks, and whether you see it. `reasoning` takes `effort` or `max_tokens`, plus `enabled` and `exclude`; an effort tier or a budget beside `enabled: false`, or `effort: "none"` beside `enabled: true`, is 400 `invalid_parameter` naming `reasoning.enabled`. `include_reasoning: true` is `reasoning: {}` and `false` is `reasoning: {exclude: true}`. Each provider receives the setting in its own terms. [Reasoning](reasoning.md). |
| `thinking`, `thinking_budget`, `enable_thinking`, `chat_template_kwargs.enable_thinking` | Other spellings of the same controls, translated for the provider. An enabled `thinking.budget_tokens` under 1,024 is 400 `invalid_parameter`. [Reasoning](reasoning.md). |
| `cache_control` | Anthropic's prompt-cache marker, `{"type": "ephemeral"}` with an optional `ttl` of `"5m"` or `"1h"`. Accepted at the top level, on function tools, messages, text parts and assistant tool calls. Passed on or translated, never added by llm.buzz. [Prompt caching](prompt-caching.md). |
| `prompt_cache_breakpoint` | OpenAI's explicit cache mark, `{"mode": "explicit"}`, on a `text` part of a `system`, `developer`, `user` or `tool` message; at the top level it is dropped and named. Sent to OpenAI as given. For Claude it becomes a five-minute `cache_control` marker, named `prompt_cache_breakpoint->translated(cache_control)`. On another kind of part, or on an assistant message, it is a 400 that names where it was. [Prompt caching](prompt-caching.md#openais-explicit-caching). |
| `prompt_cache_options`, `prompt_cache_retention` | OpenAI's cache mode and retention. Sent to OpenAI only; no other provider receives them, and where no provider of the model takes them, `x-sgr-ignored-parameters` names them. `prompt_cache_retention: "24h"` on a request that demands zero retention is a 400 naming it. [Prompt caching](prompt-caching.md#openais-explicit-caching). |
| `prompt_cache_key` | Names the conversation for caching; `promptCacheKey` works too. It names the session when `session_id` and `x-session-id` are absent. |
| `session_id` | A name for the conversation, 1 to 256 characters, so its turns stay on the provider that holds its cache. An empty or whitespace-only value counts as absent; any other value that is not a string of 1 to 256 characters, such as one over 256 characters, is 400 `invalid_parameter` naming `session_id`. The `x-session-id` header names it too. |
| `provider` | Steers one request: `order`, `only`, `ignore`, `allow_fallbacks`, `sort`, `zdr` and `data_collection`. Other keys, such as `require_parameters`, `quantizations`, `max_price` and `sort.partition`, are accepted and do not change which provider serves. [The provider object](api-reference.md#the-provider-object). |
| `gateway` | llm.buzz's own per-request bounds, `retry` and `routing.allow_fallbacks`. Never sent to a provider. [Retries and fallbacks per request](api-reference.md#retries-and-fallbacks-per-request). |
| `service_tier` | `auto` and `default` are served at the standard tier and not sent to the provider. `flex`, `priority` and `fast` are 400 `unsupported_capability` naming `service_tier`, before any provider is called: llm.buzz sells no priority tier, and no model has a price for one today. |
| `web_search_options`, the `web` plugin | Accepted, and nothing is searched: llm.buzz runs no web search. The request is served without one, nothing is billed for one, and `x-sgr-ignored-parameters` names `web_search->dropped(search_unavailable)`. For an organization whose requests llm.buzz keeps in the EU, the request is 400 `web_search_unavailable_in_region`. |
| other `plugins` entries | Dropped and named by id, as `plugins.response-healing`. |
| `models`, `route`, `transforms`, `usage`, `trace`, `debug` | Dropped and named. A model's own providers fail over through its routing preset, in place of `models`. Usage counts and `usage.cost` come back whether or not you send `usage`. |
| `audio`, `modalities`, `functions`, `function_call`, `prediction`, `moderation` | Refused: 400 `unsupported_parameter` naming the field. Send `tools` and `tool_choice` in place of `functions` and `function_call`. |
| `defer_loading` on a tool | Declare a tool search with `{"type": "openrouter:tool_search"}`, and a tool with `defer_loading: true` beside `function` stays out of the prompt until a search finds it. [Tool search](api-reference.md#tool-search). |
| Any other field | Dropped, and named in `x-sgr-ignored-parameters`, as `previous_response_id` is: it belongs to Responses. |

## Headers

You send:

| Header | Use |
|---|---|
| `Authorization` | `Bearer` and your inference key. [Authentication](authentication.md). |
| `Idempotency-Key` | Makes a retry safe: [Retries](#retries). |
| `x-session-id` | Names the conversation, as `session_id` does. |
| `X-Title`, `HTTP-Referer` | Name your app. [App attribution](api-reference.md#app-attribution). |
| `x-client-request-id` | Your own correlation ID, echoed on the answer. It also names the cache session when nothing else does, so send one stable value per conversation, or omit it: a value that changes on every request gives every request its own cache key. |

Every served answer carries:

| Header | Value |
|---|---|
| `x-request-id` | The request's ID. Its generation ID is `gen-` and the same 32 hex digits: [Cost of a finished request](api-reference.md#cost-of-a-finished-request). |
| `x-gateway-alias` | The model name that served. |
| `x-gateway-canonical-model` | The exact model that produced the answer. |
| `x-gateway-provider` | The display name of the provider that answered, `llm.buzz` for llm.buzz's own provider. |
| `x-gateway-route-depth`, `x-gateway-route-reason` | The answering provider's position in the order tried, and why the route was chosen. |

An answer that asks you to wait, a 429 and some 402 and 503 answers, carries `Retry-After`: wait that many seconds before you retry.

## The answer

A non-streamed answer is OpenAI's chat completion object. llm.buzz adds or fixes these parts:

- `choices[0].message` holds `content`, and where they apply `reasoning` (the model's readable reasoning, left out under `reasoning.exclude`), `tool_calls`, and `images`. A model that makes images returns each as a complete base64 image in `images[].image_url.url`.
- `finish_reason` is `tool_calls` when the turn ends with tool calls, `length` when it hit the output limit, and `stop` otherwise, a matched stop sequence included. A stream that fails after it opened finishes with `error`. `native_finish_reason` sits beside it.
- `system_fingerprint` is always `null`.
- `provider` is the display name of the provider that answered, after any failover; `llm.buzz` for llm.buzz's own provider.
- `usage` is described [below](#usage-and-cost). It is left out, never `null`, when the provider reported no counts.
- `x-sgr-ignored-parameters` is a key in the body, not a header. It lists every field llm.buzz dropped and every value it changed, with the reason where there is one, such as `temperature->dropped(unsupported_by_provider)`; an unknown field is named bare. Check it when a setting seems to have no effect.

## Stream

Set `stream: true` and the answer arrives as server-sent events: `data:` lines, each one `chat.completion.chunk`, ending with `data: [DONE]`.

| Chunk | What it carries |
|---|---|
| Every chunk | `provider`, the provider that is answering. |
| Content chunks | Deltas: `delta.content`, `delta.reasoning`, `delta.tool_calls` and `delta.images`, each image complete. |
| Content and finish chunks | `x-sgr-ignored-parameters`, when the request had a dropped field. The usage and error chunks do not carry it. |
| The last chunk | `usage`, with the token counts and `cost`, whenever the counts are known, whatever `stream_options.include_usage` says. |
| A failure after output | One chunk with an `error` object and `finish_reason: "error"`, then `data: [DONE]`. A failure before the stream opens is an ordinary HTTP error. |

While the model is silent, comment lines beginning with `:` keep the connection active; clients skip them. Closing a stream sent without an `Idempotency-Key` stops the request, and the prompt and the output produced up to then are billed. [Streaming](streaming.md) covers the event shapes of all three text APIs.

## Usage and cost

| Field | Value |
|---|---|
| `prompt_tokens`, `completion_tokens`, `total_tokens` | The counts the provider reported for the attempt that answered. |
| `prompt_tokens_details.cached_tokens` | Prompt tokens read from the provider's cache. |
| `prompt_tokens_details.cache_write_tokens` | Prompt tokens written to it. |
| `prompt_tokens_details.cache_write_1h_tokens` | The one-hour part of the writes, only when the provider reports the split. |
| `completion_tokens_details.reasoning_tokens` | Reasoning tokens, where the provider reports them. A stream's usage chunk always has it: `0` when the provider reported no count, and then `usage.unreported_token_details` lists `reasoning_tokens`. |
| `cost` | The request's total cost in US dollars at your rate. |

Cache reads and writes are separate parts of `prompt_tokens`. A cache field appears only when the provider reported it, except that a stream's usage chunk shows `cached_tokens: 0` when the count is unknown. `cost` is a JSON number that includes any billed tool-search rounds, so it can exceed the counts times the listed prices. A failed attempt that another provider replaced adds 0. It is left out when any part of the cost is unknown; a model with no price reports `0`. A replay with the same `Idempotency-Key` returns the original cost. [Usage and limits](usage-and-limits.md) covers the rest, and [Prompt caching](prompt-caching.md#read-the-cache-in-usage) the cache fields.

## Retries

Send an `Idempotency-Key` header and a retry with the same key and the same body never runs the request twice: it returns the first answer with its original cost, or 409 `idempotency_replay_unavailable` when that answer is not stored where the retry arrived. That 409 means the first request ran or is still running, so retry with the same key after a short backoff. The exception is a first call that ended in an error answer: its key answers 409 for good, so send a new key. [What to send next](retries-and-idempotency.md#what-to-send-next) tells the cases apart.

- The same key with another body is 409 `idempotency_conflict`. A changed suffix or `provider` hint counts as another body; a changed `stream_options.include_usage` does not.
- A retry without the key, or with a new one, runs the request again and bills it again.
- A stream you drop while it carries a key keeps running, and a retry with the key joins it instead of starting a second generation. Without a key, two requests with the same prompt are two requests.
- A provider that fails or times out before output reaches you is replaced by the next one, and the failed attempt costs nothing; a provider's own 400 is not tried on another provider. Once output has reached you, the request stays on the provider that sent it.

[Retries and idempotency](retries-and-idempotency.md) explains the rest.

## Errors

Errors come in OpenRouter's envelope. `metadata.gateway_code` is the code to branch on, and `metadata.param` names the field at fault.

```json
{"error": {"code": 400, "message": "...", "metadata": {"error_type": "invalid_request", "gateway_code": "unsupported_parameter", "param": "seed"}}}
```

| Status | Code | When |
|---|---|---|
| 400 | `invalid_parameter` | A field fails validation and `metadata.param` names it, or `body` when the fault spans fields: a missing `messages`, an out-of-range `temperature`, `top_logprobs` above 20 or without `logprobs`, an unknown `verbosity`, a `session_id` over 256 characters, two of `max_tokens`, `max_completion_tokens` and `max_output_tokens`, a `thinking.budget_tokens` under 1,024, `n` other than 1, or `store: true`. Fix the field. |
| 400 | `invalid_json` | The body is not JSON. |
| 400 | `unsupported_parameter` | You sent a refused field: `seed`, `logit_bias`, `audio`, `modalities`, `functions`, `function_call`, `prediction` or `moderation`. Remove it. |
| 400 | `unsupported_capability` | `service_tier` is `flex`, `priority` or `fast`, or no provider of the model offers what the request asks. `metadata.param` names it. |
| 400 | `invalid_request` | A refusal after llm.buzz read the body: the provider's own 400, whose sentence follows `provider rejected the request:`, or another admission refusal. The message says which. |
| 400 | `context_length_exceeded` | The prompt fits no provider's context window. |
| 400 | `web_search_unavailable_in_region` | The request asked for a web search and your organization's requests stay in the EU. |
| 409 | `idempotency_conflict` | The `Idempotency-Key` was used with another body. |
| 409 | `idempotency_replay_unavailable` | The request ran or is running, and its answer is not stored where this retry arrived. Retry with the same key after a short backoff. If the first call ended in an error answer, the key answers 409 for good: send a new key. |
| 413 | `request_too_large` | The body is over 64 MiB. |

A stream that fails after output ends with the error chunk described under [Stream](#stream). [Errors](errors.md) lists every code, including the key, balance, model and provider errors this API shares with the others.

## Limits

- One completion per request: `n` is 1.
- A model's context window is `context_length` on its entry in `GET https://api.llm.buzz/api/v1/models`, and `top_provider.max_completion_tokens` is its output ceiling.
- A request may run for 600 seconds. A stream still generating then ends with a timeout error chunk and `data: [DONE]`, and is billed for the prompt and the output you received. A request that is not streaming answers 504 `deadline_exceeded`.
- A request body over 64 MiB answers 413 `request_too_large`.
- A request makes at most 8 provider calls, every fallback included; `gateway.retry` lowers the ceiling.

[Usage and limits](usage-and-limits.md#limits) collects the limits of every API.

## Related

- [Tools and structured output](tools-and-structured-output.md), [Streaming](streaming.md), [Reasoning](reasoning.md), [Images in requests](images-in-requests.md) and [Prompt caching](prompt-caching.md) cover what the three text APIs share.
- [Responses](responses.md) and [Messages](messages.md) are the other two text APIs; [Choosing an API](choosing-an-api.md) compares them.
- [Coming from OpenRouter](from-openrouter.md) lists every difference from OpenRouter in one place.

# Responses

Responses is OpenAI's API of items in and items out, with the conversation's state kept by llm.buzz, so a follow-up sends only its own input. llm.buzz serves it at `POST https://api.llm.buzz/api/v1/responses` for every text model, and the OpenAI SDK and Codex call it with the base URL changed.

## Send a request

`input` is a string or a list of items, and `instructions` is the system instruction for this request. The OpenAI SDKs give the reply text as `output_text`.

```python tab="Python"
from openai import OpenAI
import os

client = OpenAI(
    base_url="https://api.llm.buzz/api/v1",
    api_key=os.environ["LLM_BUZZ_KEY"],
)
response = client.responses.create(
    model="openai/gpt-5.6-luna",
    instructions="You answer in one short sentence.",
    input="Why do HTTP clients retry with backoff?",
)
print(response.output_text)
```

```ts tab="TypeScript"
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llm.buzz/api/v1",
  apiKey: process.env.LLM_BUZZ_KEY,
});
const response = await client.responses.create({
  model: "openai/gpt-5.6-luna",
  instructions: "You answer in one short sentence.",
  input: "Why do HTTP clients retry with backoff?",
});
console.log(response.output_text);
```

```bash tab="curl"
curl https://api.llm.buzz/api/v1/responses \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.6-luna",
    "instructions": "You answer in one short sentence.",
    "input": "Why do HTTP clients retry with backoff?"
  }'
```

The answer is OpenAI's response object with the additions [below](#the-answer). The model name, and the models you can call, come from `GET https://api.llm.buzz/api/v1/models` ([Choosing an API](choosing-an-api.md#find-a-model-for-an-api)).

## Continue a conversation

llm.buzz keeps each finished turn. Send `previous_response_id` with the `id` of the last answer and only this turn's `input`, and llm.buzz adds the turns before it.

```python tab="Python"
first = client.responses.create(
    model="openai/gpt-5.6-luna",
    instructions="You answer in one short sentence.",
    input="Which port does HTTPS use?",
)
second = client.responses.create(
    model="openai/gpt-5.6-luna",
    instructions="You answer in one short sentence.",
    previous_response_id=first.id,
    input="And plain HTTP?",
)
print(second.output_text)
```

```ts tab="TypeScript"
const first = await client.responses.create({
  model: "openai/gpt-5.6-luna",
  instructions: "You answer in one short sentence.",
  input: "Which port does HTTPS use?",
});
const second = await client.responses.create({
  model: "openai/gpt-5.6-luna",
  instructions: "You answer in one short sentence.",
  previous_response_id: first.id,
  input: "And plain HTTP?",
});
console.log(second.output_text);
```

```bash tab="curl"
# RESPONSE_ID is the "id" of the first answer
curl https://api.llm.buzz/api/v1/responses \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.6-luna",
    "instructions": "You answer in one short sentence.",
    "previous_response_id": "'"$RESPONSE_ID"'",
    "input": "And plain HTTP?"
  }'
```

- A turn is kept for up to 24 hours. It is not a prompt log: the console does not show it, and it is kept even when your organization turned prompt logging off or the request demanded zero retention ([Models and routing](models-and-routing.md#data-policy)). Send `store: false` to keep a turn out.
- A request that continues a turn llm.buzz no longer has answers 400 `previous_response_not_found` with `param: previous_response_id`, and its message asks you to resend the whole conversation in `input`. A turn is gone after 24 hours, when it was sent with `store: false`, and sooner if llm.buzz's store of turns restarts or fills up. The Codex client resends by itself.
- When llm.buzz cannot reach its store of turns, a continuation answers 503 `continuation_store_unavailable`. Wait `Retry-After` seconds and retry. A turn that finishes while the store is unreachable is answered but not kept, so it cannot be continued later.
- Each turn follows only its own `instructions`. Send them again on every turn that should follow them.
- A kept turn belongs to the organization, key identity and model that made it. A continuation that names another model answers 400 `previous_response_not_found`, so continue with the same model.
- A continuation can stay on the provider that served the turn it continues. While that provider can still be used, `provider.order` and the routing preset do not move it, and `only` and `ignore` do not narrow it, so send `provider` preferences with the first turn. A continuation also keeps the caching session its first turn resolved, unless it sends its own `session_id`, `x-session-id` or `prompt_cache_key` ([Prompt caching](prompt-caching.md#keep-a-conversation-on-its-cache)).

## Input items

`input` is a string, which counts as one user message, or a list of items:

- Messages, `{"role": "user", "content": ...}`, with the roles `user`, `assistant`, `system` and `developer`. A `developer` or `system` item is accepted for every model. A leading one is the model's instruction. A later one stays in place when the provider takes one there, and otherwise moves into the user text at the same place, named in the body's `x-sgr-ignored-parameters`, for example `messages.system->folded(system_messages_leading_only)`.
- `function_call` and `function_call_output` items carry a tool loop. Tool call IDs are opaque strings of 1 to 65,536 characters: send each one back complete and unchanged, in the call and in its output. A `function_call_output`'s `output` is a string or a list of text and image parts, and any other part kind is a 400 that names it. [Tools and structured output](tools-and-structured-output.md) covers the loop.
- `reasoning` items from an earlier answer, sent back as they came. Ask for `include: ["reasoning.encrypted_content"]` and a provider that supports it returns the model's reasoning encrypted in its reasoning items. Send those items back with the next request to keep the model's thinking across tool calls in a conversation sent with `store: false`. An encrypted item is tied to the provider account that produced it. When a provider refuses one you sent back, llm.buzz retries without the encrypted reasoning items, keeps the rest of the conversation, and sets the response header `x-gateway-replay-repair: encrypted_reasoning_stripped`.

## Request fields

A field llm.buzz deliberately refuses is a 400 before any provider is called, and a value that fails validation is a 400 `invalid_parameter` with `metadata.param` naming the field. A field llm.buzz drops, such as `messages` or `stop`, which Responses does not know, is named in the body's `x-sgr-ignored-parameters`.

| Field | What llm.buzz does |
|---|---|
| `model` | The model's name, `author/slug`. Four suffixes change one request: `:floor` tries the cheapest provider first, `:nitro` the fastest, `:exacto` is accepted and ignored, and `:online` is accepted and searches nothing (see the `web` plugin below). [Models and routing](models-and-routing.md#suffixes). |
| `input`, `instructions` | `input` is required: a request without it is 400 `invalid_parameter`. [Input items](#input-items). `instructions` is the system instruction of this request alone; an empty or whitespace-only value counts as absent. |
| `previous_response_id`, `store` | `previous_response_id` continues a kept turn: [Continue a conversation](#continue-a-conversation). `store: false` keeps this turn out, so nothing can continue it. |
| `max_output_tokens` | The most output tokens, reasoning included. Without one, llm.buzz does not choose a smaller limit for you. A provider that needs a larger minimum than your limit is skipped, and when none is left the request is refused. |
| `temperature`, `top_p` | Sent to the provider; an out-of-range `temperature` is 400 `invalid_parameter`. A model that takes none, or takes them only at reasoning effort `none`, has the value dropped and named: [Sampling settings a model does not take](api-reference.md#sampling-settings-a-model-does-not-take). |
| `top_k` | Sent to providers that take it. Dropped and named as `top_k->dropped(unsupported_by_provider)` when none does. |
| `seed`, `logit_bias` | Dropped and named. Chat Completions refuses them; Responses does not. |
| `stop`, `n`, `frequency_penalty`, `presence_penalty`, `response_format`, `stream_options`, `audio`, `modalities`, `functions`, `function_call`, `prediction` | Not fields of the Responses API, so dropped and named. Chat Completions refuses `audio`, `modalities`, `functions`, `function_call` and `prediction`; Responses does not. |
| `stream` | `true` streams the answer as server-sent events: [Stream](#stream). |
| `tools`, `tool_choice` | Sent, in OpenAI's shape. Some models cannot be forced to call a tool, and some providers take a tool schema only in part. [Tools and structured output](tools-and-structured-output.md) says what happens to each. |
| `parallel_tool_calls` | Honored on every provider. Where a provider has no such control, `true` is its default behavior and is named `parallel_tool_calls->dropped(provider_default)`, and `false` is enforced by llm.buzz, which returns only the first tool call of a turn and names it `parallel_tool_calls->emulated(serialized_by_gateway)`. |
| `text` | `text.format` with `type: "json_schema"` asks for JSON that matches your schema: [Tools and structured output](tools-and-structured-output.md). `text.format.type` accepts only `text` and `json_schema`; any other, `json_object` included, is 400 `invalid_parameter` naming `text.format.type`. `text.verbosity` is a hint, sent to providers that take it and dropped and named elsewhere. |
| `include` | `reasoning.encrypted_content` returns encrypted reasoning ([Input items](#input-items)). `message.output_text.logprobs`, with `top_logprobs`, asks for token probabilities: llm.buzz sends both to providers that serve the native Responses API, and the provider returns, ignores or refuses them. Any other selector is 400 `invalid_parameter` naming `include`, the six that would echo server-tool output or an input image among them: `code_interpreter_call.outputs`, `computer_call_output.output.image_url`, `file_search_call.results`, `message.input_image.image_url`, `web_search_call.action.sources` and `web_search_call.results`. |
| `reasoning` | `effort`, `summary`, `generate_summary` and `context` are accepted. If you send both `summary` and `generate_summary` they must match, or the request is a 400 `invalid_parameter` naming `reasoning`. `effort` is translated into each provider's own terms ([Reasoning](reasoning.md)). Without a `summary`, llm.buzz asks OpenAI for an automatic one unless the effort is `none`. `reasoning.mode` is 400 `invalid_parameter`. |
| `truncation` | Only `"disabled"`. Any other value is 400 `invalid_parameter`. |
| `client_metadata` | Sent to providers that serve the native Responses API and dropped and named elsewhere. |
| `metadata` | Echoed on the response object: [The answer](#the-answer). |
| `cache_control` | Anthropic's prompt-cache marker, `{"type": "ephemeral"}` with an optional `ttl` of `"5m"` or `"1h"`, at the top level only. A marker on a tool is a 400. Passed on or translated, never added by llm.buzz. [Prompt caching](prompt-caching.md). |
| `prompt_cache_breakpoint` | OpenAI's explicit cache mark, `{"mode": "explicit"}`, on an `input_text` part of a `system`, `developer` or `user` message item or of a `function_call_output`. Sent to OpenAI as given. For Claude it becomes a five-minute `cache_control` marker, named `prompt_cache_breakpoint->translated(cache_control)`. On any other part, or on an assistant message, it is a 400 that names where it was. [Prompt caching](prompt-caching.md#openais-explicit-caching). |
| `prompt_cache_options`, `prompt_cache_retention` | OpenAI's cache mode and retention. Sent to OpenAI only; no other provider receives them, and where no provider of the model takes them, `x-sgr-ignored-parameters` names them as not forwarded. `prompt_cache_retention: "24h"` on a request that demands zero retention is a 400 naming it. [Prompt caching](prompt-caching.md#openais-explicit-caching). |
| `prompt_cache_key` | Names the conversation for caching; `promptCacheKey` works too. It names the session when `session_id` and `x-session-id` are absent. |
| `session_id` | A name for the conversation, 1 to 256 characters, so its turns stay on the provider that holds its cache. An empty or whitespace-only value counts as absent; any other value that is not a string of 1 to 256 characters, such as one over 256 characters, is 400 `invalid_parameter` naming `session_id`. The `x-session-id` header names it too. |
| `provider` | Steers one request: `order`, `only`, `ignore`, `allow_fallbacks`, `sort`, `zdr` and `data_collection`. Other keys, such as `require_parameters`, `quantizations`, `max_price` and `sort.partition`, are accepted and do not change which provider serves. Official SDKs send it through `extra_body`. [The provider object](api-reference.md#the-provider-object). |
| `gateway` | llm.buzz's own per-request bounds, `retry` and `routing.allow_fallbacks`. Never sent to a provider. A warm-up frame on the WebSocket refuses it. [Retries and fallbacks per request](api-reference.md#retries-and-fallbacks-per-request). |
| `service_tier` | `auto` and `default` are served at the standard tier and not sent to the provider. `flex`, `priority` and `fast` are 400 `unsupported_capability` naming `service_tier`, before any provider is called: llm.buzz sells no priority tier, and no model has a price for one today. |
| The `web` plugin | Accepted, and nothing is searched: llm.buzz runs no web search. The request is served without one, nothing is billed for one, and `x-sgr-ignored-parameters` names `web_search->dropped(search_unavailable)`. For an organization whose requests llm.buzz keeps in the EU, the request is 400 `web_search_unavailable_in_region`. |
| other `plugins` entries | Dropped and named by id, as `plugins.response-healing`. |
| `models`, `route`, `transforms`, `usage`, `trace`, `debug` | Dropped and named. A model's own providers fail over through its routing preset, in place of `models`. Usage counts and `usage.cost` come back whether or not you send `usage`. |
| `background`, `context_management`, `conversation`, `max_tool_calls`, `moderation`, `prompt` | Refused: 400 `unsupported_parameter` naming the field. |
| `defer_loading` on a tool | Keeps the tool out of the prompt until a tool search finds it. [Tool search](api-reference.md#tool-search). |

## Headers

You send:

| Header | Use |
|---|---|
| `Authorization` | `Bearer` and your inference key. [Authentication](authentication.md). |
| `Idempotency-Key` | Makes a retry safe: [Retries](#retries). |
| `x-session-id` | Names the conversation, as `session_id` does. |
| `X-Title`, `HTTP-Referer` | Name your app. [App attribution](api-reference.md#app-attribution). |
| `x-client-request-id` | Your own correlation ID. llm.buzz echoes it on the answer. It also names the conversation for caching when you send no `session_id`, `x-session-id`, `prompt_cache_key` or `previous_response_id`; a value that changes on every request gives each request its own cache key, so send one stable value per conversation, or none. |

Every served answer carries:

| Header | Value |
|---|---|
| `x-request-id` | The request's ID. Its generation ID is `gen-` and the same 32 hex digits: [Cost of a finished request](api-reference.md#cost-of-a-finished-request). |
| `x-gateway-alias` | The model name that served. |
| `x-gateway-canonical-model` | The exact model that produced the answer. |
| `x-gateway-provider` | The display name of the provider that answered, `llm.buzz` for llm.buzz's own provider. The response object has no `provider` key, so read it here. |
| `x-gateway-route-depth`, `x-gateway-route-reason` | The answering provider's position in the order tried, and why the route was chosen. |

An answer that asks you to wait, a 429 and some 402 and 503 answers, carries `Retry-After`: wait that many seconds before you retry. An HTTP answer also carries `x-gateway-replay-repair: encrypted_reasoning_stripped` when llm.buzz retried without the encrypted reasoning items you sent back ([Input items](#input-items)).

## The answer

A non-streamed answer is OpenAI's response object. llm.buzz adds or fixes these parts:

- `output` is a list of items: `message` items with `output_text` parts, `function_call` items, and `reasoning` items. A reasoning item's `summary_text` holds the model's readable reasoning where the provider gives one, and its `encrypted_content` holds the encrypted reasoning when you asked for it. [Reasoning](reasoning.md).
- `status` is `completed` for a finished answer and `incomplete` when the output ceiling cut it.
- `instructions`, `metadata` and `tools` echo the request: `instructions` is `null` when you sent none, `metadata` is `{}`, and `tools` keeps your order, with each function tool as served.
- `frequency_penalty` and `presence_penalty` are present, as in OpenAI's response object.
- `usage` is described [below](#usage-and-cost).
- `x-sgr-ignored-parameters` is a key in the body, not a header. It lists every field llm.buzz dropped and every value it changed, with the reason, such as `temperature->dropped(unsupported_by_provider)`. Check it when a setting seems to have no effect.

## Stream

Set `stream: true` and the answer arrives as server-sent events, OpenAI's Responses stream events: `response.created`, `response.in_progress`, then the output events, such as `response.output_text.delta`, and exactly one terminal event. The response object in those events carries the echo of `instructions`, `metadata` and `tools`, and `x-sgr-ignored-parameters`, as a non-streamed answer's does. Closing a stream sent without an `Idempotency-Key` stops the request, and the prompt and the output produced up to then are billed. [Streaming](streaming.md) covers the event shapes of all three text APIs.

## WebSocket

`GET https://api.llm.buzz/api/v1/responses` with a WebSocket upgrade is the transport Codex uses. Authenticate with the same `Authorization` header as an HTTP request.

- You send `response.create` frames. llm.buzz answers each with the response's stream events, one event as JSON in each text frame.
- A failure of the request itself arrives as an OpenAI error frame, `{"type": "error", ...}`, and not as an HTTP status: [Errors](#errors).
- A frame with `generate: false` is a warm-up: llm.buzz answers it without calling a provider.
- llm.buzz checks the key before it accepts the upgrade, and a management key is refused with 403 at that point. A `GET` that is not a well-formed upgrade answers 426 `upgrade_required`; the Codex client falls back to HTTP on it.
- The socket closes at the 600-second deadline ([Limits](#limits)). The requests a socket carries do not appear in prompt logs, and an `Idempotency-Key` on the upgrade request does not apply to the frames.

## Usage and cost

| Field | Value |
|---|---|
| `input_tokens`, `output_tokens` | The counts the provider reported for the attempt that answered. |
| `input_tokens_details.cached_tokens` | Input tokens read from the provider's cache. Always present, `0` when the provider reported none. |
| `input_tokens_details.cache_write_tokens` | Input tokens written to it. Always present, `0` when the provider reported none. |
| `input_tokens_details.cache_write_1h_tokens` | The one-hour part of the writes, only when the provider reports the split. |
| `cost` | The request's total cost in US dollars at your rate. |

Cache reads and writes are separate parts of `input_tokens`. `cost` is a JSON number that includes any billed tool-search rounds, so it can exceed the counts times the listed prices. A failed attempt that another provider replaced adds 0. It is left out when any part of the cost is unknown; a model with no price reports `0`. A replay with the same `Idempotency-Key` returns the original cost. [Usage and limits](usage-and-limits.md) covers the rest, and [Prompt caching](prompt-caching.md#read-the-cache-in-usage) the cache fields.

## Retries

Send an `Idempotency-Key` header on an HTTP request and a retry with the same key and the same body never runs the request twice: it returns the first answer with its original cost, or 409 `idempotency_replay_unavailable` when that answer is not stored where the retry arrived. That 409 means the first request ran or is still running, so retry with the same key after a short backoff. The exception is a first call that ended in an error answer: its key answers 409 for good, so send a new key. [What to send next](retries-and-idempotency.md#what-to-send-next) tells the cases apart.

- The same key with another body is 409 `idempotency_conflict`. A changed suffix or `provider` hint counts as another body.
- A retry without the key, or with a new one, runs the request again and bills it again.
- A provider that fails or times out before output reaches you is replaced by the next one, and the failed attempt costs nothing; a provider's own 400 is not tried on another provider. Once output has reached you, the request stays on the provider that sent it.
- The OpenAI SDK also retries failed requests on its own, and each retry is a separate request. Set `max_retries=0` when your application must bound the work of one operation.

[Retries and idempotency](retries-and-idempotency.md) explains the rest.

## Errors

Over HTTP, errors come in OpenRouter's envelope. `metadata.gateway_code` is the code to branch on, and `metadata.param` names the field at fault.

```json
{"error": {"code": 400, "message": "...", "metadata": {"error_type": "invalid_request", "gateway_code": "unsupported_parameter", "param": "background"}}}
```

On the WebSocket, a failure of the request arrives as OpenAI's error frame, with the status in the frame, for example when the key's spending limit is reached:

```json
{"type": "error", "status": 403, "error": {"message": "Key limit exceeded", "type": "permission_error", "param": null, "code": "key_limit_exceeded"}}
```

| Status | Code | When |
|---|---|---|
| 400 | `invalid_parameter` | A field fails validation and `metadata.param` names it, or `body` when the fault spans fields: a missing `input`, an out-of-range `temperature`, a `session_id` over 256 characters, a `truncation` other than `"disabled"`, `reasoning.mode`, an `include` selector other than `reasoning.encrypted_content` and `message.output_text.logprobs`, or a `text.format.type` other than `text` or `json_schema`. Fix the field. |
| 400 | `invalid_json` | The body is not JSON. |
| 400 | `unsupported_parameter` | You sent a refused field: `background`, `context_management`, `conversation`, `max_tool_calls`, `moderation` or `prompt`. Remove it. |
| 400 | `previous_response_not_found` | `previous_response_id` names a turn llm.buzz does not have: expired, sent with `store: false`, lost, or made by another model. Resend the whole conversation in `input`. |
| 400 | `unsupported_capability` | `service_tier` is `flex`, `priority` or `fast`, or no provider of the model offers what the request asks. `metadata.param` names it. |
| 400 | `invalid_request` | A refusal after llm.buzz read the body: the provider's own 400, whose sentence follows `provider rejected the request:`, or another admission refusal. The message says which. |
| 400 | `context_length_exceeded` | The prompt fits no provider's context window. |
| 400 | `web_search_unavailable_in_region` | The request asked for a web search and your organization's requests stay in the EU. |
| 409 | `idempotency_conflict` | The `Idempotency-Key` was used with another body. |
| 409 | `idempotency_replay_unavailable` | The request ran or is running, and its answer is not stored where this retry arrived. Retry with the same key after a short backoff. If the first call ended in an error answer, the key answers 409 for good: send a new key. |
| 413 | `request_too_large` | The body is over 64 MiB. |
| 426 | `upgrade_required` | A `GET` to `/responses` that is not a WebSocket upgrade. Open a WebSocket, or `POST` the request. |
| 503 | `continuation_store_unavailable` | llm.buzz cannot reach its store of turns. Retry after `Retry-After` seconds. |

[Errors](errors.md) lists every code, including the key, balance, model and provider errors this API shares with the others.

## Limits

- A model's context window is `context_length` on its entry in `GET https://api.llm.buzz/api/v1/models`, and `top_provider.max_completion_tokens` is its output ceiling.
- A request may run for 600 seconds. A stream still generating then ends with a timeout error and is billed for the prompt and the output you received, and a WebSocket closes. A request that is not streaming answers 504 `deadline_exceeded`.
- A request body over 64 MiB answers 413 `request_too_large`.
- A request makes at most 8 provider calls, every fallback included; `gateway.retry` lowers the ceiling.
- A kept turn can be continued for up to 24 hours.

[Usage and limits](usage-and-limits.md#limits) collects the limits of every API.

## Related

- [Tools and structured output](tools-and-structured-output.md), [Streaming](streaming.md), [Reasoning](reasoning.md), [Images in requests](images-in-requests.md) and [Prompt caching](prompt-caching.md) cover what the three text APIs share.
- [Chat Completions](chat-completions.md) and [Messages](messages.md) are the other two text APIs; [Choosing an API](choosing-an-api.md) compares them.
- [Coming from OpenRouter](from-openrouter.md) lists every difference from OpenRouter in one place.

# Messages

Messages is Anthropic's API. llm.buzz serves it at `POST https://api.llm.buzz/api/v1/messages` for every text model, Claude or not, so the Anthropic SDK and the tools built on it work with the base URL alone.

## Send a request

Point the Anthropic SDK at `https://api.llm.buzz/api`, without `/v1`: the SDK adds `/v1/messages` itself. `max_tokens` is required. Claude Code takes the same base URL: [Authentication](authentication.md#send-the-key).

```python tab="Python"
from anthropic import Anthropic
import os

client = Anthropic(
    base_url="https://api.llm.buzz/api",
    api_key=os.environ["LLM_BUZZ_KEY"],
)
message = client.messages.create(
    model="anthropic/claude-sonnet-5.5",
    max_tokens=2048,
    system="You answer in one short sentence.",
    messages=[{"role": "user", "content": "Why do HTTP clients retry with backoff?"}],
)
for block in message.content:
    if block.type == "text":
        print(block.text)
```

```ts tab="TypeScript"
import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({
  baseURL: "https://api.llm.buzz/api",
  apiKey: process.env.LLM_BUZZ_KEY,
});
const message = await client.messages.create({
  model: "anthropic/claude-sonnet-5.5",
  max_tokens: 2048,
  system: "You answer in one short sentence.",
  messages: [{ role: "user", content: "Why do HTTP clients retry with backoff?" }],
});
for (const block of message.content) {
  if (block.type === "text") console.log(block.text);
}
```

```bash tab="curl"
curl https://api.llm.buzz/api/v1/messages \
  -H "x-api-key: $LLM_BUZZ_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-5.5",
    "max_tokens": 2048,
    "system": "You answer in one short sentence.",
    "messages": [{"role": "user", "content": "Why do HTTP clients retry with backoff?"}]
  }'
```

The loop reads the text blocks because a model that thinks puts a `thinking` block first. The fields llm.buzz adds to Anthropic's request, `provider`, `gateway` and `session_id`, go through `extra_body` in the SDKs, for example `extra_body={"provider": {"data_collection": "deny"}}`. The model name, and the models you can call, come from `GET https://api.llm.buzz/api/v1/models` ([Choosing an API](choosing-an-api.md#find-a-model-for-an-api)).

## Request fields

A field Messages does not know, such as `seed`, is dropped, and the body's `x-sgr-ignored-parameters` names it, on `message_start` and on the final message. A field llm.buzz deliberately refuses is a 400 before any provider is called, in Anthropic's `invalid_request_error`.

| Field | What llm.buzz does |
|---|---|
| `model` | The model's name, `author/slug`. `:floor` tries the cheapest provider first and `:nitro` the fastest, and `:exacto` is accepted and ignored. `:online` is accepted and searches nothing, and `x-sgr-ignored-parameters` names it `model:online`. [Models and routing](models-and-routing.md#suffixes). |
| `max_tokens` | Required. The most output tokens, thinking included. A provider that needs a larger minimum than your limit is skipped, and when none is left the request is refused. |
| `system` | The system instruction: a string, or a list of text blocks. |
| `messages` | The conversation, with the roles `user` and `assistant`. A block is `text`; `image`, from base64 or a URL; `document`, only for a model that reads PDFs, which no model on llm.buzz does today; `tool_use`; `tool_result`, whose content may hold text and images; or `thinking` and `redacted_thinking`, sent back unchanged. A model that cannot read an attachment refuses the request before any provider is called. [Images in requests](images-in-requests.md). |
| `stop_sequences` | Sent to providers that take them. Where a provider has no stop field, llm.buzz cuts the answer at the first match itself and ends it with `stop_reason: stop_sequence`, the matched string in `stop_sequence`. |
| `temperature`, `top_p`, `top_k` | Sent to the provider. A model that takes none has the value dropped and named: [Sampling settings a model does not take](api-reference.md#sampling-settings-a-model-does-not-take). |
| `stream` | `true` streams the answer as server-sent events: [Stream](#stream). |
| `tools`, `tool_choice` | Sent, in Anthropic's shape. Some models cannot be forced to call a tool, and some providers take a tool schema only in part. [Tools and structured output](tools-and-structured-output.md) says what happens to each. |
| `thinking`, `reasoning` | How hard the model thinks, and whether you see it. [Thinking](#thinking). |
| `output_config` | `output_config.effort` sets the reasoning effort, and every provider receives it in its own terms: [Thinking](#thinking). `output_config.format` asks for structured output. An `output_config` that holds more than `effort`, such as a `format`, is sent as you wrote it when every provider that may serve the request takes Anthropic's API. Otherwise llm.buzz drops the object and names `output_config` in `x-sgr-ignored-parameters`, and its `effort` still applies. [Tools and structured output](tools-and-structured-output.md#structured-output). |
| `cache_control` | Anthropic's prompt-cache marker, taken as the Messages API defines it, at every place it allows, the top level included. Passed on or translated, never added by llm.buzz. [Prompt caching](prompt-caching.md). |
| `context_management` | Sent where the provider takes it, with the `anthropic-beta` token it needs, and dropped and named elsewhere. |
| `inference_geo`, `speed`, `diagnostics` | Sent where the provider takes them, and dropped and named elsewhere. |
| `session_id` | A name for the conversation, 1 to 256 characters, so its turns stay on the provider that holds its cache. An empty or whitespace-only value counts as absent; any other value that is not a string of 1 to 256 characters, such as one over 256 characters, is a 400 naming `session_id`. The `x-session-id` header names it too. |
| `provider` | Steers one request: `order`, `only`, `ignore`, `allow_fallbacks`, `sort`, `zdr` and `data_collection`. Other keys, such as `require_parameters`, `quantizations`, `max_price` and `sort.partition`, are accepted and do not change which provider serves. [The provider object](api-reference.md#the-provider-object). |
| `gateway` | llm.buzz's own per-request bounds, `retry` and `routing.allow_fallbacks`. Never sent to a provider. [Retries and fallbacks per request](api-reference.md#retries-and-fallbacks-per-request). |
| `plugins`, `web_search_options` | Dropped whole and named: the whole `plugins` field is named `plugins`, not `plugins.<id>` as on Chat Completions and Responses. llm.buzz runs no web search, so nothing is searched. |
| `models`, `route`, `transforms`, `usage`, `trace`, `debug` | Dropped and named. A model's own providers fail over through its routing preset, in place of `models`. Usage counts and `usage.cost` come back whether or not you send `usage`. |
| `seed`, `logit_bias`, `response_format` | Dropped and named. A Chat `response_format` is not applied on Messages: ask for structured output with `output_config.format`. |
| `defer_loading` on a tool | Keeps the tool out of the prompt until a tool search finds it. [Tool search](api-reference.md#tool-search). |
| Refused fields, including `betas`, `container`, `fallbacks`, `fallback_credit_token`, `mcp_servers`, `service_tier`, `thread` and `user_profile_id` | 400 `invalid_request_error` naming the field. Send the `anthropic-beta` header in place of `betas`. |

## Thinking

You can ask for thinking three ways, and llm.buzz translates each into the terms of the provider that serves the request.

- **`thinking`** takes Anthropic's forms: `adaptive`, `disabled`, and `enabled` with a `budget_tokens`. Every Claude model llm.buzz lists today is in an adaptive-only family, and those families refuse a budget. A `thinking` object with `budget_tokens` answers 400 `invalid_request_error` naming `thinking.budget_tokens`, and a bare `{"type": "enabled"}` runs adaptive thinking, named `thinking.type->adaptive` in `x-sgr-ignored-parameters`. Use `adaptive`, and set the depth with `output_config.effort`.
- A model from another provider gets `thinking` translated. A bare `enabled` or `adaptive` reasons at the model's default effort, and at medium when the model names none. A `budget_tokens` goes to providers that have a budget field, and is refused before any provider is called where none has one.
- For a Claude model that takes a budget, which none on llm.buzz does today, a bare `{"type": "enabled"}` takes half of the output limit as its budget, and `budget_tokens` must be below `max_tokens` (at or above it is a 400 naming `thinking.budget_tokens`).
- **`reasoning`**, OpenRouter's object, is accepted too: `effort` or `max_tokens`, plus `enabled` and `exclude`. Its fields must agree: an effort tier or a budget beside `enabled: false`, or `effort: "none"` beside `enabled: true`, is a 400 `invalid_request_error` whose message ends `(param: reasoning.enabled)`.
- **`output_config.effort`** is a reasoning control that every provider receives in its own terms. A route that cannot serve the effort refuses the request and names `output_config.effort`.

Thinking off is not always available. `thinking: {"type": "disabled"}` is refused before any provider is called for Claude Opus 5.5, Sonnet 5.5, Fable 5 and Fable 5.1, at every effort level. Leave `thinking` out, or use `adaptive` and choose an effort. Sonnet 5.5 takes `{"type": "between_tools"}` in place of `disabled`: it turns off up-front thinking, works with an `output_config.effort` of `low`, `medium` or `high`, and takes no other `thinking` field. `reasoning.exclude: true` keeps reasoning text out of the answer while the model still thinks and bills for it, but Anthropic's own signed thinking blocks stay, because replaying them depends on them.

A conversation can move between models. Claude's signed thinking blocks go back to Claude. Another provider gets the turn without them, and `x-sgr-ignored-parameters` names it `messages.thinking->dropped(unsupported_by_provider)`.

A `max_tokens` under 1,024, with no thinking or effort setting of your own, on a model whose providers all reason by default and offer a `none` effort, is sent at effort `none`, named `reasoning_effort->none(max_tokens_headroom)`. The reply is text, not thinking cut off at the ceiling. [Reasoning](reasoning.md) has the rest.

## Headers

You send:

| Header | Use |
|---|---|
| `x-api-key` or `Authorization` | Your inference key. The Anthropic SDK sends `x-api-key`; `Authorization: Bearer` carries the same key. [Authentication](authentication.md). |
| `anthropic-version` | The SDK sends it, and llm.buzz accepts it. |
| `anthropic-beta` | Anthropic's beta tokens. A token llm.buzz does not pass on is dropped, and `x-sgr-ignored-parameters` names it `anthropic-beta.<token>`; the request still serves. |
| `x-session-id` | Names the conversation, as `session_id` does. |
| `X-Title`, `HTTP-Referer` | Name your app. [App attribution](api-reference.md#app-attribution). |

Messages has no `Idempotency-Key`: see [Retries](#retries). Every answer carries:

| Header | Value |
|---|---|
| `x-request-id` | The request's ID. Its generation ID is `gen-` and the same 32 hex digits: [Cost of a finished request](api-reference.md#cost-of-a-finished-request). |
| `x-gateway-alias` | The model name that served. |
| `x-gateway-canonical-model` | The exact model that produced the answer. |
| `x-gateway-provider` | The display name of the provider that answered, `llm.buzz` for llm.buzz's own provider. The message has no `provider` key, so read it here. |
| `x-gateway-route-depth`, `x-gateway-route-reason` | The answering provider's position in the order tried, and why the route was chosen. |

An answer that asks you to wait, a 429 and some 402 and 503 answers, carries `Retry-After`: wait that many seconds before you retry.

## Count tokens

`POST https://api.llm.buzz/api/v1/messages/count_tokens` takes a Messages request body and answers how many input tokens it comes to. It needs no `max_tokens`; one is accepted and changes nothing. `gateway` is refused with a 400 `invalid_request_error` naming it.

```bash
curl https://api.llm.buzz/api/v1/messages/count_tokens \
  -H "x-api-key: $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-5.5",
    "messages": [{"role": "user", "content": "Why do HTTP clients retry with backoff?"}]
  }'
```

The answer is `{"input_tokens": N}`. The count is llm.buzz's own estimate, because no provider counts tokens for it, and the body says so with `input_tokens->estimated(gateway_tokenizer)` in `x-sgr-ignored-parameters`. It is free, and a key whose spending limit is reached can still call it. It answers the same 404 as a Messages call for a model none of whose providers meets your data policy ([Models and routing](models-and-routing.md#data-policy)).

## The answer

A non-streamed answer is Anthropic's message object. llm.buzz adds or fixes these parts:

- `content` holds `text` blocks, `tool_use` blocks, and the model's reasoning as `thinking` blocks: Anthropic's own, signed, when Anthropic serves the request, and one unsigned block with the readable reasoning from other providers. A tool turn of some models also ends with a `redacted_thinking` block. Send every thinking block back unchanged with the assistant turn. `reasoning.exclude` withholds the unsigned one.
- `stop_reason` is `end_turn` for a finished answer, `tool_use` when the model called tools, `max_tokens` when the output ceiling cut it, and `stop_sequence` when a stop sequence you sent matched, with the matched string in `stop_sequence`.
- `usage` is described [below](#usage-and-cost).
- `x-sgr-ignored-parameters` is a key in the body, not a header. It lists every field llm.buzz dropped and every value it changed, with the reason, such as `temperature->dropped(unsupported_by_provider)`. It is on the final message and on `message_start`. Check it when a setting seems to have no effect.

## Stream

Set `stream: true` and the answer arrives as Anthropic's events.

| Event | What it carries |
|---|---|
| `message_start`, then one `ping` | The message so far, with `usage` and `x-sgr-ignored-parameters`. |
| `content_block_start`, `content_block_delta`, `content_block_stop` | The content blocks: text, tool input, thinking. |
| `message_delta` | The mapped `stop_reason`, the final `usage` and the `cost`. |
| `message_stop` | The end of a finished answer. |
| `error` | One last event when a failure follows the first output, in place of `message_stop`. A failure before the stream opens is an ordinary HTTP error. |

Closing a Messages stream stops the request, and the prompt and the output produced up to then are billed. While the model is silent, comment lines beginning with `:` keep the connection active; clients skip them. [Streaming](streaming.md) covers the event shapes of all three text APIs.

## Usage and cost

| Field | Value |
|---|---|
| `input_tokens` | Uncached input tokens only. |
| `cache_creation_input_tokens` | Input tokens written to the provider's cache. |
| `cache_read_input_tokens` | Input tokens read from it. |
| `output_tokens` | Output tokens. |
| `cache_creation` | Anthropic's split of the writes, `ephemeral_5m_input_tokens` and `ephemeral_1h_input_tokens`, present when the split is known. |
| `cost` | The request's total cost in US dollars at your rate. On the final message and on `message_delta`, never on `message_start`. |

The four counts are always present, `0` when the provider reported none. The usage of `message_start` is a figure taken before the provider answers: Anthropic's own start counts when Anthropic serves the request, and llm.buzz's estimate of the prompt for any other provider, with `output_tokens` of 1 and cache counts of 0. `message_delta` carries the final counts, which replace it.

The Anthropic Python SDK does not copy `cost` into the message it assembles from a stream. Read it from the `message_delta` event, or from the answer of a request that does not stream:

```python
stream = client.messages.create(
    model="anthropic/claude-sonnet-5.5",
    max_tokens=2048,
    messages=[{"role": "user", "content": "Why do HTTP clients retry with backoff?"}],
    stream=True,
)
for event in stream:
    if event.type == "message_delta":
        print(event.usage.model_extra.get("cost"))
```

`message.usage.model_extra["cost"]` is the same value on a non-streamed message. `cost` is a JSON number that includes any billed tool-search rounds, so it can exceed the counts times the listed prices. A failed attempt that another provider replaced adds 0. It is left out when any part of the cost is unknown; a model with no price reports `0`. [Usage and limits](usage-and-limits.md) covers the rest, and [Prompt caching](prompt-caching.md#read-the-cache-in-usage) the cache fields.

## Retries

The Messages API defines no idempotency header, so llm.buzz applies none: a retry is a separate request that runs, and bills, again.

- The Anthropic SDK also retries failed requests on its own, and each retry is a separate request. Set `max_retries=0` when your application must bound the work of one operation.
- A provider that fails or times out before output reaches you is replaced by the next one, and the failed attempt costs nothing; a provider's own 400 is not tried on another provider. Once output has reached you, the request stays on the provider that sent it.

[Retries and idempotency](retries-and-idempotency.md) explains the rest.

## Errors

Errors come in Anthropic's body, `{"type": "error", "error": {"type": "...", "message": "..."}}`, and the message says what to do. A named field is appended to the message as `(param: NAME)`, and a refusal by the model adds `refusal_reason` to `error`. `error.type` follows the status:

| Status | `error.type` | When |
|---|---|---|
| 400 | `invalid_request_error` | You sent a refused field, such as `service_tier`; a field is missing or fails validation, `max_tokens` included; thinking is off on a model that cannot turn it off; a `budget_tokens` on a model that takes none; a route cannot serve `output_config.effort`; or the prompt fits no provider's context window. The message names the field and says what to change. |
| 401 | `authentication_error` | The key is missing, unknown, expired, revoked or disabled. |
| 402 | `billing_error` | Your balance is spent and the model is not free, or requests already running hold what it leaves. |
| 403 | `permission_error` | The key's spending limit is reached, or a management key called Messages. |
| 404 | `not_found_error` | The model is unknown, unavailable, has no provider that meets your data policy, or has no EU provider for an organization whose requests stay in the EU. |
| 413 | `request_too_large` | The body is over 64 MiB. |
| 429 | `rate_limit_error` | A provider is throttling, a route is unavailable, or llm.buzz is overloaded. Wait `Retry-After` seconds, then retry. |
| 503 | `overloaded_error` | Every provider is down, or llm.buzz is restarting. Retry with backoff, and after `Retry-After` seconds when the answer carries it. |
| 500, 502, 504 | `api_error` | llm.buzz or a provider failed, or the request ran past 600 seconds: retry with backoff. A 502 can also mean a provider's answer was over llm.buzz's response limit: ask for less output. |

A stream that fails after output ends with an `error` event, described under [Stream](#stream). [Errors](errors.md) lists every code, including the key, balance, model and provider errors this API shares with the others.

## Limits

- `max_tokens` is required. A model's context window is `context_length` on its entry in `GET https://api.llm.buzz/api/v1/models`, and `top_provider.max_completion_tokens` is its output ceiling.
- A request may run for 600 seconds. A stream still generating then ends with a timeout error and is billed for the prompt and the output you received. A request that is not streaming answers 504.
- A request body over 64 MiB answers 413 `request_too_large`.
- A request makes at most 8 provider calls, every fallback included; `gateway.retry` lowers the ceiling.

[Usage and limits](usage-and-limits.md#limits) collects the limits of every API.

## Related

- [Tools and structured output](tools-and-structured-output.md), [Streaming](streaming.md), [Reasoning](reasoning.md), [Images in requests](images-in-requests.md) and [Prompt caching](prompt-caching.md) cover what the three text APIs share.
- [Chat Completions](chat-completions.md) and [Responses](responses.md) are the other two text APIs; [Choosing an API](choosing-an-api.md) compares them.
- [Coming from OpenRouter](from-openrouter.md) lists every difference from OpenRouter in one place.

# Embeddings

llm.buzz serves embeddings models on the OpenAI Embeddings API, so the OpenAI and OpenRouter SDKs reach them with the base URL alone. Chat Completions, Responses and Messages refuse an embeddings model with a 400 on `model` that names `POST /api/v1/embeddings`.

The route takes your key as a bearer token, as every other model route does ([Authentication](authentication.md)).

## Find an embeddings model

`GET https://api.llm.buzz/api/v1/models` leaves embeddings models out by default: `output_modalities=embeddings` lists them, and `output_modalities=all` lists every model. `GET https://api.llm.buzz/api/v1/embeddings/models` is the same list with that filter fixed, paged with `limit` and `offset`. With your key, both list the embeddings models your key can call; without a key, the public ones.

```bash
curl https://api.llm.buzz/api/v1/embeddings/models
```

An embeddings model lists as `text->embeddings` with its context window. `pricing.prompt` is its input price in US dollars per token, at your rate, and `pricing.completion` is `"0"`: an embeddings call has no output to bill.

The [Models page](/models) lists embeddings models too, and its Output filter picks them out. Each one's page shows its input price per million tokens and its Dimensions: the length of the vectors a request without `dimensions` gets.

## Create embeddings

`POST https://api.llm.buzz/api/v1/embeddings` takes the OpenAI Embeddings API's body and answers in its shape.

```python tab="Python"
from openai import OpenAI
import os

client = OpenAI(
    base_url="https://api.llm.buzz/api/v1",
    api_key=os.environ["LLM_BUZZ_KEY"],
)
result = client.embeddings.create(
    model="qwen/qwen3-embedding-8b",
    input=["first passage", "second passage"],
)
print(len(result.data), len(result.data[0].embedding))
```

```bash tab="curl"
curl https://api.llm.buzz/api/v1/embeddings \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "qwen/qwen3-embedding-8b", "input": ["first passage", "second passage"]}'
```

| Field | Rule |
|---|---|
| `model` | Required: an embeddings model. Another model is a 400 `unsupported_capability` on `model`. |
| `input` | Required: a string, a list of strings, a list of token IDs, or a list of such lists, none of them empty. A flat list of token IDs gets one vector. Token IDs go to the provider unchanged, so they must match the model's tokenizer. |
| `encoding_format` | `float` or `base64`: how the vectors come back. |
| `dimensions` | Sent to the provider as you give it. Without it, the vectors have the model's Dimensions. |
| `provider` | Steers the request as on chat; see [Provider preferences](models-and-routing.md#provider-preferences). |
| `user` | Recorded as the end-user label, and not sent to the provider. |
| `stream` | Leave it out or send `false`. Embeddings are never streamed, and `true` is a 400. |

Fields the route does not know are dropped. The route keeps no answers and ignores an `Idempotency-Key` header, so a retry is a new call, billed again.

The answer:

```json
{"object": "list", "data": [{"object": "embedding", "index": 0, "embedding": [0.0123, -0.0456]}],
 "model": "qwen/qwen3-embedding-8b", "provider": "DeepInfra",
 "usage": {"prompt_tokens": 4, "total_tokens": 4, "cost": 4e-8}}
```

The vectors are the provider's own, one per input. `usage.prompt_tokens` is the input tokens the provider counted, which the call bills, and `usage.cost` is the call's price in US dollars at your rate. `provider` names the provider that answered.

## Billing

An embeddings call bills the input tokens its provider reports, `usage.prompt_tokens`, at that provider's input price and your rate. It has no output price. Your credit, budgets and key limits apply as on chat; [Errors](errors.md) lists the 402 and 403 answers.

- A call that fails over to another provider bills only the provider that answered.
- A call bills nothing when its provider answered an error, or never received the request (the connection was refused, or DNS or TLS failed).
- A timeout, a network fault after the request was sent, or your client disconnecting bills what llm.buzz reserved for the call before calling the provider, since the provider may have done the work and billed it.

## Not supported

Image or other multimodal input to an embeddings model, streaming, and embeddings models in the console's Playground.

# Images

llm.buzz serves image generation models on OpenRouter's Image API, `POST /api/v1/images`, and on the OpenAI Images API, `POST /api/v1/images/generations`, so the OpenRouter and OpenAI SDKs reach them with the base URL alone. An answer is never streamed, and every image comes back as base64 in `b64_json`.

Both routes serve the same models, each with the same sizes and settings, and take your key as a bearer token ([Authentication](authentication.md)). Chat Completions, Responses and Messages refuse an image model with a 400 on `model` that names `POST /api/v1/images`. A chat model that draws returns its images in a Chat Completions answer instead, as the [API reference](api-reference.md#responses) shows; Responses and Messages refuse such a model.

## Find an image model

`GET https://api.llm.buzz/api/v1/models?output_modalities=image` lists the models that make images: the image generation models these routes serve, and chat models that return images in a chat answer. An image generation model lists as `text->image`, with a `context_length` of 32,000, the most characters its prompt takes. The [Models page](/models)'s Output filter picks out the models that make images, and an image model's page shows its price: per image, or per million input and output tokens.

In this list, `pricing.completion` of an image model is `"0"`, and its image price is in other keys. A model priced per token lists its input price as `pricing.prompt` and its output price as `pricing.image_output`, in US dollars per token. A model priced per image lists `pricing.image_token` and `pricing.image_output`, its price per image token; the price of one image is the `cost_usd` of its endpoints listing, below.

With your key, `GET https://api.llm.buzz/api/v1/images/models` lists exactly the image models your key can call, in OpenRouter's image model shape. Each one's `supported_parameters` names every field the model takes with its values: its sizes, aspect ratios, resolutions, qualities, backgrounds and formats, and the range of `n` up to its most images per request. `size` lists the model's sizes, where OpenRouter lists a boolean.

```bash
curl https://api.llm.buzz/api/v1/images/models \
  -H "Authorization: Bearer $LLM_BUZZ_KEY"
```

`GET https://api.llm.buzz/api/v1/images/models/{author}/{slug}/endpoints`, or `/images/models/{name}/endpoints` for a name with no `/`, lists the providers that may serve you the model, each with its `provider_tag`, its `supported_parameters` and its `pricing` at your rate. A model priced per image has one line, the price of one image:

```json
{"billable": "output_image", "unit": "image", "cost_usd": 0.03}
```

A model priced per token has an `input_text` and an `output_image` line, in US dollars per token. A model that makes no images answers 404, naming `GET /api/v1/images/models`. Both image model routes need a key: without one, they answer 401.

## Generate an image

```bash
curl https://api.llm.buzz/api/v1/images \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "qwen/qwen-image-3", "prompt": "A line drawing of a beehive", "aspect_ratio": "16:9"}'
```

With the OpenAI SDK, call the OpenAI route. `model` and `prompt` are enough: the model's first size is its default.

```python
import base64
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.llm.buzz/api/v1",
    api_key=os.environ["LLM_BUZZ_KEY"],
)
result = client.images.generate(
    model="qwen/qwen-image-3",
    prompt="A line drawing of a beehive",
)
# PNG, JPEG or WebP: output_format names which when the answer carries it; name the file to match.
with open("beehive.png", "wb") as file:
    file.write(base64.b64decode(result.data[0].b64_json))
```

The fields of `POST https://api.llm.buzz/api/v1/images`:

| Field | Rule |
|---|---|
| `model` | Required: an image model. Another model is a 400 on `model` that names `GET /api/v1/images/models` and the route you called. `:floor` and `:nitro` choose the provider order as on chat. |
| `prompt` | Required: 1 to 32,000 characters. |
| `n` | 1 to 10 images, 1 when omitted, and at most the model's most images per request. |
| `size` | One of the model's sizes, `WIDTHxHEIGHT` or `auto`, or a resolution tier such as `1K` or `2K`, read as `resolution`. |
| `aspect_ratio`, `resolution` | Choose one of the model's sizes, as below. |
| `quality`, `background`, `output_format` | One of the values the model takes. A model that takes none ignores the field and never sends it. `output_format` is `png`, `jpeg` or `webp`. |
| `output_compression` (0 to 100), `seed` | Sent only to a model that takes them, else ignored. |
| `provider` | Steers the request as on chat: `order`, `only`, `ignore`, `allow_fallbacks`, `sort`, `zdr` and `data_collection`; see [Provider preferences](models-and-routing.md#provider-preferences). |
| `user`, `session_id`, `trace` | Accepted. `user` is recorded as the end-user label; none of them is sent to the provider. |

`POST https://api.llm.buzz/api/v1/images/generations` takes the OpenAI SDK's fields under the same rules: `model`, `prompt`, `n`, `size`, `quality`, `background`, `output_format`, `output_compression`, `moderation`, `user`, and `provider` for routing. `moderation` is sent only to a model that takes it, and `style` is accepted and never sent. The OpenAI SDK's `standard` and `hd` qualities match none of a model's qualities, so send one that `GET /api/v1/images/models` lists. Both routes drop fields they do not know.

### How a size is chosen

A request resolves to exactly one of the model's sizes:

- An explicit `size` must be one of the model's sizes, and an `aspect_ratio` or `resolution` beside it must be that size's own.
- Otherwise `resolution` keeps the sizes of that tier, and `aspect_ratio` picks the first of them with that ratio (`auto` picks the model's `auto` size, else its first).
- With neither, the model's first size, its default.

A size, ratio or tier the model does not take is a 400 that names the field and the values it takes, such as "qwen/qwen-image-3 serves aspect_ratio 1:1, 4:3, 3:4, 16:9, 9:16, 3:2, 2:3; resend with one of them." Only what the model takes reaches the provider: `model`, `prompt`, `n`, the size (left out for `auto`), and the `quality`, `background`, `output_format`, `output_compression`, `seed` and `moderation` the model takes.

### The answer

```json
{"created": 1792900000, "data": [{"b64_json": "iVBORw0...", "media_type": "image/png"}],
 "usage": {"prompt_tokens": 0, "completion_tokens": 4175, "total_tokens": 4175, "cost": 0.03}}
```

Every image is PNG, JPEG or WebP, in base64 in `b64_json`, and `media_type` names its format from its bytes. `usage.cost` is what the call billed, in US dollars at your rate. A model priced per image counts each image as a fixed number of output tokens, so its `completion_tokens` is that count per image and its `prompt_tokens` is 0: 4,175 tokens an image on most models, the count OpenRouter's Image API reports for one image, and 1,120 on the Gemini image models. The OpenAI route answers in OpenAI's shape: `data` with `b64_json` and any `revised_prompt`, the `background`, `output_format`, `quality` and `size` the provider reports, and `usage` in OpenAI's names (`input_tokens`, `output_tokens`, `total_tokens`, `input_tokens_details` and `cost`). Both carry an `X-Generation-Id` header, which names the call's generation record: `GET /api/v1/generation?id=` answers what it cost ([Cost and the generation record](audio.md#cost-and-the-generation-record)).

A provider may return fewer images than `n`, as on OpenRouter: you get and pay for the images returned. A high-quality image can take minutes, and each provider gets at least 300 seconds to answer, so give your client a long timeout.

A thinking image model, such as `google/gemini-3-pro-image`, can draw up to two interim images before its final one, and a provider may send them ahead of it. llm.buzz then serves the last `n` images of the answer, and bills only those.

## Refusals

Each is a 400 before any provider is called, with `error.metadata.param` naming the field, and bills nothing:

| Request | Answer |
|---|---|
| A chat or embeddings model on either image route | `unsupported_capability` on `model`, naming the route you called: "The model 'deepseek/deepseek-v4.1-flash' does not generate images. Choose an image model from GET /api/v1/images/models and resend the request to POST /api/v1/images." |
| An image model on Chat Completions, Responses or Messages | `model`: "The model 'qwen/qwen-image-3' generates images and serves no chat request; send it to POST /api/v1/images." |
| A `size`, `aspect_ratio` or `resolution` the model does not take | The field and the values the model takes. |
| A `quality`, `background` or `output_format` value outside the ones the model takes | The field and the values the model takes. A model that takes none of a field ignores it. |
| `n` above the model's most images | `n`, with the most: "qwen/qwen-image-3 makes at most 6 images per request; resend with n at most 6." |
| `input_references`, `stream: true`, `output_format: "svg"`, `response_format: "url"` | The field and what to send: a text prompt only, no `stream`, `png`, `jpeg` or `webp`, and no `response_format`, since every image is `b64_json`. |

## Billing

An image model bills in one of two ways, which its price shows:

- **Per image:** each image the call returns, at the model's price per image and your rate. The model's pages show the price of one image, such as "$0.03/image".
- **Per token:** the input and output tokens the provider reports, at the model's input and output prices per million tokens and your rate.

Your credit, budgets and key limits apply as on chat; [Errors](errors.md) lists the 402 and 403 answers.

- A call that fails over to another provider bills only the provider that answered.
- A call bills nothing when it is refused, when its provider answered an error (a refusal of your prompt by the provider's moderation is a 400 you receive), when its provider never received the request, or when what the provider answered cannot be delivered, such as no image, more images than you asked for (apart from a thinking model's interim images, above), bytes that are not a PNG, JPEG or WebP image, an image over 16 MiB, or an answer over 48 MiB. Such an answer moves the call to the next provider.
- A timeout, a network fault after the request was sent, or your client disconnecting bills what llm.buzz reserved for the call, since the provider may have made and billed images nobody received: for a model priced per image, the `n` images you asked for; for a model priced per token, the most your prompt and `n` images can cost. A client that gives up on a slow image can be charged.
- Neither route reads an `Idempotency-Key`: a retry is a new request, served and billed again.

## Not supported

Image-to-image (`input_references`), image edits and variations, streaming partial images, SVG output, image URLs in the answer (every image is `b64_json`), separate prices per resolution or megapixel (a model's sizes share one price per image), the image model routes without a key, and image models in the console's Playground.

# Audio

llm.buzz serves speech to text and text to speech with OpenRouter's request and answer shapes, so the OpenAI and OpenRouter SDKs reach them with the base URL alone. Each audio model serves one of the two. Chat Completions, Responses and Messages refuse an audio model with a 400 that names its route.

Every audio route takes your key as a bearer token, as on every other route ([Authentication](authentication.md)).

## Find an audio model

`GET https://api.llm.buzz/api/v1/models` lists audio models as OpenRouter does. Its default, `output_modalities=text`, leaves them out: with `output_modalities=transcription` it lists the speech-to-text models, with `output_modalities=speech` the text-to-speech models, and with `output_modalities=all` every model.

```bash
curl "https://api.llm.buzz/api/v1/models?output_modalities=speech" \
  -H "Authorization: Bearer $LLM_BUZZ_KEY"
```

A speech-to-text model's `architecture` is `audio->transcription`, and a text-to-speech model's is `text->speech`, with its voices in `supported_voices`; the model's page on the [Models page](/models) lists them too, under Voices. `pricing.prompt` is the price in USD per second of audio or per character of input text, at your rate, and `pricing.completion` is `"0"`.

Three more keys say what a model takes. A speech-to-text model's `verbose_json` says whether it answers `response_format: "verbose_json"`. A text-to-speech model's `speech_speed` says whether it takes `speed`, and its `default_voice` names the voice that speaks when you send none, or is `null` when the model needs a `voice`.

The [Models page](/models) shows each audio model's price per million seconds of audio or per million characters, such as "$3.33/M seconds", and its Output filter picks out transcription and speech models.

## Speech to text

`POST https://api.llm.buzz/api/v1/audio/transcriptions` transcribes one audio file. Send it as OpenRouter's JSON body, the file in base64 in `input_audio`, or as an OpenAI multipart upload in `file`, as the OpenAI SDK does.

```python tab="Python"
from openai import OpenAI
import os

client = OpenAI(
    base_url="https://api.llm.buzz/api/v1",
    api_key=os.environ["LLM_BUZZ_KEY"],
)
with open("call.mp3", "rb") as audio:
    transcript = client.audio.transcriptions.create(
        model="openai/whisper-large-v3-turbo",
        file=audio,
    )
print(transcript.text)
```

```bash tab="curl (JSON)"
curl https://api.llm.buzz/api/v1/audio/transcriptions \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "openai/whisper-large-v3-turbo",
       "input_audio": {"data": "'"$(base64 -w0 call.mp3)"'", "format": "mp3"}}'
```

```bash tab="curl (multipart)"
curl https://api.llm.buzz/api/v1/audio/transcriptions \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -F model=openai/whisper-large-v3-turbo \
  -F file=@call.mp3
```

The fields are OpenRouter's. A multipart upload sends each one as a text part, `provider` and `trace` as JSON text, and `timestamp_granularities[]` once per value.

| Field | Rule |
|---|---|
| `model` | Required: a speech-to-text model. Another model is a 400 on `model`. |
| `input_audio` | The JSON form: `data`, the file in base64, and `format`. A `url` is a 400: send the bytes. |
| `file` | The multipart form: the audio file. A `source_url` is a 400. |
| `response_format` | `json`, the default, answers `text` and `usage`. `verbose_json`, on a model whose `verbose_json` is `true`, adds `task`, `language`, `duration`, `segments`, and `words` when the provider sends them; on another model it is a 400. `text`, `srt`, `vtt` and any other value are a 400. |
| `language` | An ISO-639-1 code such as `en`. Omitted, the provider detects the language. |
| `temperature` | 0 to 1. |
| `timestamp_granularities` | `segment` and `word`, with `verbose_json` only. |
| `diarize`, `keyterms` | `diarize: true` and any `keyterms` are a 400: no model here diarizes or takes key terms. |
| `provider` | Only `zdr` and `data_collection` apply, as described in [Data policy](models-and-routing.md#data-policy). `order`, `only`, `ignore` and `options` are accepted and not applied, as on OpenRouter for audio. |
| `user`, `session_id`, `trace`, `prompt` | Accepted. `user` is recorded as the end-user label; the others are not sent to the provider. |

The file is at most 25 MiB: a larger file is a 413, and so is a JSON body over 39,234,221 bytes. It is one of `wav`, `mp3`, `flac`, `ogg` (Opus or Vorbis), `webm`, `m4a`, `aac`, or `pcm` (headerless signed 16-bit little-endian mono at 16 kHz). The format comes from `input_audio.format`, or from a multipart file's extension, else its content type.

llm.buzz reads the file and measures its duration before it calls a provider. A file it cannot read to the end, an empty file, or one too small to hold its duration (under 1,000 bits per second) is a 400 on `input_audio.data` or `file` that names the formats it takes. Such a file is never sent to a provider and never billed.

The answer:

```json
{"text": "Thanks for calling.", "usage": {"seconds": 12.48, "cost": 0.000041558}}
```

`usage.seconds` is what the call billed, and `usage.cost` its price in USD at your rate. With `verbose_json`, the provider's `task`, `language`, `duration`, `segments` and `words` come with it unchanged.

## Text to speech

`POST https://api.llm.buzz/api/v1/audio/speech` speaks one text and answers with the audio.

These samples send no `voice`, so the model's default voice speaks. To choose a voice, send `voice` with one of the IDs listed under Voices on the model's page, or as `supported_voices` by `GET https://api.llm.buzz/api/v1/models?output_modalities=speech`. The Python sample posts over plain HTTP with `httpx`, which the OpenAI Python SDK installs, because the OpenAI SDKs require `voice`.

```python tab="Python"
import os
import httpx

reply = httpx.post(
    "https://api.llm.buzz/api/v1/audio/speech",
    headers={"Authorization": f"Bearer {os.environ['LLM_BUZZ_KEY']}"},
    json={"model": "hexgrad/kokoro-82m", "input": "Your order has shipped.", "response_format": "mp3"},
    timeout=60,
)
reply.raise_for_status()
with open("reply.mp3", "wb") as file:
    file.write(reply.content)
```

```bash tab="curl"
curl https://api.llm.buzz/api/v1/audio/speech \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "hexgrad/kokoro-82m", "input": "Your order has shipped.", "response_format": "mp3"}' \
  -o reply.mp3
```

| Field | Rule |
|---|---|
| `model` | Required: a text-to-speech model. Another model is a 400 on `model`. |
| `input` | Required: one string of 1 to 4,096 characters, counted as Unicode code points. An empty or longer string, or an array of turns, is a 400 on `input`. |
| `voice` | One of the model's voices, listed as `supported_voices` by `GET https://api.llm.buzz/api/v1/models?output_modalities=speech`; another voice is a 400 on `voice`. Omitted, the model's `default_voice` speaks, and a model whose `default_voice` is `null` answers a 400 on `voice`. With the OpenAI SDKs, which require `voice`, pass one of those IDs. |
| `response_format` | `pcm`, the default, or `mp3`. Any other value is a 400. |
| `speed` | 0.25 to 4.0, on a model whose `speech_speed` is `true`. A speed other than 1 on another model is a 400 on `speed`. |
| `input_references` | Voice cloning: a 400 unless empty. |
| `instructions` | Accepted and ignored, as on OpenRouter for these models. |
| `provider`, `user`, `session_id`, `trace` | As on speech to text. |

The answer is the whole audio, not streamed: `Content-Type: audio/mpeg` for `mp3`, or `audio/pcm` for `pcm` (signed 16-bit little-endian mono at the provider's sample rate, 24 kHz for `hexgrad/kokoro-82m`), with `Cache-Control: no-store`. A provider answer that is not valid audio in the format you asked for moves the call to the next provider and is not billed; from the last provider, the call bills the characters, as [Billing](#billing) says.

## Cost and the generation record

Every audio answer carries `X-Generation-Id`: `gen-` and 32 hex digits. A text-to-speech answer has no JSON body for its cost, so read it from the generation record. The record covers any finished request of your organization, not only audio: any request's generation ID is the `gen-` form of its `x-request-id`.

```bash
curl "https://api.llm.buzz/api/v1/generation?id=gen-..." \
  -H "Authorization: Bearer $LLM_BUZZ_KEY"
```

The answer is OpenRouter's `{"data": {...}}`, read in your key's organization:

| Key | Value |
|---|---|
| `total_cost`, `usage` | What you were charged, in USD at your rate. |
| `api_type` | `stt` for speech to text, `tts` for text to speech, `completions` for Chat Completions, Responses and Messages, `embeddings`, `image` and `decisions`. |
| `model` | The model's name. |
| `provider_name` | The provider that served the call. |
| `latency`, `generation_time` | In milliseconds. |
| `data_region` | `europe` for a provider in the EU, else `global`. |

A generation ID of another organization, a malformed one, and one whose request has not finished all answer 404. A request without `id` answers 400.

## Billing

Speech to text is billed per second of audio and text to speech per character of input text, at the serving provider's price and your rate. Neither has an output price. `GET https://api.llm.buzz/api/v1/models/{author}/{slug}/endpoints` lists each provider's price.

- Speech to text bills the seconds of audio llm.buzz measures from the file, or the provider's own count when it is longer, up to the measured duration rounded up to a whole multiple of 30 seconds. Some providers count a minimum or whole chunks; such a count is billed up to that bound, never more.
- Text to speech bills the characters of `input`.
- Before any provider is called, llm.buzz reserves the most the call can cost. A budget, a key limit or credit that cannot cover it answers 402 or 403, and nothing is sent to a provider. [Errors](errors.md) lists the codes.
- An attempt that fails and moves to the next provider bills nothing. When the last provider fails after it received the request, the call bills the measured seconds or the characters, never the 30-second rounding: a 1.5-second file whose only provider answers 500 bills 1.5 seconds. When the last attempt never reaches its provider (the connection is refused, or DNS or TLS fails), the call bills nothing.
- Neither route reads an `Idempotency-Key`: a retry is a new request, served and billed again.

## Privacy

No audio or text of an audio call is kept in llm.buzz's prompt logs.

## Not supported

Streaming synthesis, URL inputs, diarization, key terms, voice cloning, multi-speaker input, the `text`, `srt` and `vtt` transcription formats, and audio output from chat models (`modalities: ["text", "audio"]`). Audio input to a chat model (`input_audio` content parts on Chat Completions) is a chat capability of its own, apart from these routes.

# Decisions

The decisions routes take a `state` and up to 32 typed questions about it, and answer each one in a fixed shape: a probability for `noul`, the chosen category and a probability per category for `choice`, and a position on your scale for `score`. No public model on llm.buzz answers them.

## Availability

`GET https://api.llm.buzz/api/v1/models?output_modalities=decisions` lists the models that answer decisions. Today it lists none, so a decisions request answers an error.

```bash
curl -s "https://api.llm.buzz/api/v1/models?output_modalities=decisions"
```

```json
{"object": "list", "data": [], "total_count": 0, "links": {"next": null}}
```

A model that answers decisions appears under that filter, and under `output_modalities=all`. [Choosing an API](choosing-an-api.md#find-a-model-for-an-api) lists the filters for every kind of model.

## Routes

| Route | What it is |
|---|---|
| `POST https://api.llm.buzz/api/v1/systemone` | TypeSafe's System One route. A model name without a `/` is read as `typesafe/<name>`. |
| `POST https://api.llm.buzz/api/alpha/decisions` | The route OpenRouter's SDKs call as `alpha.decisions`. They build it from the host alone, so give that one call the origin of `https://api.llm.buzz/api/v1` as its server URL, the same address without `/api/v1`. Model names are taken as written. |

Both routes take the same body and answer in the same shape, which follow TypeSafe's System One and OpenRouter's Decisions API. They take your inference key as a bearer token ([Authentication](authentication.md)); a management key answers 403 `management_key_not_allowed`. A browser page can call both: their preflight needs no key ([Browser access](api-reference.md#browser-access)).

## The request

Three fields are required, and the body is not chat: it has no messages, tools or generation settings.

| Field | What llm.buzz does |
|---|---|
| `model` | Required: a model that answers decisions. |
| `state` | Required: what the questions are about. Text, a JSON object or a JSON array; objects and arrays keep their structure. |
| `questions` | Required: 1 to 32 questions, each under an ID you choose. The ID comes back in `answers`. A question has a `type`, its `instructions` (text, an object or an array) and, for some types, `criteria`: [Question types](#question-types). |
| `provider` | Read only for `zdr` and `data_collection`, which keep the request on providers that meet the policy ([Data policy](models-and-routing.md#data-policy)). Every other key is ignored. |
| `session_id`, `trace`, `user` | Dropped. |
| Any other field | Refused before any provider is called, `stream` and chat fields included. |

## Question types

| Type | What it asks | `criteria` | The answer holds |
|---|---|---|---|
| `noul` | The probability that the statement in `instructions` holds. | Optional: an object with descriptions for `true` and `false`. | `noul`, a probability from 0 to 1. |
| `choice` | Which of your categories fits. | 2 to 64 category names, each with a description (text, an object or an array) or `null`. | `choice`, the category chosen; `confidence`; `probabilities`, one per category, keyed by your category names. |
| `score` | How the state rates on a scale whose steps you list in order. | A list of 2 to 10 descriptions, one per step, in order. | `score`, `confidence`, `legend`, and `probabilities` with one entry per step, keyed by the step's zero-based position written as a string. |

## The answer

The body carries `id`, the `model` you asked for, and `answers`: one entry under each of your question IDs, with its `type` and the fields above. `usage` holds `input_tokens` and `output_tokens`, the counts the provider reported, and `usage.cost`, the request's cost in US dollars at your rate as a JSON number. `cost` is left out when any part of it is unknown. The body has no `provider` key: `x-gateway-provider` in the response headers names the provider that answered, when it has a display name.

These are decision values, not assistant text. llm.buzz returns the provider's probabilities and scores as the provider gave them and never recomputes or normalizes them. It checks each answer before it reaches you: probabilities and confidence fall between 0 and 1, a distribution sums to 1 and the chosen category is the most probable (both within rounding), and the keys match your criteria. Only the validated fields come back, and any other metadata from the provider is left out.

## Limits

- 1 to 32 questions per request, and 1 to 256 bytes for a question ID or a category name.
- 262,144 bytes for the body. Nested data may go at most 64 levels deep.
- An answer is buffered and never streamed.
- An `Idempotency-Key` is ignored: sending the same request again runs it again as a request of its own.
- Not offered: `box` questions, and image, audio or video state.

## Errors

Errors come in OpenRouter's envelope: `error.metadata.gateway_code` says what went wrong and `metadata.param` names the field at fault. [Errors](errors.md) lists the key, balance and model errors. Two are specific to these routes, and both are refused before any provider is called:

- A body that fails the decision schema answers 400 `invalid_parameter` with `param` set to `body`. The message lists the fields the route accepts.
- A model that serves no decisions, which is every public model today, answers 400 `unsupported_capability` with `param` set to `model`.

# Tools and structured output

Tools let a model ask your code to run a function; structured output makes it answer in JSON that matches your schema. Both work on Chat Completions, Responses and Messages, each in its own shape.

## Which models

A model takes tools and structured output when its entry in `GET https://api.llm.buzz/api/v1/models` lists `tools`, `tool_choice`, `response_format` and `structured_outputs` in `supported_parameters`. Most text models do today. Before a provider serves a model, an automated check sends it real requests for tool calls and structured output where the model declares them, and a provider that fails stays out of routing: [Which providers serve](models-and-routing.md#which-providers-serve).

## Tools in each API

You declare the tools, the model asks for calls, and you run them and send the results back in the next request. Tool descriptions reach the provider unchanged and have no length limit of their own; the 64 MiB request limit and the model's context window still apply.

| | Chat Completions | Responses | Messages |
|---|---|---|---|
| You declare | `tools`, each `{"type": "function", "function": {...}}` with `name`, `description` and `parameters` | `tools`, each `{"type": "function", ...}` with `name`, `description`, `parameters` and `strict` | `tools`, each with `name`, `description` and `input_schema` |
| The model asks | `message.tool_calls`, each with an `id` and `function.arguments`, a JSON string; `finish_reason` is `tool_calls` | `function_call` items in `output`, each with a `call_id` and `arguments` | `tool_use` blocks in `content`, each with an `id` and `input`, an object; `stop_reason` is `tool_use` |
| You answer | A `tool` message with the call's `tool_call_id` | A `function_call_output` item with the call's `call_id` | A `tool_result` block with the call's `tool_use_id`, in a user message |

A Chat Completions loop on `deepseek/deepseek-v4.1-flash` asks, runs what the model asked for, and asks again until the model answers in text:

```python tab="Python"
import json
import os

from openai import OpenAI

client = OpenAI(
    base_url="https://api.llm.buzz/api/v1",
    api_key=os.environ["LLM_BUZZ_KEY"],
)
stock_schema = {
    "type": "object",
    "properties": {"sku": {"type": "string"}},
    "required": ["sku"],
    "additionalProperties": False,
}
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_stock_level",
            "description": "Return how many units of one SKU are in the warehouse.",
            "parameters": stock_schema,
        },
    }
]


def get_stock_level(sku: str) -> str:
    """Return the stock level of one SKU."""
    return {"KB-204": "37 units"}.get(sku, "unknown SKU")


messages = [{"role": "user", "content": "How many units of KB-204 are in stock?"}]
while True:
    reply = client.chat.completions.create(
        model="deepseek/deepseek-v4.1-flash", messages=messages, tools=tools
    )
    turn = reply.choices[0].message
    if not turn.tool_calls:
        print(turn.content)
        break
    messages.append(
        {
            "role": "assistant",
            "content": turn.content,
            "tool_calls": [call.model_dump() for call in turn.tool_calls],
        }
    )
    for call in turn.tool_calls:
        result = get_stock_level(**json.loads(call.function.arguments))
        messages.append({"role": "tool", "tool_call_id": call.id, "content": result})
```

```ts tab="TypeScript"
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llm.buzz/api/v1",
  apiKey: process.env.LLM_BUZZ_KEY,
});
const tools: OpenAI.Chat.Completions.ChatCompletionTool[] = [
  {
    type: "function",
    function: {
      name: "get_stock_level",
      description: "Return how many units of one SKU are in the warehouse.",
      parameters: {
        type: "object",
        properties: { sku: { type: "string" } },
        required: ["sku"],
        additionalProperties: false,
      },
    },
  },
];
const stock: Record<string, string> = { "KB-204": "37 units" };

const messages: OpenAI.Chat.Completions.ChatCompletionMessageParam[] = [
  { role: "user", content: "How many units of KB-204 are in stock?" },
];
while (true) {
  const reply = await client.chat.completions.create({
    model: "deepseek/deepseek-v4.1-flash",
    messages,
    tools,
  });
  const turn = reply.choices[0].message;
  if (!turn.tool_calls?.length) {
    console.log(turn.content);
    break;
  }
  messages.push({ role: "assistant", content: turn.content, tool_calls: turn.tool_calls });
  for (const call of turn.tool_calls) {
    if (call.type !== "function") continue;
    const { sku } = JSON.parse(call.function.arguments);
    messages.push({ role: "tool", tool_call_id: call.id, content: stock[sku] ?? "unknown SKU" });
  }
}
```

The same exchange on the other two APIs, with the `client`, `stock_schema` and `get_stock_level` above. Responses sends the results with `previous_response_id`, so the follow-up carries only them. Each snippet sends the follow-up only when the model asked for a call, because a request with no results to send is a 400:

```python tab="Responses"
tools = [
    {
        "type": "function",
        "name": "get_stock_level",
        "description": "Return how many units of one SKU are in the warehouse.",
        "parameters": stock_schema,
        "strict": True,
    }
]
response = client.responses.create(
    model="openai/gpt-5.6-luna", input="How many units of KB-204 are in stock?", tools=tools
)
results = [
    {
        "type": "function_call_output",
        "call_id": item.call_id,
        "output": get_stock_level(**json.loads(item.arguments)),
    }
    for item in response.output
    if item.type == "function_call"
]
if results:
    response = client.responses.create(
        model="openai/gpt-5.6-luna",
        previous_response_id=response.id,
        input=results,
        tools=tools,
    )
print(response.output_text)
```

```python tab="Messages"
from anthropic import Anthropic

claude = Anthropic(
    base_url="https://api.llm.buzz/api",
    api_key=os.environ["LLM_BUZZ_KEY"],
)
tools = [
    {
        "name": "get_stock_level",
        "description": "Return how many units of one SKU are in the warehouse.",
        "input_schema": stock_schema,
    }
]
messages = [{"role": "user", "content": "How many units of KB-204 are in stock?"}]
reply = claude.messages.create(
    model="anthropic/claude-sonnet-5.5", max_tokens=2048, tools=tools, messages=messages
)
results = [
    {
        "type": "tool_result",
        "tool_use_id": block.id,
        "content": get_stock_level(**block.input),
    }
    for block in reply.content
    if block.type == "tool_use"
]
if results:
    messages.append({"role": "assistant", "content": reply.content})
    messages.append({"role": "user", "content": results})
    reply = claude.messages.create(
        model="anthropic/claude-sonnet-5.5", max_tokens=2048, tools=tools, messages=messages
    )
print("".join(block.text for block in reply.content if block.type == "text"))
```

A Messages assistant turn goes back with every block it came with, `thinking` blocks included ([Messages](messages.md#the-answer)). A Responses follow-up that does not use `previous_response_id` sends the earlier items in `input` instead ([Input items](responses.md#input-items)). The Chat Completions loop above sends back only `content` and `tool_calls`. A reasoning model's tool turn may also carry `reasoning_content`, and sending it back keeps the model's thinking across the call: [Keep reasoning in the conversation](reasoning.md#keep-reasoning-in-the-conversation).

## Choosing the tool

`tool_choice` takes `auto`, which lets the model decide; `none`, which allows no call; `required`, which makes it call some tool; or one named function, which makes it call that one. Each API spells them its own way:

| | Chat Completions | Responses | Messages |
|---|---|---|---|
| Let the model decide | `"auto"` | `"auto"` | `{"type": "auto"}` |
| No call | `"none"` | `"none"` | `{"type": "none"}` |
| Some tool | `"required"` | `"required"` | `{"type": "any"}` |
| A named function | `{"type": "function", "function": {"name": "get_stock_level"}}` | `{"type": "function", "name": "get_stock_level"}` | `{"type": "tool", "name": "get_stock_level"}` |

Some models cannot be forced:

- **Claude Opus 5.5, Sonnet 5.5 and Fable 5.1** take no forced choice from any provider. llm.buzz serves `required` or a named function as `auto` and names it `tool_choice->auto` in the body's `x-sgr-ignored-parameters`, and the model may then answer without calling a tool. When the result must fit a schema, use `auto` with strict tools or structured output.
- **Moonshot AI's own API** takes no named function. llm.buzz sends a named choice to another provider of the model when one takes it, and otherwise serves the request as `auto`, named `tool_choice->auto`. `required` reaches Moonshot AI as sent and still forces a call, so with one tool that must be called, send `required`.

## Parallel calls

A model may ask for several tools in one turn. `parallel_tool_calls: false` on Chat Completions and Responses asks for one at a time, and llm.buzz honors it on every provider:

- A provider that has the switch receives it.
- On a provider that has none, llm.buzz enforces `false` itself. It returns the first call of a turn, drops the later ones, and names it `parallel_tool_calls->emulated(serialized_by_gateway)`. The model sees one result and asks for the remaining calls on the next turn.
- On such a provider `true` is the provider's own behavior, and `x-sgr-ignored-parameters` names it `parallel_tool_calls->dropped(provider_default)`.

Messages has no `parallel_tool_calls` field: one you send is dropped and named.

## Strict tools

`strict: true` asks the provider to hold the model to your schema: `function.strict` on Chat Completions and `strict` on a Responses function tool or a Messages tool. llm.buzz sends it to providers that enforce it. When no provider that may serve the request enforces it, llm.buzz serves the tool without `strict`, your schema unchanged, and names it `tools.strict->false`.

On Responses, a function tool sent without `strict` is not strict: llm.buzz sends it to the provider as `strict: false`, where OpenAI's own Responses API defaults to `true`. Send `strict: true` for strict schema adherence, as the Responses example above does.

- Claude's strict mode cannot express every JSON Schema keyword, among them `maxItems`, `oneOf`, `minimum`, some `format` values and a recursive `$ref`. A strict tool that uses one goes to a provider that can enforce it. When none can, llm.buzz drops `strict` and keeps the schema as you wrote it, never removing a keyword, and names the drop `tools.strict->false`.
- Claude requires `additionalProperties: false` on every object of a strict schema. llm.buzz closes the objects of a schema sent to Claude and names it `tools.parameters.additionalProperties->false`.

## IDs, arguments and tool results

- **IDs.** On Chat Completions and Responses a tool call's ID is an opaque string of 1 to 65,536 characters. Send each one back complete, signature suffix included, in the assistant call and in its result. On Messages the result names the call by the `id` of its `tool_use` block.
- **Streamed arguments.** The fragments of a streamed call add up, in order, to one JSON object. Join them and parse the result once the call is complete. On a Chat Completions stream llm.buzz stops relaying a call's arguments where that object closes, so whatever a provider sends after it never reaches you.
- **Arguments cut off.** If the output limit ends a call's arguments in the middle, llm.buzz drops that call and ends the turn as cut by the limit: `finish_reason: "length"` on Chat Completions, `status: "incomplete"` on Responses, `stop_reason: "max_tokens"` on Messages. Raise the output limit and ask again. llm.buzz does not repair the arguments or retry the turn, and you must not run a partial call from an answer the limit ended. On a stream, the fragments you already received end with the cut outcome: do not run them.
- **Check what you receive.** A finished call is JSON, but llm.buzz does not promise that it satisfies every constraint of your schema. Validate the arguments before you run the function.
- **Images in results.** A tool can return an image, such as a screenshot. A Chat Completions `tool` message takes `image_url` parts beside its text, a Responses `function_call_output` takes a list of text and image parts, and a Messages `tool_result` holds text and image blocks. All three APIs refuse any other kind of content in a tool result, a Messages `document` block included, with a 400 that names the message, item or block. Where a provider's tool results take only text, llm.buzz moves the images into one user message that follows the results and names it `messages.content.tool_result.image->following_user_message`. A provider that takes no image input at all gets placeholder text where the image was, named `messages.content.tool_result.image->placeholder`. [Images in requests](images-in-requests.md) covers images in the rest of a request.

## Structured output

Ask for JSON in the field your API has for it:

| API | How to ask |
|---|---|
| Chat Completions | `response_format` with `type: "json_schema"` for your schema, or `type: "json_object"` for any JSON object. |
| Responses | `text.format` with `type: "json_schema"` for your schema. `type: "json_object"` is a 400 `invalid_parameter` naming `text.format.type`: use `json_schema`. |
| Messages | Anthropic's `output_config`, with its `format` for a schema. An `output_config` that holds more than `effort` is sent as you wrote it when every provider that may serve the request takes Anthropic's API. Otherwise llm.buzz drops it and names `output_config` in `x-sgr-ignored-parameters`, and an `effort` in it still applies. Chat's `response_format` is not applied on Messages: it is dropped and named. |

The examples below reuse `client` and `json` from the tool loop.

```python tab="Chat Completions"
schema = {
    "type": "object",
    "properties": {"city": {"type": "string"}, "year": {"type": "integer"}},
    "required": ["city", "year"],
    "additionalProperties": False,
}
reply = client.chat.completions.create(
    model="deepseek/deepseek-v4.1-flash",
    messages=[{"role": "user", "content": "Lisbon hosted the world expo in 1998. Extract the city and the year."}],
    response_format={
        "type": "json_schema",
        "json_schema": {"name": "place_and_year", "strict": True, "schema": schema},
    },
)
data = json.loads(reply.choices[0].message.content)
```

```python tab="Responses"
response = client.responses.create(
    model="deepseek/deepseek-v4.1-flash",
    input="Lisbon hosted the world expo in 1998. Extract the city and the year.",
    text={
        "format": {
            "type": "json_schema",
            "name": "place_and_year",
            "strict": True,
            "schema": schema,
        }
    },
)
data = json.loads(response.output_text)
```

- **`json_schema` is the one to use when the shape matters.** `json_object` asks for valid JSON and nothing more. Each provider uses its own JSON mode and also gets a short instruction asking for JSON. Claude has no JSON mode and gets the instruction alone, which `x-sgr-ignored-parameters` names `response_format->instruction(json_object)`. llm.buzz sends no empty schema in its place.
- **Claude closes the objects.** Claude requires `additionalProperties: false` on every object of a schema. On Chat Completions and Responses, llm.buzz closes the objects of a strict schema sent to Claude, and `x-sgr-ignored-parameters` says so. On Messages, `output_config` is not rewritten: it reaches Anthropic as you wrote it, under the rule in the table above. Write the schema closed and the two stay the same.
- **A cut answer is cut.** An answer that hits the output limit ends with `finish_reason: "length"` on Chat Completions, `status: "incomplete"` on Responses or `stop_reason: "max_tokens"` on Messages, and may be unfinished JSON. Raise the limit and ask again. llm.buzz does not run OpenRouter's `response-healing` plugin: on Chat Completions and Responses it is dropped and named `plugins.response-healing`, and Messages drops and names `plugins` whole.

## Tools llm.buzz runs

llm.buzz runs one kind of tool for you, **tool search**. Mark most tools `defer_loading: true` and declare a tool search, and the model sees only the other tools and a search. The tools a search finds join the request. [Tool search](api-reference.md#tool-search) has each API's spelling, and `cost` includes the billed search rounds.

Web search is not available on llm.buzz today. Chat Completions and Responses accept `:online` and the `web` plugin, and Chat Completions also `web_search_options`: the request is served without a search, with nothing billed for one, and `x-sgr-ignored-parameters` names `web_search->dropped(search_unavailable)`. For an organization whose requests llm.buzz keeps in the EU such a request answers 400 `web_search_unavailable_in_region`. Messages drops `:online`, `plugins` and `web_search_options` and names them. [Coming from OpenRouter](from-openrouter.md) lists the other tools OpenRouter runs that llm.buzz does not.

## Related

- [Chat Completions](chat-completions.md), [Responses](responses.md) and [Messages](messages.md) list every request field of their API.
- [Streaming](streaming.md) lists the events of a stream, the ones that carry tool calls included.

# Streaming

With `stream: true` the answer arrives as server-sent events while the model writes it. Chat Completions, Responses and Messages all stream, each with its own events.

## Stream a request

The SDKs read the events for you. This loop prints each piece of text as it arrives, and the last chunk carries the usage and its `cost`. You do not need `stream_options`: the usage chunk comes whenever the counts are known.

```python tab="Python"
from openai import OpenAI
import os

client = OpenAI(
    base_url="https://api.llm.buzz/api/v1",
    api_key=os.environ["LLM_BUZZ_KEY"],
)
stream = client.chat.completions.create(
    model="deepseek/deepseek-v4.1-flash",
    messages=[{"role": "user", "content": "Explain a database index in two sentences."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:
        print(f"\n{chunk.usage.total_tokens} tokens, cost {chunk.usage.model_extra.get('cost')}")
```

```ts tab="TypeScript"
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llm.buzz/api/v1",
  apiKey: process.env.LLM_BUZZ_KEY,
});
const stream = await client.chat.completions.create({
  model: "deepseek/deepseek-v4.1-flash",
  messages: [{ role: "user", content: "Explain a database index in two sentences." }],
  stream: true,
});
for await (const chunk of stream) {
  const text = chunk.choices[0]?.delta?.content;
  if (text) process.stdout.write(text);
  if (chunk.usage) {
    const { cost } = chunk.usage as { cost?: number };
    console.log(`\n${chunk.usage.total_tokens} tokens, cost ${cost}`);
  }
}
```

```bash tab="curl"
curl -N https://api.llm.buzz/api/v1/chat/completions \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek/deepseek-v4.1-flash",
    "stream": true,
    "messages": [{"role": "user", "content": "Explain a database index in two sentences."}]
  }'
```

`-N` turns off curl's own buffering, so the lines print as they arrive. Responses and Messages take the same `stream: true` and send their own events, described next.

## Events in each API

| | Chat Completions | Responses | Messages |
|---|---|---|---|
| Opens with | A chunk with the assistant role | `response.created`, then `response.in_progress` | `message_start`, with the message so far |
| Content | `data:` lines, each a `chat.completion.chunk` with `delta.content`, `delta.reasoning`, `delta.tool_calls` or `delta.images`. Every chunk carries `provider`. | OpenAI's output events for text, function-call arguments and reasoning summaries, such as `response.output_text.delta` | `content_block_start`, `content_block_delta` and `content_block_stop` for text, tool input and thinking. One `ping` follows `message_start`. |
| Stop reason | `finish_reason` on the finish chunk, whose `delta` is empty; the usage chunk repeats it | `status` of the final response | `stop_reason` on `message_delta` |
| Usage and cost | A last chunk with `usage`, `cost` inside it | `usage`, `cost` inside it, on the final response | `usage`, `cost` inside it, on `message_delta`. `message_start` carries an early `usage` and never `cost`. |
| Ignored fields | `x-sgr-ignored-parameters` on content and finish chunks, not on the usage or error chunk | On the response object in the events | On `message_start` |
| Ends with | `data: [DONE]` | Exactly one terminal event, which carries the final response | `message_stop` |
| A failure after output | One chunk with an `error` object and `finish_reason: "error"`, then `data: [DONE]` | The terminal event reports the failure | One `error` event in place of `message_stop` |

`x-sgr-ignored-parameters` is a key in the body of an event, not a header. `cost` is left out of `usage` when any part of the cost is unknown. The usage in `message_start` is an early figure: Anthropic's own start counts when Anthropic serves the request, and llm.buzz's estimate of the prompt for any other provider. `message_delta` carries the final counts: [Usage and cost](messages.md#usage-and-cost). The response object in Responses events also echoes `instructions`, `metadata` and `tools`: [The answer](responses.md#the-answer).

## Comments

While the model is silent, llm.buzz sends comment lines, which begin with `:`, to keep the connection active. Any server-sent events client ignores them; ignore them in a parser of your own too. They carry no data, and they do not extend the 600-second limit below.

## Failures

- **Before output reaches you,** a provider that fails or times out is replaced by the next provider of the model, unless you turned fallbacks off, and the failed attempt costs nothing; a provider's own 400 is not tried on another provider. Your stream opens from the provider that answered. When every provider fails before it opens, you get an ordinary HTTP error: [Errors](errors.md).
- **Once output has reached you,** the request stays on the provider that sent it. The first text, tool call or refusal that reaches you fixes the provider, and llm.buzz never splices another provider's output into the stream. A refusal is not retried on another provider.
- **A later failure** ends the stream with an error in your API's form, shown in the table above. The HTTP status was already 200 when the stream opened, so read the last event of every stream before you treat the answer as complete.
- **A provider that stops after output,** closing its stream without a terminal event, is not an error. llm.buzz finishes the answer from what was served: with the finish the provider had already declared, when it answers in OpenAI's Chat Completions form, or otherwise as incomplete (`finish_reason: "length"` on Chat Completions, `status: "incomplete"` on Responses, `stop_reason: "max_tokens"` on Messages), with open items closed and a tool call cut in the middle dropped. A provider that stops before any output is replaced by the next provider.
- **At 600 seconds** a stream still generating ends with a timeout error, followed on Chat Completions by `data: [DONE]`, and is billed for the prompt and the output you received. The request time limit is shared by every provider the request tried.
- **A balance that runs out** while a stream runs does not cut it off. llm.buzz charges the stream when it completes.

## Stopping a stream

On Chat Completions, Responses and Messages, closing the connection makes llm.buzz stop reading from the provider, provided the request carried no `Idempotency-Key`. Messages has no such key, so closing a Messages stream always stops it. A provider may still finish work it had begun.

- You are billed for the prompt and for the output you received. When the provider has not reported the counts, llm.buzz counts them itself.
- A stream you close before the provider accepted the request is charged what llm.buzz reserved for it before calling the provider, the most it could cost.
- A Chat Completions or Responses stream sent with an `Idempotency-Key` keeps running after you leave, within the limit of what llm.buzz keeps for a replay, and a retry with the same key joins it or returns its answer instead of starting a second generation. The request is billed once: [Retries and idempotency](retries-and-idempotency.md).

## What never streams

Embeddings and Images answer 400 to `stream: true`, Audio returns its whole result at once, and Decisions are buffered and refuse a `stream` field. Responses also has a WebSocket transport that carries the same events, one JSON event in each frame: [WebSocket](responses.md#websocket).

## Related

- [Chat Completions](chat-completions.md#stream), [Responses](responses.md#stream) and [Messages](messages.md#stream) list what each API adds to a stream.
- [Tools and structured output](tools-and-structured-output.md) covers streamed tool arguments and what a cut-off call looks like.

# Reasoning

Reasoning models think before they answer. You set how much in the field your API uses, and llm.buzz sends it to each provider in that provider's terms, by OpenRouter's rules.

## Which models reason

`GET https://api.llm.buzz/api/v1/models` lists a model's efforts as `reasoning_efforts`, in order from `none` to `max`, when the model's lead provider's listing names them. The key is left out otherwise, including where llm.buzz knows a model's efforts only from its own rules for the model's family. A model without the key can still reason. Do not filter on `supported_parameters` for reasoning: no model in the public list names `reasoning` there today.

The efforts differ from model to model. `openai/gpt-5.6-luna` takes `none`, `low`, `medium`, `high`, `xhigh` and `max`, with no `minimal`. `anthropic/claude-sonnet-5.5` takes `low` to `max` and has no `none`, because it cannot turn thinking off. A request may name `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, `ultra` or `max`; Chat Completions and Responses answer 400 `invalid_parameter`, naming the field, for any other value.

## The controls in each API

| | Chat Completions | Responses | Messages |
|---|---|---|---|
| Effort | `reasoning_effort`, or `reasoning.effort` | `reasoning.effort` | `output_config.effort`, or `reasoning.effort` |
| Token budget | `reasoning.max_tokens`, `thinking.budget_tokens` or `thinking_budget` | None | `thinking.budget_tokens` or `reasoning.max_tokens` |
| On or off | `reasoning.enabled`, `thinking`, `enable_thinking` or `chat_template_kwargs.enable_thinking` | An effort of `none` is off | `thinking`, or `reasoning.enabled` |
| Hide the reasoning text | `reasoning.exclude: true` or `include_reasoning: false` | None | `reasoning.exclude: true` |

- **Chat Completions.** `reasoning` is OpenRouter's object: `effort` or `max_tokens`, not both (a request with both is 400 `invalid_parameter`), plus `enabled` and `exclude`. Its fields must agree: an effort tier or a budget beside `enabled: false`, or `effort: "none"` beside `enabled: true`, is 400 `invalid_parameter` naming `reasoning.enabled`. `include_reasoning: true` is `reasoning: {}` and `false` is `reasoning: {"exclude": true}`. `thinking` takes `enabled`, `adaptive` or `disabled`: a bare `enabled` or `adaptive` switches reasoning on at llm.buzz's default effort for the model ([below](#how-an-effort-is-mapped)), an `enabled` with `budget_tokens` is a budget ([Token budgets](#token-budgets)), and `disabled` is effort `none`. The body's `x-sgr-ignored-parameters` names the move to an effort as `thinking->translated(reasoning_effort)`. It says what llm.buzz did with the field, not that it dropped it.
- **Responses.** `reasoning` takes `effort`, `summary`, `generate_summary` and `context`; `summary` and `generate_summary` sent together must match. `reasoning.mode` is 400 `invalid_parameter`, and so is any other key. Without a `summary`, llm.buzz asks OpenAI for an automatic one unless the effort is `none`.
- **Messages.** `thinking` takes Anthropic's forms, described under [Thinking](messages.md#thinking). On the Claude models llm.buzz lists today, a bare `{"type": "enabled"}` runs adaptive thinking, named `thinking.type->adaptive`, and `budget_tokens` is refused ([Token budgets](#token-budgets)). `reasoning` is OpenRouter's object, as on Chat Completions, and its fields must agree: an effort tier or a budget beside `enabled: false`, or `effort: "none"` beside `enabled: true`, is a 400 `invalid_request_error` whose message ends `(param: reasoning.enabled)`. `reasoning.effort` with `reasoning.max_tokens`, or a `reasoning.effort`, `reasoning.max_tokens` or `output_config.effort` beside a `thinking.budget_tokens`, is a 400 `invalid_request_error`.
- A control that belongs to another API is dropped and named in `x-sgr-ignored-parameters`, as Chat's `reasoning_effort` is on Messages. Send one depth control per request: an effort and a budget in one Chat Completions request, or two budgets, are 400 `invalid_parameter`, and `metadata.param` names one of the controls. Two efforts in one Chat Completions request, `reasoning_effort` with `reasoning.effort`, are not refused: `reasoning_effort` is used, in either order and in a stream, and `x-sgr-ignored-parameters` names the other as `reasoning->ignored(explicit_reasoning_effort)`. If one of them is `none` and the other is not, the request is 400 `invalid_parameter` naming `reasoning_effort`.

## Set the effort

This request asks `openai/gpt-5.6-luna` for a low effort and reads the model's reasoning beside its answer:

```python tab="Python"
from openai import OpenAI
import os

client = OpenAI(
    base_url="https://api.llm.buzz/api/v1",
    api_key=os.environ["LLM_BUZZ_KEY"],
)
reply = client.chat.completions.create(
    model="openai/gpt-5.6-luna",
    messages=[{"role": "user", "content": "A train leaves at 9:40 and the trip takes 2 h 35 min. When does it arrive?"}],
    extra_body={"reasoning": {"effort": "low"}},
)
message = reply.choices[0].message
print((message.model_extra or {}).get("reasoning"))
print(message.content)
```

```ts tab="TypeScript"
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llm.buzz/api/v1",
  apiKey: process.env.LLM_BUZZ_KEY,
});
const reply = await client.chat.completions.create({
  model: "openai/gpt-5.6-luna",
  messages: [{ role: "user", content: "A train leaves at 9:40 and the trip takes 2 h 35 min. When does it arrive?" }],
  reasoning: { effort: "low" },
} as OpenAI.Chat.ChatCompletionCreateParamsNonStreaming);
const message = reply.choices[0].message as { content: string | null; reasoning?: string };
console.log(message.reasoning);
console.log(message.content);
```

```bash tab="curl"
curl https://api.llm.buzz/api/v1/chat/completions \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.6-luna",
    "messages": [{"role": "user", "content": "A train leaves at 9:40 and the trip takes 2 h 35 min. When does it arrive?"}],
    "reasoning": {"effort": "low"}
  }'
```

`reasoning` is not a field of the SDKs' Chat Completions method, so the Python call sends it through `extra_body` and the TypeScript call casts its parameters; `reasoning_effort` is one, and takes the effort alone. On Responses `reasoning` is an SDK field, and it can ask for a summary of the reasoning:

```python
response = client.responses.create(
    model="openai/gpt-5.6-luna",
    input="A train leaves at 9:40 and the trip takes 2 h 35 min. When does it arrive?",
    reasoning={"effort": "low", "summary": "auto"},
)
for item in response.output:
    if item.type == "reasoning":
        for part in item.summary:
            print(part.text)
print(response.output_text)
```

On Messages, `anthropic/claude-sonnet-5.5` thinks adaptively and takes its depth from `output_config.effort`. The Anthropic SDK builds `client` as in [Messages](messages.md#send-a-request):

```python
message = client.messages.create(
    model="anthropic/claude-sonnet-5.5",
    max_tokens=4096,
    thinking={"type": "adaptive"},
    output_config={"effort": "low"},
    messages=[{"role": "user", "content": "A train leaves at 9:40 and the trip takes 2 h 35 min. When does it arrive?"}],
)
for block in message.content:
    if block.type == "thinking":
        print(block.thinking)
    elif block.type == "text":
        print(block.text)
```

## How an effort is mapped

A model takes some efforts and not others, and llm.buzz moves your effort to one the model takes instead of failing the request.

- Where llm.buzz knows a model's efforts, an effort the model does not take becomes the nearest one it does, and a tie takes the lower. `x-sgr-ignored-parameters` names the move as `reasoning_effort-><level>` on every API. `minimal` on `openai/gpt-5.6-luna` is served as `low`, named `reasoning_effort->low`, and `xhigh` on `openai/gpt-5-mini`, whose highest effort is `high`, is served as `high`.
- `none` never becomes an active effort, and an active effort never becomes `none`. A model with no off level refuses `none` with a 400 before any provider is called, naming `reasoning_effort` on Chat Completions, whichever spelling you sent, and `reasoning.effort` on Responses. Today that includes Claude Opus 5.5, Sonnet 5.5, Fable 5 and Fable 5.1, and OpenAI's gpt-5-mini, gpt-5-nano, gpt-6-astra and gpt-6.1-sol. On Messages, `thinking: {"type": "disabled"}` is refused for the Claude models among them; Sonnet 5.5 offers `{"type": "between_tools"}` in its place. The refusal holds on every route of these models, llm.buzz's own provider included: a Claude model takes the efforts its family takes, so an effort the family refuses is refused before any provider is called, whichever API the provider speaks.
- A switch with no level, such as `reasoning.enabled: true` or `enable_thinking: true`, takes llm.buzz's default effort for the model, which is medium unless the model has another. On Messages, a bare `adaptive` sent to Claude carries no effort, so Claude's own default applies.
- Where llm.buzz does not know a model's efforts, your value goes to the provider as you sent it. A provider that refuses it answers 400, with its own sentence after `provider rejected the request:` and `metadata.param` naming the field.
- If you send no setting, llm.buzz sends no effort, and the model runs at its own default. One exception: on Messages, a `max_tokens` under 1,024 with no setting of your own, on a model whose providers all reason by default and offer `none`, is sent at effort `none`, named `reasoning_effort->none(max_tokens_headroom)`, so the reply is text ([Thinking](messages.md#thinking)).
- OpenAI's reasoning models and the Claude models that think adaptively take sampling settings only where the provider does: [Sampling settings a model does not take](api-reference.md#sampling-settings-a-model-does-not-take).

## Token budgets

Instead of an effort you can give a number of thinking tokens. Not every provider has a field for one, and llm.buzz never turns a budget into an effort: where none of a model's providers can take it, the request is refused before any provider is called, naming the field.

- OpenAI's models take no budget. Neither do the Claude models llm.buzz lists today, all of which are in the families that think adaptively (Opus 4.7 onward, Sonnet 5, Haiku 5 and Fable 5). A budget sent to one answers 400 `unsupported_parameter` on Chat Completions and 400 `invalid_request_error` on Messages, naming the budget field you sent. Choose an effort for these models.
- On Chat Completions, `thinking.budget_tokens` is an integer of at least 1,024 and below the output limit; any other value is 400 `invalid_parameter` naming it. On Messages, `thinking.budget_tokens` and `reasoning.max_tokens` must be below `max_tokens`, so the answer has room.
- On a Claude model that takes a budget, which none on llm.buzz does today, a `reasoning.max_tokens` under 1,024 is raised to 1,024 and named `reasoning.max_tokens->1024(anthropic_minimum)`.

## Read the reasoning

| | Chat Completions | Responses | Messages |
|---|---|---|---|
| Where | `message.reasoning`, and `delta.reasoning` in a stream | A `reasoning` item in `output`, with `summary_text` parts | `thinking` blocks in `content` |

Readable reasoning is what the provider returns: reasoning text, thinking blocks or a summary. Where a provider withholds it by default, llm.buzz asks for it: Claude's thinking as a summary, and OpenAI's reasoning as an automatic summary unless you chose a `summary` or the effort is `none`. On Messages, Anthropic's own thinking blocks are signed. Reasoning from a provider other than Anthropic arrives as one unsigned `thinking` block.

`reasoning: {"exclude": true}` and `include_reasoning: false` keep the text out of the answer, in a stream too. The model still reasons, and you are billed for it. On Messages, Anthropic's own signed thinking blocks stay in the answer, because the next request needs them to continue. Responses has no field to hide a summary.

## What reasoning costs

- Reasoning tokens are output tokens. They count toward `max_tokens`, `max_completion_tokens` or `max_output_tokens`, so a limit that is too low can end the answer inside the reasoning, with `finish_reason: length` on Chat Completions, `status: incomplete` on Responses or `stop_reason: max_tokens` on Messages.
- They are billed at the reasoning price of the provider that answered, and at its output price when the provider sets none. A model's public document lists `reasoning` among its prices ([Public models](/docs/api/public-models)). [Pricing](pricing.md) explains how a request is priced. Hiding the reasoning text saves nothing.
- Chat Completions reports them in `usage.completion_tokens_details.reasoning_tokens`, a part of `completion_tokens`, where the provider reports a count. A stream's usage chunk always has it, `0` when the provider reported none, with `usage.unreported_token_details` listing `reasoning_tokens` ([Usage and cost](chat-completions.md#usage-and-cost)). Claude reports no separate count: its thinking is inside `completion_tokens` (`output_tokens` on Messages).

## Keep reasoning in the conversation

Send the conversation back unchanged, reasoning included.

- **Chat Completions.** An assistant message may carry `reasoning`, `reasoning_content` and OpenRouter's `reasoning_details` (its `reasoning.text` blocks). A provider that takes earlier reasoning back gets it. For another, llm.buzz drops it and names it, as `messages.reasoning_content->dropped(unsupported_by_provider)`; the request still serves.
- **Responses.** Send `reasoning` items back as they came, and ask for `include: ["reasoning.encrypted_content"]` when you send `store: false`: [Input items](responses.md#input-items).
- **Messages.** Send every `thinking` and `redacted_thinking` block back with its assistant turn. Claude's signed blocks go back to Claude; another provider receives the turn without them, and `x-sgr-ignored-parameters` names it `messages.thinking->dropped(unsupported_by_provider)`.

A tool turn whose reasoning a provider keeps for the next request stays with that provider. While it can serve, `provider.order`, `only`, `ignore` and the routing preset do not move the request off it, and `allow_fallbacks: false` keeps it there alone. If it is down, the model's other providers serve the turn without that reasoning, and the visible conversation is unchanged ([Provider preferences](models-and-routing.md#provider-preferences)).

## Related

- [Chat Completions](chat-completions.md), [Responses](responses.md) and [Messages](messages.md) list every field of their API.
- [Tools and structured output](tools-and-structured-output.md) and [Streaming](streaming.md) cover the rest of what the three text APIs share.
- [Usage and limits](usage-and-limits.md) collects the limits of every API.

# Images in requests

Models that read images take them inside a message. This page covers images on Chat Completions and Messages, and a message can mix text and several images.

## Which models read images

A model reads images when `image` is in `architecture.input_modalities` on its entry in `GET https://api.llm.buzz/api/v1/models`. Most text models do today, and this lists them. Before a provider serves a model that takes images, an automated check sends it real requests for each capability the model declares, image input included, and a provider that fails stays out of routing: [Which providers serve](models-and-routing.md#which-providers-serve).

```bash
curl -s https://api.llm.buzz/api/v1/models \
  | jq -r '.data[] | select(.architecture.input_modalities | index("image")) | .id'
```

The [vision and image pipelines guide](/docs/guides/vision-and-image-pipelines) lists the models that read images and the ones that make them.

## Chat Completions

An image is an `image_url` part beside the text of a `user` message. This request sends a local file as a `data:` URL:

```python tab="Python"
import base64
import os

from openai import OpenAI

client = OpenAI(
    base_url="https://api.llm.buzz/api/v1",
    api_key=os.environ["LLM_BUZZ_KEY"],
)
with open("receipt.png", "rb") as image:
    encoded = base64.b64encode(image.read()).decode()
reply = client.chat.completions.create(
    model="deepseek/deepseek-v4.1-flash",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What is the total on this receipt?"},
                {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{encoded}"}},
            ],
        }
    ],
)
print(reply.choices[0].message.content)
```

```ts tab="TypeScript"
import { readFileSync } from "node:fs";
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llm.buzz/api/v1",
  apiKey: process.env.LLM_BUZZ_KEY,
});
const encoded = readFileSync("receipt.png").toString("base64");
const reply = await client.chat.completions.create({
  model: "deepseek/deepseek-v4.1-flash",
  messages: [
    {
      role: "user",
      content: [
        { type: "text", text: "What is the total on this receipt?" },
        { type: "image_url", image_url: { url: `data:image/png;base64,${encoded}` } },
      ],
    },
  ],
});
console.log(reply.choices[0].message.content);
```

```bash tab="curl"
base64 -w0 receipt.png | jq -Rs '{
  model: "deepseek/deepseek-v4.1-flash",
  messages: [{role: "user", content: [
    {type: "text", text: "What is the total on this receipt?"},
    {type: "image_url", image_url: {url: ("data:image/png;base64," + .)}}
  ]}]
}' | curl https://api.llm.buzz/api/v1/chat/completions \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d @-
```

To send a URL instead, put it in `url`: `{"type": "image_url", "image_url": {"url": "https://example.com/receipt.png", "detail": "high"}}`.

- `url` is a `data:` URL with a base64 PNG, JPEG, GIF or WebP image, or an `http` or `https` URL. llm.buzz does not download a URL: the provider fetches it. A data URL that is not a base64 image of one of those types is 400 `invalid_parameter` naming the part.
- `detail` is `auto`, `low` or `high`, and is passed on to the provider. Another value is 400 `invalid_parameter`.
- `media_type` is an optional hint, one of `image/png`, `image/jpeg`, `image/gif` and `image/webp`, for clients such as GitHub Copilot that send it. llm.buzz accepts it and does not send it on: the `url` and `detail` decide. A data URL keeps its own type.
- An `image_url` object with a key other than `url`, `detail` and `media_type`, or an `image_url` that is a plain string, is 400 `invalid_parameter`.
- Image parts belong in `user` messages. A `tool` message may carry them too ([Images in tool results](#images-in-tool-results)), and an `assistant` message may carry the images a model that makes images returned, sent back to continue the conversation. An image part in a `system` or `developer` message is 400 `invalid_parameter`.
- An inline image is part of the request body, so it counts toward the 64 MiB limit: a larger body answers 413 `request_too_large`.

## Messages

An `image` block in a `user` message takes a `source`: base64 data with its `media_type`, or a URL.

```python
import base64

with open("receipt.png", "rb") as image:
    encoded = base64.standard_b64encode(image.read()).decode()
message = client.messages.create(
    model="anthropic/claude-sonnet-5.5",
    max_tokens=4096,
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "image", "source": {"type": "base64", "media_type": "image/png", "data": encoded}},
                {"type": "text", "text": "What is the total on this receipt?"},
            ],
        }
    ],
)
for block in message.content:
    if block.type == "text":
        print(block.text)
```

The Anthropic SDK builds `client` as in [Messages](messages.md#send-a-request). A URL source is `{"type": "url", "url": "https://example.com/receipt.png"}`. The formats are PNG, JPEG, GIF and WebP. A base64 source of another type, and an image block in an `assistant` message, is a 400 `invalid_request_error` naming the field.

## Images in tool results

A tool can return an image, such as a screenshot, in the result it sends back to the model. Chat Completions, Responses and Messages each carry it in their own shape, and a provider whose tool results take only text gets the image in the user message that follows: [Tools and structured output](tools-and-structured-output.md#ids-arguments-and-tool-results).

## PDFs, audio and video

No chat model on llm.buzz reads PDFs, audio or video today: none lists them in `architecture.input_modalities`. A model that does lists them there. A Messages `document` block and a Chat `input_audio` part reach the same check as an image, so a request that carries one for a model that cannot read it is refused. To turn speech into text, use a transcription model: [Audio](audio.md).

## When a request is refused

llm.buzz refuses a part in a user or assistant message that the model cannot read with a 400 before any provider is called, instead of answering from the text around it. That covers an image for a model that does not read images, a URL image when every provider of the model takes inline images only, and a PDF, audio or video part. An image in a tool result is not refused: a provider that takes no image input gets placeholder text where the image was, named `messages.content.tool_result.image->placeholder`. Chat Completions answers 400 `unsupported_capability` with `metadata.param` set to `messages`. Messages answers 400 `invalid_request_error` and ends the message with `(param: messages)`. Send the image inline (a `data:` URL on Chat Completions, a base64 source on Messages) when a URL is refused, or choose a model that lists `image` in its `architecture.input_modalities`.

## Related

- [Chat Completions](chat-completions.md#the-messages-array) and [Messages](messages.md#request-fields) list every field of their API.
- [Errors](errors.md) lists every code.

# Prompt caching

A provider that caches a prompt bills the repeated start of your next request at its cached-input price, a fraction of its input price. Most providers cache on their own. Claude caches only where your request marks it, and llm.buzz never adds a mark for you.

## Which providers cache

- **Claude models** (`anthropic/...`) cache a prompt only where the request carries a cache marker, on every provider. A request without one, such as a plain OpenAI SDK or Codex call, pays the full input price on every turn.
- **OpenAI models, and most other models**, cache a repeated prefix on their own, without a marker. Some providers do not cache at all.

`GET https://api.llm.buzz/api/v1/models/{author}/{slug}/endpoints` says what each provider of a model does. `pricing.input_cache_read` is its cached-input price, and `supports_implicit_caching` is `true` when it caches a request without markers at a lower price. The [Models page](/models) marks the models that cache.

## Mark a prompt for Claude

From the OpenAI SDK, send Anthropic's `cache_control` at the top level of the request. The provider puts the cache breakpoint on the last block it can cache, so the breakpoint moves with the conversation and each turn reads what the turn before it wrote. On Chat Completions, a marker on the last function tool also caches the tool definitions.

```python tab="OpenAI SDK"
reply = client.chat.completions.create(
    model="anthropic/claude-sonnet-5.5",
    messages=messages,
    extra_body={"cache_control": {"type": "ephemeral"}},
)
```

```python tab="Anthropic SDK"
message = client.messages.create(
    model="anthropic/claude-sonnet-5.5",
    max_tokens=1024,
    system=[{"type": "text", "text": instructions, "cache_control": {"type": "ephemeral"}}],
    messages=messages,
)
```

A marker is `{"type": "ephemeral"}`, with an optional `ttl` of `"5m"`, the default, or `"1h"`. `null` carries nothing, and any other value is a 400 that names where it was, such as `cache_control` or `tools.1.cache_control`.

| Surface | Where a marker goes |
|---|---|
| Chat Completions | The top level, function tools, messages, text content parts and assistant tool calls. A marker on a server tool, such as `openrouter:tool_search`, is a 400. |
| Responses | The top level only. A marker on a tool is a 400. |
| Messages | Every place the Anthropic Messages API takes one, as the Anthropic SDK sends it. |

A marker reaches every provider of the model that takes Claude's markers. A provider that caches on its own, such as OpenAI, receives none. `x-sgr-ignored-parameters` names a marker that stayed behind, as `cache_control->not_forwarded(provider_decides_caching; ...)`: a top-level or message marker when no provider of the model can take it, and a tool marker when any provider cannot.

Claude takes at most 4 breakpoints in a request, and the top-level marker counts as one unless the last block of the last message already carries a marker. A one-hour breakpoint must come before every five-minute one, in the order the provider reads them: tools, then system, then messages, then the top-level marker. A provider that refuses such a layout is skipped and the model's other providers serve the request; when every provider refuses it, the answer is a 400 `invalid_parameter` with `param: cache_control` (Anthropic's `invalid_request_error` on Messages), and its message says what to remove or move.

On Claude Sonnet 5.5, whose cached input costs a twentieth of its input, a 40-turn session that grows from 20,000 to 120,000 tokens pays about $5.60 for its input without a marker, and under $1 with one top-level marker.

## What a cached read costs

Each provider lists its cache prices beside its input and output prices, in US dollars per token on the endpoints route and per million tokens on the model pages:

| Price | On the endpoints route | What it bills |
|---|---|---|
| Cached input | `pricing.input_cache_read` | Input read from the cache. |
| Cache write | `pricing.input_cache_write` | Input written to the cache for five minutes. |
| One-hour cache write | `pricing.input_cache_write_1h` | Input written to the cache with `ttl: "1h"`. |

A request bills its cached input at the cached price, the input written to the cache at the write price, and the rest of its input at the input price, and `usage.cost` includes all three. Claude Sonnet 5.5, for example, lists $2 per million input tokens, $0.10 cached, $2.50 for a five-minute write and $4 for a one-hour write.

## Read the cache in usage

| Surface | Fields |
|---|---|
| Chat Completions | `usage.prompt_tokens_details.cached_tokens` (read from the cache), `cache_write_tokens` (written to it) and `cache_write_1h_tokens` (the one-hour part of the writes) |
| Responses | `usage.input_tokens_details.cached_tokens`, `cache_write_tokens` and `cache_write_1h_tokens` |
| Messages | Anthropic's `cache_read_input_tokens`, `cache_creation_input_tokens` and `cache_creation`, its split of the writes into five-minute and one-hour; `input_tokens` counts only the input that was not cached |

On Chat Completions and Responses, the reads and the writes are separate parts of the prompt's token count. `cache_write_1h_tokens` appears only when the provider reports the split. Chat Completions shows a field only when the provider reported it; Responses always shows `cached_tokens` and `cache_write_tokens`, with `0` when the provider reported none.

## Keep a conversation on its cache

A cache lives at one provider. Name the conversation with `session_id` in the body or the `x-session-id` header, on every turn, and its turns stay on the provider that holds its warm cache, ahead of a cheaper or faster provider. On Chat Completions and Responses, `prompt_cache_key` names it too. A client that sends none of these still gets a session from the conversation's opening: its first system text and first message.

The conversation moves when that provider fails or your request can no longer use it, such as a `provider.only` that leaves it out, and a `provider.order` in the request turns this off.

Providers keep their caches per account, and llm.buzz calls each provider from its own account for every organization. So two organizations that send the same prompt prefix to the same provider can share its cache entry, and how fast an answer comes back can show that someone sent that prefix recently.

## OpenAI's explicit caching

OpenAI's GPT-5.6 and later models, such as `openai/gpt-5.6-luna` and `openai/gpt-6-sol`, take OpenAI's explicit caching controls on Chat Completions and Responses, as the OpenAI SDK sends them:

```python
reply = client.responses.create(
    model="openai/gpt-5.6-luna",
    input=[{"role": "user", "content": [
        {"type": "input_text", "text": long_context, "prompt_cache_breakpoint": {"mode": "explicit"}},
        {"type": "input_text", "text": question},
    ]}],
    extra_body={"prompt_cache_options": {"mode": "explicit"}},
)
```

- `prompt_cache_breakpoint: {"mode": "explicit"}` marks the end of a prefix to reuse, on a text part: on Chat Completions a `text` part of a system, developer, user or tool message, and on Responses an `input_text` part of a system, developer or user message or of a `function_call_output`. On any other part, or on an assistant message, it is a 400 that names where it was.
- `prompt_cache_options` takes `mode`, `implicit` (OpenAI's default) or `explicit` (OpenAI then writes the cache only at breakpoints), and `ttl` `30m`.
- `prompt_cache_retention` is `in_memory` or `24h`. `24h` keeps prefixes for up to 24 hours, so on a request with `provider.zdr: true`, `provider.data_collection: "deny"` or your organization's no-retention policy it is a 400 `invalid_parameter` on `prompt_cache_retention`. OpenAI may refuse `in_memory` on GPT-5.5 and later, so on such a request to those models send no retention at all.

OpenAI receives these controls as you send them. Sent to a Claude model, a breakpoint becomes a five-minute Claude marker, named in `x-sgr-ignored-parameters` as `prompt_cache_breakpoint->translated(cache_control)`, and the options and retention are not sent. A provider that cannot take them receives none of them, and when no provider of the model can, `x-sgr-ignored-parameters` names them. Your `cache_control` never becomes an OpenAI breakpoint, so you never pay for an OpenAI cache write you did not ask for. OpenAI reports its writes in `cache_write_tokens`, billed at the five-minute write price.

# Authentication

Every request carries one of your keys. You create keys in the console, send them in the `Authorization` header, and can give each one a spending limit.

## Create a key

Open [Keys](https://app.llm.buzz/console/keys) in the console and create a key. A key looks like `exp_vk_...`. The console shows the whole key once, when you create it, so copy it then and keep it in an environment variable or a secret store.

A key is one of two kinds:

| Kind | What it calls |
|---|---|
| Inference | The model API: Chat Completions, Responses, Messages and its `count_tokens`, Embeddings, Images, audio, the generation record (`/api/v1/generation`) and the model listings. |
| Management | Only the key routes: `/api/v1/key`, `/api/v1/credits` and `/api/v1/keys`. |

A key expires after 30, 90 or 365 days, or never, as you choose when you create it. On the Keys page you can disable a key and enable it again, or revoke it for good.

## Send the key

Send the key as a bearer token on every request:

```bash
curl https://api.llm.buzz/api/v1/models \
  -H "Authorization: Bearer $LLM_BUZZ_KEY"
```

The Messages API (`/api/v1/messages`) also takes the key in `x-api-key`, the header the Anthropic SDK sends. Point the Anthropic SDK at `https://api.llm.buzz/api`, without `/v1`: the SDK adds `/v1/messages` itself.

```python tab="Anthropic SDK"
from anthropic import Anthropic
import os

client = Anthropic(
    base_url="https://api.llm.buzz/api",
    api_key=os.environ["LLM_BUZZ_KEY"],
)
message = client.messages.create(
    model="deepseek/deepseek-v4.1-flash",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello"}],
)
print(message.content[0].text)
```

```bash tab="Claude Code"
export ANTHROPIC_BASE_URL="https://api.llm.buzz/api"
export ANTHROPIC_AUTH_TOKEN="$LLM_BUZZ_KEY"
export ANTHROPIC_MODEL="anthropic/claude-sonnet-5.5"
export ANTHROPIC_DEFAULT_HAIKU_MODEL="anthropic/claude-haiku-5.5"
claude
```

Model names are matched exactly, and Claude Code's own default model names do not resolve on llm.buzz, so set every model it uses to a name from llm.buzz: `ANTHROPIC_MODEL` for the main model, and `ANTHROPIC_DEFAULT_HAIKU_MODEL` for the small model it runs background tasks on. If you switch models with `/model`, set `ANTHROPIC_DEFAULT_SONNET_MODEL` and `ANTHROPIC_DEFAULT_OPUS_MODEL` too. Any model on llm.buzz works there, Claude or not.

## Organizations

Keys, credit and logs belong to an organization. Signing up creates your own organization, and you can join others by invitation: an organization's owners and admins invite its members from the console, each with a role (owner, admin, billing or member). In an organization, each member has their own keys and logs, every member's requests draw on the organization's one prepaid balance, and its admins set its data policy and prompt logging on the Prompt logs page. The console's organization switcher picks the organization you work in.

## Check a key

`GET /api/v1/key` describes the key you send: its name, its limit, what is left of it and when it resets, its spend today, this week, this month and in total, and when it expires.

```bash
curl https://api.llm.buzz/api/v1/key \
  -H "Authorization: Bearer $LLM_BUZZ_KEY"
```

## Spending limits

An inference key can carry a limit in US dollars, per day, per week, per month or for the key's whole life. Days, weeks and months are UTC; a week starts on Monday. Set the limit when you create the key, or change it later on the Keys page. A management key can change it with `PATCH /api/v1/keys/{hash}`.

The limit is checked when a request arrives, against the key's spend in the current period. The request that crosses the limit completes, and the next one is refused with 403 `key_limit_exceeded`. Its `metadata.resets_at` says when the period ends, and is `null` for a lifetime limit. Model listings spend nothing and are never refused for the limit.

## When a key is refused

| Status | Code | What it means |
|---|---|---|
| 401 | `invalid_key` | The request has no key, or the key is unknown, expired, revoked or disabled. |
| 401 | `missing_authentication` | A key route was called without an `Authorization` header. |
| 403 | `key_limit_exceeded` | The key's spend in its period has reached its limit. Wait for `resets_at`, or raise the limit. |
| 403 | `management_key_not_allowed` | A management key called a model route. Use an inference key. |
| 403 | `management_key_required` | An inference key called `/api/v1/keys` or `/api/v1/credits`. Use a management key. |

# Models and routing

Each model on llm.buzz is served by one or more providers. You name the model; llm.buzz picks a provider, moves to the next one when a provider fails, and follows your preferences when you give them.

## Model names

A model's name is `author/slug`, such as `qwen/qwen3.8-27b`, or a bare slug. Names use lowercase letters and digits joined by `.`, `_`, `:` or `-`, and are matched exactly. A variant such as `:thinking` is part of the name.

`GET https://api.llm.buzz/api/v1/models` lists the models your key can call. `GET https://api.llm.buzz/api/v1/models/{author}/{slug}/endpoints` lists one model's providers, each with its prices, limits and `tag`; for a bare name it is `/models/{name}/endpoints`. Both answer without a key too, for the public models.

## Suffixes

A suffix after the name changes one request:

| Suffix | Effect |
|---|---|
| `:floor` | The cheapest provider first. |
| `:nitro` | The fastest provider first, by recent time to first token. |
| `:exacto` | Accepted for compatibility, and ignored. |
| `:online` | Asks for a web search on Chat Completions and Responses. llm.buzz runs no web search today: see below. |

```python
model="qwen/qwen3.8-27b:floor"
```

llm.buzz runs no web search today. A request that asks for one, with `:online`, `web_search_options` or `plugins: [{"id": "web"}]`, is still served, without the search: nothing is sent to a search service, no search is billed, and `x-sgr-ignored-parameters` names it as `web_search->dropped(search_unavailable)`. For an organization whose requests llm.buzz keeps in the EU, such a request answers 400 `web_search_unavailable_in_region` instead.

## How a provider is chosen

Each model has a routing preset that orders its providers:

| Preset | Order |
|---|---|
| `smart` | By price and time to first token together. On a model with the llm.buzz provider, llm.buzz goes first and the other providers are the failover. The default: every new model starts on it, and a model on `order` takes it when its first independent provider goes live. |
| `cheapest` | By the estimated cost of the request at each provider's price. |
| `fastest` | By recent time to first token. |
| `order` | In the order llm.buzz set. |

The request goes to the first provider in that order. If it errors or times out, the next provider for the same model answers.

`provider.sort` overrides the preset for one request: `"price"` works as `:floor`, `"latency"` and `"throughput"` as `:nitro`, and `"smart"` ranks by price and speed.

```python
extra_body={"provider": {"sort": "price"}}
```

## Which providers serve

A model serves one API: every provider of it serves every capability the model declares. Before a provider serves a model, an automated check sends it real requests for each one: a streamed answer with usage, then tool calls, structured output and image input where the model declares them. A provider that fails stays out of routing.

Each provider's prices follow the prices it publishes, such as DeepInfra's, Novita's or Scaleway's price lists, and sync about every six hours. Where a provider publishes no price list, its prices come from its public listed prices, or llm.buzz sets them by hand; a price set by hand is what you pay and what the model pages show. Scaleway lists its prices in euros, converted to US dollars at the European Central Bank's daily rate plus 3%, the cost of converting. Every price you see or pay is in US dollars.

## Discounts

A provider may run a discount for a set time. While it runs, the model page shows a shield and "N% off" next to that provider, with each price struck through beside the price you pay, and every request that provider serves is billed at the discounted price, including requests that started before the discount ended. `smart` and `cheapest` routing compare the discounted prices. `GET https://api.llm.buzz/api/v1/models/{author}/{slug}/endpoints` lists the discounted prices with `discount`, the fraction off (`0.51` for 51% off), as OpenRouter does, and `0` when none runs. The price history on a model page shows prices before discounts.

## Provider preferences

The `provider` object narrows or reorders the providers of one request, as on OpenRouter. It works on Chat Completions, Responses, Messages, Embeddings and the image routes. On the audio routes only `zdr` and `data_collection` apply; see [Audio](audio.md).

| Field | Effect |
|---|---|
| `order` | Try these providers first, in this order, then the others in the preset's order. |
| `only` | Use only these providers. |
| `ignore` | Never use these providers. |
| `allow_fallbacks` | `false` stops moving to other providers: without `order` only the first provider is tried, and with `order` only the listed ones. |

A name matches a provider's display name, its tag or the tag's first part, ignoring case. Tags come from the endpoints route; an EU provider's tag ends in `/eu`, such as `scaleway/eu`. With an `order`, sort suffixes and `provider.sort` no longer apply.

```python
extra_body={"provider": {"order": ["deepinfra"], "allow_fallbacks": False}}
```

When your preferences leave the model no provider, the answer is 404 `no_allowed_providers`, and its message names the providers that serve the model. Nothing is sent to a provider.

How a request moves to the next provider, and what a failed attempt or a retry of your own costs, is on [Retries and idempotency](retries-and-idempotency.md).

## The llm.buzz provider

Many models list a provider named llm.buzz beside the companies that serve them directly. It is llm.buzz's own route to the model, and it takes part in routing and failover like any other provider.

- **Its price.** llm.buzz has no price list of its own. For each price (input, cached input, cache writes, output and reasoning) it bills the lowest price among the model's live direct providers, their running discounts applied, and any discount llm.buzz runs on top. A request it serves never costs more than the cheapest direct provider of each price. The model pages show its prices with an Auto tag, and the price history shows only the direct providers.
- **Its place in the order.** Under `smart` it goes first while it is healthy, and the model's direct providers are the failover. Under `cheapest`, `fastest` and `order`, and in your own `provider.order`, it ranks like any provider.
- **Its names.** `llm.buzz` or `llm-buzz` in `provider.order`, `only` or `ignore` names it, and its tag on the endpoints route is `llm-buzz`. `{"ignore": ["llm.buzz"]}` keeps a request on the direct providers. `llm-buzz/eu` names only its capacity in the EU; when it has none for the model, the request answers 404 `no_allowed_providers`, saying so.
- **Its answers.** The answer's `provider` field and its `x-gateway-provider` header say `llm.buzz`, and `usage` carries the tokens billed at its price.
- **Its data policy.** Its declared data policy holds for every request it serves, and `provider.zdr` and `provider.data_collection` judge it by that declaration, as they judge any provider.

## EU

Each provider serves from the EU or from outside it, and an EU provider's tag ends in `/eu`. To keep a request on EU providers, list their EU tags in `provider.only`:

```python
extra_body={"provider": {"only": ["scaleway/eu", "simpleminds-eu/eu", "surogate-spark/eu"]}}
```

A request that continues an earlier one (a Responses `previous_response_id`, or a reasoning continuation) can stay on the provider that served the first, whatever `only` says.

For an organization whose requests llm.buzz keeps in the EU, a model with no EU provider answers 404 `model_not_available_in_region`.

## Data policy

Each provider declares whether it trains on prompts and how many days it keeps them. Two request fields keep a request away from every provider whose declaration does not qualify:

| Field | Providers that qualify |
|---|---|
| `provider.zdr: true` | Declare that they keep prompts for 0 days. |
| `provider.data_collection: "deny"` | Declare that they do not train on prompts and keep them for 0 days. |

```python
extra_body={"provider": {"data_collection": "deny"}}
```

Your organization can require the same of every request: on the console's Prompt logs page, set its data policy to no training and no retention. It applies from the next request, and a request can tighten it but never loosen it.

A provider that has not declared a policy never qualifies. When no provider of the model qualifies, the answer is 404 `model_not_available_under_data_policy`, and nothing is sent to a provider.

For OpenRouter's privacy settings, see [Coming from OpenRouter](from-openrouter.md#prompt-logs).

## Prompt logs

llm.buzz keeps the prompts and answers of your organization's completed Chat Completions, Responses and Messages requests in prompt logs, and logging is on by default; Embeddings, Images and the Responses WebSocket are not logged. Your organization's admins read the logged prompts, turn logging off, and delete every logged prompt on the console's Prompt logs page, the page that also sets the organization's data policy.

- **What is never logged.** A request with `provider.zdr: true`, with `provider.data_collection: "deny"`, or sent while your organization's data policy is no training and no retention is never logged, whatever the logging setting says.
- **How long.** Logged prompts are kept for 30 days by default; llm.buzz sets one window for every organization, and the Prompt logs page shows the number of days in force.
- **Turning logging off** applies from the next request. It removes nothing already logged: a logged prompt stays until its days pass or an admin deletes it.
- **Deleting** removes every logged prompt of your organization, whichever member's request it was. They disappear from the page at once and are removed within about a minute. Requests sent a few seconds after the delete are logged again while logging is on.
- **What stays.** Usage, cost and the request log stay when you turn logging off or delete.
- **Not prompt logs.** llm.buzz holds a Chat Completions or Responses request sent with an `Idempotency-Key` for up to 24 hours, so a retry can return its answer, and a Responses turn for up to 24 hours, so `previous_response_id` can continue it. The console does not show either, and neither follows the logging setting or a zero-retention demand. A Responses request with `store: false` keeps its turn out.

# Pricing and discounts

You pay per token, or per second of audio, per character or per image on a model priced that way, at the price of the provider that answered your request, from prepaid credit. Every price is public: on each model's page, in the model list, and in the market log.

## How a request is priced

Each provider of a model sets its own prices for input, output, cached input and cache writes. A request is billed at the prices of the provider that answered it, so when a request fails over, you pay the provider that produced the answer, and the failed attempts cost nothing. The llm.buzz provider has no price list of its own: for each price it bills the lowest of the model's direct providers, shown with an Auto tag ([The llm.buzz provider](models-and-routing.md#the-llmbuzz-provider)).

- `usage.cost` in every answer is the request's cost in US dollars, when every part of it is known.
- `GET /models/{author}/{slug}/endpoints` lists each provider's prices, in US dollars per token, and the `tag` you can name in `provider.order` or `provider.only`.
- The model list shows one price per model: with your key, the average over the providers that may serve you; without a key, the price of the provider requests go to first.
- The site shows prices per million tokens, except where a price names its unit (per million seconds of audio or characters, or per image). The API lists them as OpenRouter does: per token, or per second of audio or per character on an audio model, and an image model's output price in `pricing.image_output`, as [Images](images.md) explains.

Add `:floor` to a model's name, or send `provider: {"sort": "price"}`, to try the cheapest provider first. [Models and routing](models-and-routing.md) explains the order providers are tried in.

## Prepaid credit

Requests draw on your organization's credit: one balance that every member's keys share. Buy $5 to $25,000 of credit on the console's Credits page. A service fee of 5.5%, at least $0.80, is added on top:

| Credit | Service fee | Charged |
|---|---|---|
| $10 | $0.80 | $10.80 |
| $25 | $1.38 | $26.38 |
| $100 | $5.50 | $105.50 |

With auto top-up on, llm.buzz charges a saved card for an amount you choose, $5 to $25,000 plus the service fee, whenever your balance falls below your threshold, $2 to $25,000. It tops up at most 5 times per UTC day. It turns itself off, and the Credits page says why, when the card is declined, when the bank asks to confirm the payment (buy credit once with the card, then turn it on again), when the day's 5 top-ups are used, and when the saved card is removed with no default card left.

When the balance is spent, a request to a paid model answers 402 `insufficient_credits`; free models still answer.

You can refund a purchase or an auto top-up from the Credits page for 24 hours after the payment. It refunds the smaller of the payment's credit not yet refunded and your balance less what running requests hold. The service fee is never refunded.

## Provider discounts

A provider can run a discount for a set period, a whole percent off every price it bills. While it runs:

- requests routed to that provider are priced, budgeted and ranked at the discounted price, and `usage.cost` shows it;
- the endpoints route lists the discounted prices with `pricing.discount`, the fraction taken off, such as `0.51` for 51% off;
- the site shows the original price struck through beside the discounted one, with the end date.

A request admitted before a discount ends pays the discounted price even if it finishes after the end. A discount never changes which provider holds a conversation's cache.

## Free offers

A provider discount of 100% makes that provider free until the discount ends, up to an allowance: a number of input and output tokens for each organization over the whole offer, and a number of requests per minute for each key. The site marks such a provider "free", with the end date.

- Once your organization has used its tokens, or your key reaches the per-minute limit, the model's other providers serve the request at their own prices.
- A model with no other provider then answers 429 `free_allowance_exceeded`. Its message names the allowance and when the offer ends. For the per-minute limit it carries `Retry-After: 60`: retry after a minute, or choose a paid model.
- Requests already running when the allowance runs out finish, so use can go a little over it.

## Offers on the site

The banner at the top of every page carries the current offer: a promotion the llm.buzz team sets, such as a model that is free for a week, or else the biggest provider discount running now. Close it and it stays closed until the next offer.

## The market log

The [market log](/market) is the price history of the whole market, newest first: new models, new providers serving a model, and every change to a provider's input or output price, each dated and linked to the model. It has an RSS feed for agents and dashboards.

It also lists provider discounts as they start and end, and the prices other providers list for the same models, read from their public listings, with how much the llm.buzz price saves against each. Every row says where its price comes from and when it was read.

## The inventory

The [inventory](/inventory) lists everything on llm.buzz in one place: every provider, model, app and agent, with which serve from the EU. Providers and models the team is bringing on appear there as coming soon before they serve.

# Usage and limits

Every answer says what the request used and what it cost, and a few limits decide when a request is refused. This page lists both, and where to read them again afterwards.

## Usage in each answer

Each API reports usage under its own field names and adds `cost`: the request's price in US dollars at your organization's rate.

| API | `usage` holds | Where |
|---|---|---|
| Chat Completions | `prompt_tokens`, `completion_tokens`, `total_tokens`, the cache and reasoning counts in `prompt_tokens_details` and `completion_tokens_details`, and `cost` | The body. On a stream, the last chunk. |
| Responses | `input_tokens`, `output_tokens`, `total_tokens`, the counts in `input_tokens_details` and `output_tokens_details`, and `cost` | The response object. On a stream, the final response in the last event. |
| Messages | `input_tokens`, `output_tokens`, `cache_creation_input_tokens`, `cache_read_input_tokens` and `cost` | The message. On a stream, `message_delta`: `message_start` carries an early count and no `cost`. |
| Embeddings | `prompt_tokens`, `total_tokens` and `cost` | The body. |
| Images | On `POST /images`: `prompt_tokens`, `completion_tokens`, `total_tokens` and `cost`. On `POST /images/generations`: `input_tokens`, `output_tokens`, `total_tokens`, `input_tokens_details` and `cost`. | The body. |
| Speech to text | `seconds` and `cost` | The body. |
| Text to speech | None: the answer is audio. | [The generation record](#the-cost-of-a-finished-request). |
| Decisions | `input_tokens`, `output_tokens` and `cost` | The body. |

The fields are described where each API is: [Chat Completions](chat-completions.md#usage-and-cost), [Responses](responses.md#usage-and-cost), [Messages](messages.md#usage-and-cost), [Embeddings](embeddings.md#create-embeddings), [Images](images.md#the-answer), [Audio](audio.md#cost-and-the-generation-record) and [Decisions](decisions.md#the-answer). [Prompt caching](prompt-caching.md#read-the-cache-in-usage) lists the cache counts.

- `cost` is a JSON number. It includes any billed tool-search rounds, so it can be more than the counts times the listed prices. A model with no price reports `0`.
- A failed attempt that another provider replaced adds 0 to `cost`. A retry with the same `Idempotency-Key` returns the first call's `cost` and is not billed again: [Retries and idempotency](retries-and-idempotency.md#idempotency-key).
- `cost` is left out when any part of it is unknown, which happens when the provider reported no counts for a turn that delivered output: [When a provider reports no usage](#when-a-provider-reports-no-usage).
- On Chat Completions and Responses, a count that is `0` only because the provider reported nothing is named in `usage.unreported_token_details`. The names it can hold are `cached_tokens`, `cache_write_tokens`, `cache_write_1h_tokens` and `reasoning_tokens`. The `usage` in the last chunk of a Chat Completions stream from such a provider looks like this:

```json
{"prompt_tokens": 12, "completion_tokens": 7, "total_tokens": 19,
 "prompt_tokens_details": {"cached_tokens": 0}, "completion_tokens_details": {"reasoning_tokens": 0},
 "unreported_token_details": ["cached_tokens", "cache_write_tokens", "reasoning_tokens"],
 "cost": 0.000026}
```

- You do not need OpenRouter's `usage: {"include": true}`. llm.buzz drops it, names it in `x-sgr-ignored-parameters`, and sends `usage` and `cost` with every answer whose provider reported counts.

## The cost of a finished request

`GET /generation?id=` answers the record and cost of any finished request of your organization. The ID is `gen-` and the 32 hex digits of the request's `x-request-id`:

```bash
curl "https://api.llm.buzz/api/v1/generation?id=gen-..." \
  -H "Authorization: Bearer $LLM_BUZZ_KEY"
```

`total_cost` is what you were charged, in US dollars at your rate. Text to speech has no JSON body for its cost, so this record is where you read it. [Cost of a finished request](api-reference.md#cost-of-a-finished-request) lists the rest of the record. An error answer carries no `x-request-id`, so look for that request on the Logs page.

## When a provider reports no usage

Occasionally a provider answers without token counts, or with counts that are all zero. What llm.buzz charges depends on what the attempt delivered:

- A turn that finished without any text or tool output is charged an estimate of its prompt: llm.buzz's own count of the input tokens, with the cached share at the cached rate, and nothing for output. A finished turn is a stop, a stop cut by your token limit, or an empty completion on a stream that opened. The estimate applies to the request's last attempt, and an attempt that another provider replaced costs 0. The answer keeps the counts the provider sent: no `usage` on Chat Completions and Responses when it sent none, zeros otherwise, and zeros on Messages. Where the answer has a `usage`, its `cost` is the estimate. The generation record shows the estimate as `total_cost`.
- A turn that delivered text, reasoning text or a tool call, or ran a tool search, is charged what llm.buzz reserved when the attempt started: the most the attempt could cost, which is the prompt plus the output limit at the provider's prices and your organization's rate. The answer carries no `cost`: no `usage` on Chat Completions and Responses when the provider sent no counts, and zero counts on Messages. The generation record shows the amount as `total_cost`.

A lower `max_tokens` lowers the reserved amount. Without one, the output limit is the model's own ceiling, so a model with a large ceiling reserves more. On Chat Completions, Responses and Messages, a request whose last provider failed before its stream opened costs nothing when that provider answered an error status (a 5xx, its own 400 or a 429) or never received the request: you received nothing. The reserved amount is charged when the last provider timed out, when it failed after the request was sent, when a stream opened and then broke off without usage, and when you close a stream before the provider accepted the request ([Stopping a stream](streaming.md#stopping-a-stream)), because the provider may have generated output you never saw: [Retries and idempotency](retries-and-idempotency.md#inside-one-request). Embeddings, Images and Audio bill a failed last attempt by their own rules, often nothing: [Embeddings](embeddings.md#billing), [Images](images.md#billing) and [Audio](audio.md#billing).

## In the console

The console shows the same figures over time:

- [Activity](https://app.llm.buzz/console/activity) shows spend, requests, tokens, latency and error rate over 1 to 90 days, a per-day chart by model, and breakdowns by model, provider and calling app.
- [Logs](https://app.llm.buzz/console/logs) lists every request, filtered by state and model. A request opens to its route timeline: the attempts it made, in order.
- [Keys](https://app.llm.buzz/console/keys) lists each key with its limit against its spend, its spend today, this week and this month, its last use and its status.
- [Credits](https://app.llm.buzz/console/credits) shows your balance, the amount your running requests hold, and your credit history.

These pages show your own requests and keys. An organization's admins can switch Activity, Logs and Keys to the whole organization's.

llm.buzz does not serve OpenRouter's `/activity` or analytics routes: they answer 404 `unknown_route`. Read totals on Activity, and one request with `GET /generation`.

## Limits

| Limit | What you get | What to do |
|---|---|---|
| Your balance is spent | 402 `insufficient_credits`. A free model still answers. | Add credit on the Credits page, or turn on auto top-up: [Pricing](pricing.md#prepaid-credit). |
| A request could cost more than your balance | 402 `weight_exceeds_budget` | Add credit, or lower `max_tokens`. |
| Running requests hold what your balance leaves | 402 `in_flight_budget_exhausted`, with `Retry-After: 5` | Retry after the wait, or add credit. |
| Your organization's monthly budget is spent | 402 `insufficient_quota`, with `Retry-After`; the message names when the month resets | Wait for the reset, or ask an owner or admin of your organization to raise the budget on the console's Keys page. |
| A key's spending limit is reached | 403 `key_limit_exceeded`, with `metadata.resets_at` | Wait for `resets_at`, or raise the key's limit: [Authentication](authentication.md#spending-limits). |
| A free offer's allowance is used, and the model has no other provider | 429 `free_allowance_exceeded`. The per-minute limit carries `Retry-After: 60`. | Choose a paid model, or retry after a minute: [Pricing](pricing.md#free-offers). |
| Providers are throttling, or the model is at capacity | 429 `unavailable_route`, with `Retry-After` | Wait that long, then retry: [Retries and idempotency](retries-and-idempotency.md#your-retries). |
| The prompt fits no provider's context window | 400 `context_length_exceeded` | Shorten the prompt. A model's `context_length` is on its entry in the model list. |
| A request runs for 600 seconds | 504 `deadline_exceeded`. A stream ends with a timeout error and is billed for the prompt and the output you received. | Ask for a shorter answer, for example with a lower `max_tokens`. |
| The request body is over 64 MiB, or an audio file is over 25 MiB | 413 `request_too_large` | Send less. |

Chat Completions serves one completion per request, so `n` is 1. A request makes at most 8 provider calls, every fallback included; `gateway.retry` lowers that ceiling: [Retries and idempotency](retries-and-idempotency.md#the-gateway-object). 402 `weight_exceeds_budget` and `in_flight_budget_exhausted` apply when your balance is low, because llm.buzz then holds the most each running request could cost against it. [Errors](errors.md) lists every code, and each API's page lists the limits of its own.

## Check your key and credit

`GET /key` describes the key you send, with an inference key or a management key:

```bash
curl https://api.llm.buzz/api/v1/key \
  -H "Authorization: Bearer $LLM_BUZZ_KEY"
```

- `limit`, `limit_remaining` and `limit_reset` give the key's spending limit, what is left of it and its period (`daily`, `weekly`, `monthly`, or `null` for the key's whole life). `budgets` lists the limit with its `spend_usd`, `remaining_usd` and `resets_at`, the start of the next period, and is empty for a key without a limit.
- `usage`, `usage_daily`, `usage_weekly` and `usage_monthly` are the key's spend in US dollars over its life and in the current UTC day, week and month.
- `is_free_tier` is `true` until your organization has bought credit.
- `free_model_daily_requests` reads 0 for `limit`, `remaining` and `used`. `rate_limit` is deprecated and safe to ignore.

`GET /credits` answers your organization's credit and usage in US dollars. It takes a management key; an inference key gets 403 `management_key_required`:

```bash
curl https://api.llm.buzz/api/v1/credits \
  -H "Authorization: Bearer $LLM_BUZZ_MANAGEMENT_KEY"
```

```json
{"data": {"total_credits": 25.0, "total_usage": 3.25}}
```

An organization without credit gets zeros. [Authentication](authentication.md#create-a-key) explains the two kinds of key.

# Retries and idempotency

llm.buzz retries across a model's providers inside one request, and an `Idempotency-Key` makes your own retries safe. This page says what each does, and what a retry costs.

## Inside one request

A model usually has several providers. Until a provider sends its first text, tool call or refusal, a provider that fails or times out is replaced by the next one, in the order the model's routing preset gives ([How a provider is chosen](models-and-routing.md#how-a-provider-is-chosen)). llm.buzz does not call a provider again for an ordinary failure. The answer's `x-gateway-provider` names the provider that answered, and `x-gateway-route-depth` says where it stood in the order, counting from 0.

Replacing a provider stops when:

- **Output has started.** The first text, tool call or refusal fixes the provider for the rest of the request, and llm.buzz never splices another provider's output into it. A later failure ends the answer with an error, or as incomplete when the provider's stream just stops: [Streaming](streaming.md#failures).
- **The model refused.** A refusal reaches you from the provider that refused it and is not tried on another.
- **The provider rejected the request.** A provider's own 400, whose message starts `provider rejected the request:`, reaches you and is not tried on another provider.
- **A ceiling is reached.** A request makes at most 8 provider calls and runs for at most 600 seconds, and the bounds you set with [`gateway`](#the-gateway-object) apply.

What a Chat Completions, Responses or Messages request costs when providers fail:

| What happened | What you pay |
|---|---|
| A provider failed and another provider answered | The provider that answered. The failed attempt adds 0. |
| A provider billed tokens but sent no text, tool call or refusal, and another provider answered | The provider that answered. llm.buzz treats the empty answer as a failure, so it adds 0. |
| Every provider sent an empty answer | The empty turn comes back as a 200 with the header `x-gateway-warning: empty_completion`, on Chat Completions, Responses and Messages. You pay for the usage the last provider reported, or, when it reported none, an estimate of its prompt: [When a provider reports no usage](usage-and-limits.md#when-a-provider-reports-no-usage). |
| The last provider tried failed before its stream opened, and reported no usage, because it answered an error status (a 5xx, its own 400 or a 429) or never received the request | Nothing: you received nothing. The request answers an error. |
| The last provider tried finished without any text or tool output and reported no usage or only zeros | An estimate of the prompt only: llm.buzz's own count of the input tokens, and nothing for output: [When a provider reports no usage](usage-and-limits.md#when-a-provider-reports-no-usage). |
| The last provider tried timed out, failed after the request was sent, or broke off after its stream opened (a stream that stopped after its output started included), and reported no usage | What llm.buzz reserved for that attempt: [When a provider reports no usage](usage-and-limits.md#when-a-provider-reports-no-usage). The request answers an error, or ends as incomplete. |

Embeddings, Images and Audio bill a failed last attempt by their own rules, often nothing: [Embeddings](embeddings.md#billing), [Images](images.md#billing) and [Audio](audio.md#billing).

A request llm.buzz refuses before it calls any provider costs nothing. That covers a 400 for a bad field, a 401, a 404, a 402 for a spent balance, and a 503 when no provider is live.

## The gateway object

`gateway` sets bounds for one request on Chat Completions, Responses and Messages:

- `retry.max_attempts_per_route` caps the calls to one provider, 1 to 4. It is a cap and adds no retries, because a failed provider is not called again.
- `retry.max_total_attempts` caps the calls in all, 1 to 8, fallbacks included. It is 8 when you leave it out.
- `retry.backoff` sets the wait before a throttled provider is called again.
- `routing.allow_fallbacks: false` keeps the request on the first provider, as `provider.allow_fallbacks: false` does.

These are ceilings: your spending limits, the 600-second limit and a provider's health can stop a request sooner. [Retries and fallbacks per request](api-reference.md#retries-and-fallbacks-per-request) has the fields and an example. A retry with an `Idempotency-Key` must send the same `gateway` object: a changed one is a 409 `idempotency_conflict`.

## Your retries

Your own retries are requests of their own. Every one that runs is billed, and an error answer may have cost something, as the table above says.

| Answer | What to do |
|---|---|
| 429 | Wait `Retry-After` seconds, then retry. A free offer's token allowance is the exception: retrying cannot help, so choose a paid model. |
| 500, 502, 503, 504 | Retry with backoff, and after `Retry-After` seconds when the answer carries it. |
| Any other 4xx, apart from 402 `in_flight_budget_exhausted` (retry after `Retry-After`) and 409 `idempotency_replay_unavailable` ([below](#when-the-answer-is-not-stored)) | Do not resend it unchanged. Fix the field, the key or the balance first: [Errors](errors.md#retries). |

The OpenAI and Anthropic SDKs retry on their own, twice by default, after a connection error and after a 408, 409, 429 or 5xx answer. Each of their retries is a request of its own, and without an `Idempotency-Key` it runs and bills again. If you keep the SDK's retries and send an `Idempotency-Key`, the SDK resends the same key. After a dropped connection that replays safely, and after a 429 with `x-gateway-admission-refused: true` the retry runs the request once. After a 502, a 503 or a 429 `unavailable_route` without that header, its retries end in 409 `idempotency_replay_unavailable`, which is then the error you see. Set `max_retries=0` in Python or `maxRetries: 0` in TypeScript when you handle retries yourself, or when one operation must bound the work it does. Which `Idempotency-Key` to send with a retry is covered [below](#when-the-answer-is-not-stored).

## Idempotency-Key

Send an `Idempotency-Key` header and a retry with the same key and the same body never runs the request twice. Use one key for each distinct request, such as a UUID, and send the same key only when you retry that request.

| API | `Idempotency-Key` |
|---|---|
| Chat Completions | Replays the first answer. |
| Responses over HTTP | Replays the first answer. A key on the WebSocket upgrade does not apply to the frames sent over it. |
| Messages | Not read: Anthropic's API has no such header, so a retry is a new request. |
| Embeddings, Images, Audio and Decisions | Not read: a retry is a new request, served and billed again. |

```python tab="Python"
import os
import time
import uuid

from openai import APIConnectionError, APIStatusError, ConflictError, OpenAI

client = OpenAI(
    base_url="https://api.llm.buzz/api/v1",
    api_key=os.environ["LLM_BUZZ_KEY"],
    max_retries=0,
)


def summarize(text: str) -> str | None:
    key = str(uuid.uuid4())  # one key for this request and its retries
    for attempt in range(5):
        try:
            reply = client.chat.completions.create(
                model="deepseek/deepseek-v4.1-flash",
                messages=[{"role": "user", "content": f"Summarize in one sentence: {text}"}],
                max_tokens=200,
                extra_headers={"Idempotency-Key": key},
            )
            return reply.choices[0].message.content
        except (APIConnectionError, ConflictError):
            # No answer, or 409 idempotency_replay_unavailable: the request may have run.
            # Ask again with the same key.
            delay = 2**attempt
        except APIStatusError as error:
            if error.status_code not in (429, 500, 502, 503, 504):
                raise
            # An error answer ended the request, and its key cannot run it again.
            key = str(uuid.uuid4())
            delay = float(error.response.headers.get("retry-after", 2**attempt))
        if attempt < 4:
            time.sleep(delay)
    raise RuntimeError("no answer after 5 attempts")


print(summarize("HTTP clients retry with backoff so they do not add to an outage."))
```

```ts tab="TypeScript"
import OpenAI from "openai";
import { randomUUID } from "node:crypto";

const client = new OpenAI({
  baseURL: "https://api.llm.buzz/api/v1",
  apiKey: process.env.LLM_BUZZ_KEY,
  maxRetries: 0,
});
const key = randomUUID(); // one key for this request and its retries
const reply = await client.chat.completions.create(
  {
    model: "deepseek/deepseek-v4.1-flash",
    messages: [{ role: "user", content: "Summarize in one sentence: HTTP clients retry with backoff." }],
    max_tokens: 200,
  },
  { headers: { "Idempotency-Key": key } },
);
console.log(reply.choices[0].message.content);
```

```bash tab="curl"
KEY=$(uuidgen)
for attempt in 1 2; do
  curl -s https://api.llm.buzz/api/v1/chat/completions \
    -H "Authorization: Bearer $LLM_BUZZ_KEY" \
    -H "Idempotency-Key: $KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "deepseek/deepseek-v4.1-flash",
      "max_tokens": 200,
      "messages": [{"role": "user", "content": "Summarize in one sentence: HTTP clients retry with backoff."}]
    }' | jq -r '.id, .usage.cost'
done
```

The curl loop prints the same `id` and `cost` twice: the second call is the stored answer of the first.

What a retry gets:

- **After the first call answered:** the stored answer, with its original `cost` and `x-request-id`. No provider is called and nothing is billed. A stream comes back as the same stream.
- **While the first call is still running:** the retry waits for it and returns its answer. It does not start a second generation.
- **After you closed a keyed stream:** the request keeps running and is billed when it finishes, and a retry with the key joins it or gets its answer. A Chat Completions, Responses or Messages stream without a key stops reading from the provider when you close it.
- **With a different request:** 409 `idempotency_conflict`. Any change to the body counts, including `stream`, the `gateway` object, a `provider` hint and a model suffix such as `:floor`. A changed `stream_options.include_usage` does not count. Send a new key for each distinct request.
- **Under another data-policy demand:** a key replays only for a call that makes the same data-policy demand as the first one did: no demand, zero retention, or no training and no retention, whether it came from `provider.zdr`, `provider.data_collection` or your organization's data policy. The call must also run under the same residency, EU or anywhere. A retry that makes another demand is not a replay: it runs and bills as a new request.
- **After a limit or a spent balance:** a retry of a request that completed still returns its stored answer once the key has reached its spending limit or your balance is spent. A new request is refused.

`x-client-request-id`, `session_id` and `x-session-id` never act as a key. Two requests with the same prompt and no `Idempotency-Key` are two requests.

## When the answer is not stored

llm.buzz keeps the answer to a keyed request for up to 24 hours, or less when it needs the room. This copy is not a prompt log: it is kept whether prompt logging is on or off, and a zero-retention demand does not remove it. You cannot read it on the console. A Responses turn kept for `previous_response_id` follows the same limit.

llm.buzz keeps a stored answer on the server that answered the first call, and a retry can reach another server. When the key belongs to a request llm.buzz accepted and no answer is stored where the retry arrived, the retry answers 409 `idempotency_replay_unavailable`. It does not run the request again, and it is not billed. This happens when:

- the retry reached a different server than the first call did;
- the answer is older than 24 hours, was dropped to make room, or was lost when llm.buzz restarted;
- the first call ended in an error after llm.buzz had accepted it. There is no answer to store, and the key cannot run the request again. A 502, a 503 with no live provider and a provider's own 400 all end a key this way.

### What to send next

| What you got | What to send |
|---|---|
| A 2xx answer | Nothing. Read the answer. |
| No answer: a timeout, or a connection that closed | The same request with the same key. You get the stored answer, or you join the request that is still running. |
| 409 `idempotency_replay_unavailable` after no answer | The same key again after a short backoff, a few times: a retry that arrives where the first call was served gets the stored answer. If it keeps answering 409, look up the first request on the [Logs](https://app.llm.buzz/console/logs) page. A new key runs and bills the request again, so send one only when a duplicate is acceptable. |
| An error answer worth retrying: 429, 500, 502, 503 or 504 | The request with a new key, after `Retry-After` or a backoff. A new key is always safe; after most of these errors the old key answers 409. |
| A 429 `unavailable_route` with `x-gateway-admission-refused: true` | The same request with the same key, after `Retry-After` (5 seconds). The header means llm.buzz refused the request at capacity before it called any provider: the request ended without effects and cost nothing, so its key can run it once. Only this 429 carries the header. A 429 without it may have reached a provider, so send a new key. |
| A 400, 401, 402, 403 or 404 | Fix the cause, then send the request with a new key. The same key still works after a request that did nothing: a field error (`invalid_parameter`, `unsupported_parameter`, `invalid_json`), a 401, a 403 `key_limit_exceeded` or `management_key_not_allowed`, a 404 `model_not_found`, a 402 `insufficient_credits`, or a 402 `weight_exceeds_budget` or `in_flight_budget_exhausted` refused before the first provider call. After `context_length_exceeded`, `no_allowed_providers` or a 402 `insufficient_quota`, the same key answers 409. |
| 409 `idempotency_conflict` | A new key. This key already belongs to a different request. |

The Python example above follows this table: it reuses the key when it got no answer or a 409, and takes a new key after an error answer, which is also safe after the capacity 429.

# Errors

Every error on the model API comes in one envelope: the HTTP status, a message that says what to do, and a code your program can branch on.

## The envelope

```json
{"error": {"code": 404, "message": "The requested model does not exist or is not granted to this key. GET /api/v1/models lists the model aliases available to this key.", "metadata": {"error_type": "not_found", "gateway_code": "model_not_found", "param": "model"}}}
```

- `code` is the HTTP status.
- `metadata.gateway_code` is llm.buzz's own code, the one the tables below list.
- `metadata.error_type` is OpenRouter's error type, for code written against OpenRouter.
- `metadata.param` names the request field at fault, when there is one.

Two surfaces keep their protocol's own format. The Messages API (`/api/v1/messages`) answers Anthropic's error body, `{"type": "error", "error": {"type": "...", "message": "..."}}`, where a 402 is a `billing_error` and a 403 a `permission_error`. The Responses WebSocket sends OpenAI's error frames.

A failure after a stream has started arrives as one last chunk with an `error` object and `finish_reason: "error"`, followed by `[DONE]`.

## Dropped fields are not errors

A field the API does not know is dropped, and so is a sampling setting a model refuses, such as `temperature` on a model that takes none: the request still answers. Chat Completions, Responses and Messages name each one in `x-sgr-ignored-parameters` in the response body, as `temperature->dropped(unsupported_by_provider)`. Check that list when a setting seems to have no effect. A field llm.buzz deliberately refuses is not dropped: it is a 400 before any provider is called, `unsupported_parameter` for a refused field or `invalid_parameter` for a refused value (Anthropic's `invalid_request_error` on Messages), listed [below](#the-request).

## Retries

Retry a 429, 500, 502, 503 or 504 with backoff, and wait `Retry-After` seconds first when the answer carries it. Do not resend any other 4xx unchanged: fix the request, the key or the balance first. The exceptions are 402 `in_flight_budget_exhausted`, which you retry after `Retry-After`, and 409 `idempotency_replay_unavailable`, below.

On Chat Completions and Responses, send an `Idempotency-Key` header. A retry with the same key and the same body never runs the request twice: it returns the first answer, or 409 `idempotency_replay_unavailable` when that answer is not stored where the retry arrived.

That 409 means the first request ran or is still running, so retry with the same key after a short backoff: the retry may reach the stored answer. The exception is a first call that ended in an error answer: its key answers 409 for good, so send a new key. A retry without the key, or with a new one, runs the request again and is billed again, so send one only when a duplicate is acceptable. [What to send next](retries-and-idempotency.md#what-to-send-next) tells the cases apart.

## The request

| Status | Code | What to do |
|---|---|---|
| 400 | `invalid_parameter` | A field fails validation, such as a missing `messages` or `input`, an out-of-range `temperature`, or `n` other than 1; `metadata.param` names it, or `body` when the fault spans fields. Fix the field. |
| 400 | `invalid_request` | llm.buzz read the body and the request was refused: a provider's own 400, whose sentence follows `provider rejected the request:`, or another refusal the message explains. Fix the request. |
| 400 | `unsupported_parameter` | You sent a field llm.buzz refuses, such as `seed` or `logit_bias` on Chat Completions or `background` on Responses; `metadata.param` names it. Remove it. [Chat Completions](chat-completions.md#request-fields), [Responses](responses.md#request-fields) and [Messages](messages.md#request-fields) list the fields each API refuses; Messages answers Anthropic's `invalid_request_error`, with `(param: NAME)` at the end of the message. |
| 400 | `unsupported_capability` | The model or its providers cannot do what the request asks, or, on Chat Completions and Responses, `service_tier` is `flex`, `priority` or `fast`: no model on llm.buzz has a price for a flex or priority tier today, and `metadata.param` names `service_tier`. Remove that part, or choose another model. |
| 400 | `previous_response_not_found` | Responses: `previous_response_id` names a turn llm.buzz does not have: expired, sent with `store: false`, lost, or made by another model. Resend the whole conversation in `input`. |
| 400 | `context_length_exceeded` | The prompt fits no provider's context window. Shorten it. |
| 400 | `refusal` | The model refused the request; `metadata.refusal_reason` names the category. |
| 400 | `content_filter` | A content filter stopped the request. |
| 409 | `idempotency_conflict` | An `Idempotency-Key` was reused with a different body. Use a new key for each distinct request. |
| 409 | `idempotency_replay_unavailable` | A request with this `Idempotency-Key` was already accepted, and its answer is not stored where this retry arrived, so it was not run again. The first call may still be running, or its answer may exist: retry with the same key after a short backoff first. A new key runs the request again and bills it again. If the first call ended in an error answer, the key answers 409 for good: send a new key. |
| 413 | `request_too_large` | The request body is over the size limit, such as an audio file over 25 MiB. Send less. |
| 426 | `upgrade_required` | A `GET` to `/responses` that is not a WebSocket upgrade. Open a WebSocket, or `POST` the request. |

An empty answer is not an error. When a provider bills tokens but sends no text, tool call or refusal, llm.buzz tries the next provider, and that empty attempt costs nothing. When every provider answers empty, the request answers 200 with the header `x-gateway-warning: empty_completion`, and the last attempt is billed the usage its provider reported, or an estimate of its prompt when it reported none: [When a provider reports no usage](usage-and-limits.md#when-a-provider-reports-no-usage).

Two prompt caching refusals are 400 `invalid_parameter`: a Claude cache layout that every provider of the model refuses (more than 4 breakpoints, or a one-hour breakpoint after a five-minute one), with `param: cache_control` (Anthropic's `invalid_request_error` on Messages), and, on Chat Completions and Responses, `prompt_cache_retention: "24h"` on a request that demands zero retention, with `param: prompt_cache_retention`. [Prompt caching](prompt-caching.md) explains both.

## The key and the balance

| Status | Code | What to do |
|---|---|---|
| 401 | `invalid_key` | Check the key, and that it is not disabled or revoked. |
| 402 | `insufficient_credits` | Your balance is spent and the model is not free. Add credit, or turn on auto top-up. Free models still answer. |
| 402 | `weight_exceeds_budget` | The request could cost more than your balance. Add credit, or lower `max_tokens`. |
| 402 | `in_flight_budget_exhausted` | Requests already running hold what your balance leaves. Retry after `Retry-After` seconds, or add credit. |
| 402 | `insufficient_quota` | Your organization's monthly budget is spent. The answer carries `Retry-After`, and the message names when the month resets. Wait for the reset, or ask an owner or admin of your organization to raise the budget on the console's Keys page. |
| 403 | `key_limit_exceeded` | The key's spend in its period has reached its limit. Wait for `metadata.resets_at`, or raise the limit. |
| 403 | `management_key_not_allowed` | A management key called a model route. Use an inference key. |

[Authentication](authentication.md) explains keys and their limits.

## The key routes

The key routes, `/api/v1/key`, `/api/v1/credits` and `/api/v1/keys`, answer these besides `invalid_key`, and every other route refuses a management key:

| Status | Code | What to do |
|---|---|---|
| 400 | `invalid_json` | The body is not JSON. Send a JSON object. |
| 400 | `invalid_parameter` | A field is invalid; `metadata.param` names it. |
| 400 | `invalid_request` | A `PATCH` sets none of `name`, `disabled`, `limit` and `limit_reset`. Send at least one. |
| 401 | `missing_authentication` | The request has no `Authorization` header. Send the key as a bearer token. |
| 403 | `management_key_required` | An inference key called `/api/v1/credits` or `/api/v1/keys`. Use a management key. |
| 403 | `management_key_not_allowed` | A management key called a route other than the key routes, such as a model route or a model listing. Use an inference key. |
| 404 | `key_not_found` | The hash names none of your inference keys. List them with `GET /api/v1/keys`. |

## The model and its providers

| Status | Code | What to do |
|---|---|---|
| 404 | `model_not_found` | No model has this name, or your key may not call it. Check the name against `GET /api/v1/models`. |
| 404 | `model_unavailable` | No provider of the model can serve it right now. Retry later, or choose another model. |
| 404 | `no_allowed_providers` | Your `provider.only`, `ignore` or `order` leave no provider. The message names the providers that serve the model. |
| 404 | `model_not_available_in_region` | Your organization's requests are kept in the EU, and the model has no EU provider. Choose a model with an EU provider. On Messages the 404 is Anthropic's `not_found_error`. |
| 404 | `model_not_available_under_data_policy` | No provider of the model meets `provider.zdr`, `provider.data_collection` or your organization's data policy. Remove the demand, choose another model, or review the policy on the console's Prompt logs page. |

[Models and routing](models-and-routing.md) explains provider preferences and data policy.

A path that is not a route llm.buzz serves answers 404 `unknown_route`: check it against the [Endpoints](api-reference.md#endpoints).

## Retry later

| Status | Code | What to do |
|---|---|---|
| 429 | `unavailable_route` | Providers are throttling, or the model is at capacity. Wait `Retry-After` seconds, then retry. |
| 429 | `gateway_overloaded` | Too many requests are waiting for replay. Retry with backoff. |
| 429 | `free_allowance_exceeded` | A free offer's allowance is used: your organization's tokens, or your key's requests per minute. Choose a paid model, or for the per-minute limit retry after `Retry-After` seconds. |
| 502 | `provider_output_too_large` | The provider's answer was over llm.buzz's response limit. Ask for less output, for example with a lower `max_tokens`. |
| 502 | `all_routes_failed` | Every provider tried failed. Retry with backoff. |
| 503 | `unavailable_route` | No provider could answer just now. Retry with backoff. |
| 503 | `continuation_store_unavailable` | Responses: llm.buzz cannot reach its store of turns. Retry after `Retry-After` seconds. |
| 503 | `gateway_unavailable` | llm.buzz could not take the request. Retry after `Retry-After` seconds. |
| 503 | `gateway_draining` | The server that took the request is restarting. Retry after `Retry-After` seconds. |
| 504 | `deadline_exceeded` | The request passed its 600-second limit. A stream cut at the limit ends with an error chunk instead, and is billed for the prompt and the output you received. Retry, or ask for a shorter answer. |

# API reference

The llm.buzz model API speaks the OpenAI, Anthropic and OpenRouter request and response formats under `https://api.llm.buzz/api/v1`. This page covers what every route shares: the base URL, authentication, the `provider` object, request fields and errors. Each endpoint has its own page, with its fields and examples.

## Base URL and SDKs

Every model route is under `https://api.llm.buzz/api/v1`. Point an SDK at it and change nothing else:

| SDK | Setting |
|---|---|
| OpenAI | `base_url="https://api.llm.buzz/api/v1"` |
| OpenRouter | `server_url="https://api.llm.buzz/api/v1"` (Python), `serverURL: "https://api.llm.buzz/api/v1"` (TypeScript) |
| Anthropic | `base_url="https://api.llm.buzz/api"`, without `/v1`: the SDK adds `/v1/messages` itself |

## Authentication

Send an inference key as a bearer token:

```bash
curl https://api.llm.buzz/api/v1/models \
  -H "Authorization: Bearer $LLM_BUZZ_KEY"
```

The Messages API also takes the key in `x-api-key`, the header the Anthropic SDK sends. A management key calls only the key routes below: create one on the console's [Keys](https://app.llm.buzz/console/keys) page, as you create an inference key, and send it the same way. The samples for those routes read it from `$LLM_BUZZ_MANAGEMENT_KEY`. [Authentication](authentication.md) covers keys, expiry and spending limits.

## Endpoints

| Method and path | What it does |
|---|---|
| `POST /chat/completions` | Chat Completions. Streams with `stream: true`. |
| `POST /responses` | Responses. The same path also takes a WebSocket upgrade on `GET`. |
| `POST /messages` | The Anthropic Messages API. |
| `POST /messages/count_tokens` | An estimate of a Messages request's input tokens, `{"input_tokens": N}`. Costs nothing. |
| `POST /embeddings` | Embeddings, for embeddings models. Never streamed. [Embeddings](embeddings.md) covers it. |
| `POST /images` | OpenRouter's Image API: images from a text prompt, each as base64. Never streamed. [Images](images.md) covers it. |
| `POST /images/generations` | The OpenAI Images API (generations), serving the same image models. Never streamed. |
| `POST /audio/transcriptions` | Speech to text: one audio file in, its text out. Never streamed. [Audio](audio.md) covers both audio routes. |
| `POST /audio/speech` | Text to speech: one text in, the whole audio out as MP3 or PCM. |
| `GET /generation?id=` | What a finished request cost, [below](#cost-of-a-finished-request). |
| `GET /models` | The models your key can call, in OpenRouter's shape. Without a key, the public models. |
| `GET /models/{author}/{slug}` | One model. A model whose name has no `/` is `GET /models/{name}`. |
| `GET /model/{author}/{slug}` | The same model, wrapped as `{"data": ...}`. |
| `GET /models/{author}/{slug}/endpoints` | One model's providers, each with its prices, context length, `tag` and recent uptime, latency and throughput. `GET /models/{name}/endpoints` for a name with no `/`. |
| `GET /embeddings/models` | The embeddings models, as `GET /models?output_modalities=embeddings` lists them. Without a key, the public ones. |
| `GET /images/models` | The image models your key can call, each with the sizes and values its fields take. Key required. |
| `GET /images/models/{author}/{slug}/endpoints` | One image model's providers, each with the values its fields take and its price per token or per image. `GET /images/models/{name}/endpoints` for a name with no `/`. Key required. |
| `GET /key` | The key you send: its kind, its limit, what is left and its spend. |
| `GET /credits` | Your organization's prepaid credit and usage. Management key. |
| `GET, POST /keys`; `GET, PATCH, DELETE /keys/{hash}` | List, create, change and revoke inference keys. Management key. |

Paths are relative to `https://api.llm.buzz/api/v1`. Model routes outside it, such as `/v1/chat/completions` on the bare host, answer 404.

Decisions are served on `POST /systemone` and on `POST https://api.llm.buzz/api/alpha/decisions`, and no listed model answers them today. [Choosing an API](choosing-an-api.md) says which API each kind of model takes, and [Decisions](decisions.md) covers the two routes.

:::note
llm.buzz does not enable batch jobs: with a key, `/batches` and `/files` answer 404, "batches are not enabled on this gateway". Send a large job as single requests; the [batch jobs guide](/docs/guides/batch-jobs) shows how.
:::

The keyless public catalog under `/public/v1` (the site facts, every public model and provider) is what the llm.buzz website is built from, and the [API reference](/docs/api/public-models) lists it too. It needs no key, but it is the website's data, not a stable API: its fields can change without notice. Build on `GET /models` and the endpoints route instead.

## The model list

`GET /models` returns `data`, `total_count` and `links`. Prices are decimal strings in US dollars per token. An audio model's `prompt` price is per second of audio or per character of input, and an image model lists its output price in `image_output` ([Images](images.md) explains image prices). Its query parameters:

| Parameter | Effect |
|---|---|
| `output_modalities` | `text` by default; `all` lists every model, and a comma list such as `text,embeddings` lists models that produce any of them. The values are `text`, `image`, `embeddings`, `transcription` (speech to text) and `speech` (text to speech). |
| `limit`, `offset` | Page the list. |
| `region=eu` | Only models with an EU provider, priced over those providers. It changes the listing, never how a call routes. |

A model whose lead provider's listing names its reasoning efforts carries them as `reasoning_efforts`, in order from `none` to `max`. The key is left out otherwise, including for a model whose efforts llm.buzz knows only from its own rules for the model's family, so a model without it can still reason: [Reasoning](reasoning.md#which-models-reason). A speech-to-text model carries `verbose_json`, and a text-to-speech model `supported_voices`, `default_voice` and `speech_speed`; [Audio](audio.md) explains them.

Without an `Authorization` header the list, a model and its endpoints answer the public catalog, at most 1,000 models, priced at each model's first-ranked public provider.

On the endpoints route, llm.buzz's own provider is one endpoint, `provider_name` `llm.buzz` and `tag` `llm-buzz`, priced at the lowest live direct price of each component. [The llm.buzz provider](models-and-routing.md#the-llmbuzz-provider) explains it.

## The provider object

Send `provider` in the request body (with the OpenAI SDK, in `extra_body`) to steer one request. Chat Completions, Responses, Messages, Embeddings and the image routes read every field; the audio routes read only `zdr` and `data_collection`. Every field accepts `null`.

| Field | Type | Meaning |
|---|---|---|
| `order` | list of strings | Try these providers first, in this order, then the others. |
| `only` | list of strings | Use only these providers. |
| `ignore` | list of strings | Never use these providers. |
| `allow_fallbacks` | boolean | `false` stops moving to other providers: without `order` only the first provider is tried, with `order` only the listed ones. |
| `sort` | string | `price` tries the cheapest provider first, `throughput` and `latency` the fastest, `smart` weighs price and speed. Ignored when `order` is set. |
| `zdr` | boolean | `true` keeps the request on providers that declare 0 days of retention. |
| `data_collection` | string | `deny` keeps the request on providers that declare no training and 0 days of retention. `allow` demands nothing. |

`order`, `only` and `ignore` take at most 64 names. A name matches a provider's display name, its tag, or the tag's first part, ignoring case; an EU provider's tag ends in `/eu`. `llm.buzz` or `llm-buzz` names llm.buzz's own provider, and `llm-buzz/eu` only its capacity in the EU. [Models and routing](models-and-routing.md) explains the order a model's providers are tried in.

```python
extra_body={"provider": {"only": ["scaleway/eu"], "zdr": True}}
```

## Model suffixes

A suffix on the model's name changes one request: `:floor` tries the cheapest provider first, `:nitro` the fastest, and `:exacto` is accepted and ignored. `:online` asks for a web search on Chat Completions and Responses; on Messages it does nothing and is named in `x-sgr-ignored-parameters` as `model:online`.

llm.buzz runs no web search today. A request that asks for one, with `:online`, `web_search_options` or `plugins: [{"id": "web"}]`, is still served, without the search: nothing is sent to a search service, no search is billed, and `x-sgr-ignored-parameters` names it as `web_search->dropped(search_unavailable)`. For an organization whose requests llm.buzz keeps in the EU, such a request answers 400 `web_search_unavailable_in_region` instead.

## Request fields

Each API's page lists every field and what llm.buzz does with it: [Chat Completions](chat-completions.md), [Responses](responses.md) and [Messages](messages.md). Fields the surface does not know are dropped instead of refused, and Chat Completions, Responses and Messages name them in `x-sgr-ignored-parameters` in the response body. A field llm.buzz refuses answers 400 before any provider is called: [Errors](errors.md#the-request). These fields do something on llm.buzz:

| Field | Surfaces | Meaning |
|---|---|---|
| `tools` | Chat Completions, Responses, Messages | Functions the model can call, in each API's own shape. |
| `response_format` | Chat Completions | `json_schema` asks for an answer in a JSON schema; `json_object` for any JSON object. |
| `reasoning` | Chat Completions, Messages | `effort` or `max_tokens`, plus `enabled` and `exclude`. Each provider receives it in its own terms. Where a model's efforts are known, an effort it does not take becomes the nearest one it does, named in `x-sgr-ignored-parameters`; a provider that refuses the setting answers 400 naming the field. `exclude: true` keeps the reasoning text out of the answer while the model still reasons. Responses takes `reasoning.effort`, and Chat Completions also takes OpenRouter's `include_reasoning`: `true` is `reasoning: {}`, `false` is `reasoning: {exclude: true}`. |
| `cache_control` | Chat Completions, Responses | Anthropic's prompt-cache marker, `{"type": "ephemeral"}` with an optional `ttl` of `"5m"` or `"1h"`. llm.buzz passes it on and never adds one. Messages takes Anthropic's own markers. [Prompt caching](prompt-caching.md) says when to send it. |
| `prompt_cache_breakpoint`, `prompt_cache_options`, `prompt_cache_retention` | Chat Completions, Responses | OpenAI's explicit caching controls, for OpenAI's GPT-5.6 and later models. [Prompt caching](prompt-caching.md#openais-explicit-caching) explains them. |
| `session_id` | Chat Completions, Responses, Messages | A name for the conversation, 1 to 256 characters, so its turns stay on the provider that holds its cache. The `x-session-id` header works too. |
| `web_search_options`, `plugins: [{"id": "web"}]` | Chat Completions, Responses | Ask for a web search, as `:online` does. llm.buzz runs none today: see [Model suffixes](#model-suffixes). |
| `provider` | Every model route | The provider object above. |
| `gateway` | Chat Completions, Responses, Messages | Per-request retries and fallbacks, below. |
| `defer_loading` | Chat Completions, Responses, Messages | On a tool, keeps it out of the prompt until a tool search finds it, below. |

On Chat Completions and Responses, send an `Idempotency-Key` header. A retry with the same key and the same body never runs the request twice: it returns the first answer, or 409 `idempotency_replay_unavailable` when that answer is not stored where the retry arrived. Send the same key again only when the first call got no answer, or after an answer carrying `x-gateway-admission-refused: true`; after any other error answer send a new key. A retry without the key, or with a new one, runs the request again and is billed again. [Retries and idempotency](retries-and-idempotency.md) says which key to send after each answer, and [Errors](errors.md#retries) lists the codes.

### Sampling settings a model does not take

Some models refuse sampling settings. The Claude models that only think adaptively (Opus 4.7 onward, Sonnet 5, Haiku 5, Fable 5, Mythos 5) take no `temperature`, `top_p` or `top_k`, and OpenAI's reasoning models take `temperature` and `top_p` only where OpenAI does: some never, others only at reasoning effort `none`. llm.buzz drops such a value instead of failing the request, and names it in `x-sgr-ignored-parameters`, as `temperature->dropped(unsupported_by_provider)`, or `temperature->dropped(set_reasoning_effort_none)` where setting the effort to `none` keeps it.

Two more rules follow the model or the provider:

- **Claude models that always think.** Claude Opus 5.5, Sonnet 5.5 and the Fable models take no `none` effort: `reasoning_effort: none` and `thinking.type: disabled` answer 400 before any provider is called, whichever provider would serve the model. Leave thinking out, or choose an effort: [How an effort is mapped](reasoning.md#how-an-effort-is-mapped). Claude Opus 5.5, Sonnet 5.5 and Fable 5.1 also cannot be forced to call a tool: a `tool_choice` that forces one is served as `auto` and named as `tool_choice->auto`, so the answer may come without a tool call.
- **Moonshot AI's own API**, one provider of the Kimi models such as `moonshotai/kimi-k3`, takes no `temperature`, `top_p`, `top_k`, `frequency_penalty` or `presence_penalty`: llm.buzz drops them there, names them in `x-sgr-ignored-parameters`, and the model answers at its fixed settings. A `tool_choice` that names one function goes to another provider of the model that takes it, or is served as `auto` and named as `tool_choice->auto`. To force a tool call there, send `tool_choice: "required"`. The Kimi models' other providers keep these settings.

## Retries and fallbacks per request

The `gateway` object sets bounds for one request. Every field is optional.

```python
extra_body={"gateway": {
    "retry": {"max_attempts_per_route": 2, "max_total_attempts": 3,
              "backoff": {"type": "exponential", "base_delay_ms": 500, "max_delay_ms": 8000}},
    "routing": {"allow_fallbacks": False},
}}
```

| Field | Meaning |
|---|---|
| `retry.max_attempts_per_route` | 1 to 4 calls to one provider, the first included. A cap, not extra retries: a provider that failed is not called again for an ordinary failure. |
| `retry.max_total_attempts` | 1 to 8 calls in all, every fallback included. 8 when omitted. |
| `retry.backoff` | `{"type": "none"}`, or `{"type": "exponential"}` with `base_delay_ms` (default 500), `max_delay_ms` (default 8000) and `multiplier` (default 2): the wait before calling a throttled provider again. |
| `routing.allow_fallbacks` | `false` keeps the request on the first provider, as `provider.allow_fallbacks: false` does. |

These are ceilings: the request's time limit, your spending limits and a provider's health can stop it sooner. Once output has reached you, the request stays on the provider that sent it.

## Tool search

With many tools, mark most of them `defer_loading: true` and declare a tool search tool. The model then sees only the other tools and a search; the tools a search finds join the request. Where every provider of the model searches natively, the provider does it; otherwise llm.buzz runs the search, up to 3 rounds per request.

| Surface | Declare | Mark |
|---|---|---|
| Messages | `{"type": "tool_search_tool_bm25"}` or `{"type": "tool_search_tool_regex"}` | `defer_loading: true` on the tool |
| Responses | `{"type": "tool_search"}` | `defer_loading: true` on the function tool |
| Chat Completions | `{"type": "openrouter:tool_search"}` | `defer_loading: true` beside `function` |

Usage counts the searches in `server_tool_use_details.tool_search_requests` (`server_tool_use.tool_search_requests` on Messages).

## Responses

- `usage.cost` is the request's cost in US dollars, on every surface, when every part of it is known.
- Chat Completions and Embeddings bodies, and every Chat stream chunk, carry `provider`: the name of the provider that answered, `llm.buzz` for llm.buzz's own provider.
- Cached input shows in `usage.prompt_tokens_details.cached_tokens` on Chat Completions and `usage.input_tokens_details.cached_tokens` on Responses, and cache writes in `cache_write_tokens` beside it, with `cache_write_1h_tokens` for the one-hour part when the provider reports it. Messages keeps Anthropic's fields, `cache_read_input_tokens` and `cache_creation_input_tokens`. [Prompt caching](prompt-caching.md#read-the-cache-in-usage) lists them.
- Every answer carries the headers `x-request-id`, `x-gateway-alias` (the model that served), `x-gateway-canonical-model`, `x-gateway-provider` (the provider that answered), `x-gateway-route-depth` and `x-gateway-route-reason`. Image and audio answers add `X-Generation-Id`.
- Readable reasoning comes back beside the answer: `message.reasoning` (`delta.reasoning` in a stream) on Chat Completions, a `thinking` block on Messages, and a reasoning item's `summary_text` on Responses. `reasoning.exclude: true` turns it off.
- A Chat Completions model that makes images returns them in `message.images[].image_url.url` (`delta.images` in a stream), each a complete base64 image. Send one back as an assistant `image_url` content part to continue the conversation.

## Cost of a finished request

`GET /generation?id=` answers the record and cost of any finished request of your organization. A request's generation ID is `gen-` and the 32 hex digits of its `x-request-id`, which is `request-` and the same digits. Image and audio answers also name it in `X-Generation-Id`, and a text-to-speech answer, which has no JSON body, has its cost only here.

```bash
curl "https://api.llm.buzz/api/v1/generation?id=gen-..." \
  -H "Authorization: Bearer $LLM_BUZZ_KEY"
```

The answer is OpenRouter's `{"data": {...}}`. `total_cost` and `usage` are what you were charged, in US dollars at your rate; `api_type` is `completions` (Chat Completions, Responses and Messages), `embeddings`, `image`, `stt`, `tts` or `decisions`; `provider_name` is the provider that served it. An ID of another organization, a malformed one, and one whose request has not finished all answer 404; a request without `id` answers 400. [Audio](audio.md#cost-and-the-generation-record) lists every key.

## Time limit

A request may run for 600 seconds. A stream still generating then ends with a timeout error chunk and the stream terminator, and is billed for the prompt and the output you received, not for its whole reservation. Any other request past the limit answers 504 `deadline_exceeded`. [Usage and limits](usage-and-limits.md#limits) collects the limits of every API.

## Browser access

A web page can call the model API from the browser with its visitor's own key, as on OpenRouter. The key travels in `Authorization` or `x-api-key`, never in a cookie, so every answer under `/api/` allows any origin and none allows credentials.

- A preflight answers 204 without a key, and a browser keeps its answer for a day. It allows the headers the API reads: `authorization`, `content-type`, `accept`, `x-api-key`, `anthropic-version`, `anthropic-beta`, `x-title`, `http-referer`, `idempotency-key`, `x-session-id`, `x-client-request-id`, `originator`, `openai-organization`, `openai-project`, and any `x-sgr-` or `x-stainless-` header (the OpenAI and Anthropic SDKs send the latter). The browser refuses to send any other header.
- Every answer, errors included, lets the page read the headers `x-request-id`, `x-generation-id`, `retry-after`, `x-ratelimit-*` and `x-gateway-*`, and, in the body, the `x-sgr-ignored-parameters` key. No answer sends that name as a header.
- The keyless catalog under `/public/v1` sends no CORS headers.
- A key used from a page is visible to that page's visitors. Give a browser app a key with a spending limit, or let each visitor bring their own.

## App attribution

Coding agents and apps that llm.buzz recognizes get their usage counted under their own name on [Rankings](/rankings) and their app page. Most are recognized by their `User-Agent`. An app can also name itself with OpenRouter's headers:

```bash
-H "X-Title: Cline" -H "HTTP-Referer: https://cline.bot"
```

A title or a site llm.buzz does not know counts as other usage. Neither header is ever published.

## Errors

Errors on the model routes come in one envelope; `metadata.gateway_code` is the code to branch on. The Messages API keeps Anthropic's error body.

```json
{"error": {"code": 404, "message": "...", "metadata": {"error_type": "not_found", "gateway_code": "model_not_found", "param": "model"}}}
```

| Status | Code | Meaning |
|---|---|---|
| 400 | `invalid_parameter` | A request field fails validation, such as a missing `messages` or `input`, an out-of-range `temperature` or `n` other than 1; `metadata.param` names it, or `body` when the fault spans fields. |
| 400 | `invalid_json` | The body is not JSON. |
| 400 | `unsupported_parameter` | A field llm.buzz refuses; `metadata.param` names it. The pages of Chat Completions, Responses and Messages list the fields each refuses. |
| 400 | `invalid_request` | llm.buzz read the body and refused the request: a provider's own 400, whose sentence follows `provider rejected the request:`, or another refusal the message explains. |
| 400 | `unsupported_capability` | The model or its providers cannot do what the request asks, such as a `service_tier` of `flex`, `priority` or `fast` on Chat Completions and Responses. |
| 400 | `context_length_exceeded` | The prompt fits no provider's context window. |
| 400 | `web_search_unavailable_in_region` | The request asked for a web search, and your organization's requests are kept in the EU. |
| 401 | `invalid_key` | No key, or the key is unknown, expired, revoked or disabled. |
| 402 | `insufficient_credits` | Your balance is spent and the model is not free. |
| 403 | `key_limit_exceeded` | The key's spend in its period has reached its limit. |
| 403 | `management_key_not_allowed` | A management key called a model route. |
| 404 | `model_not_found` | No model has this name, or your key may not call it. |
| 404 | `model_unavailable` | No provider of the model can serve it right now. |
| 404 | `no_allowed_providers` | Your `provider` preferences leave no provider. Nothing was sent. |
| 404 | `model_not_available_under_data_policy` | No provider meets `zdr`, `data_collection` or your organization's data policy. Nothing was sent. |
| 409 | `idempotency_conflict` | An `Idempotency-Key` was reused with a different body. |
| 409 | `idempotency_replay_unavailable` | A request with this `Idempotency-Key` was already accepted, and its answer is not stored where this retry arrived, so it was not run again. Retry with the same key after a short backoff first, when the first call got no answer; a new key runs the request again and bills it again, and after an error answer a new key is the only way. |
| 413 | `request_too_large` | The request body is over the size limit, such as an audio file over 25 MiB. |
| 429 | `unavailable_route` | Providers are throttling. Retry after `Retry-After` seconds. |
| 429 | `free_allowance_exceeded` | A free offer's allowance is used: your organization's tokens, or your key's requests per minute. Choose a paid model, or retry after `Retry-After` seconds for the per-minute limit. |
| 502 | `all_routes_failed` | Every provider tried failed. Retry with backoff. |
| 503 | `unavailable_route` | No provider could answer just now. Retry with backoff. |
| 504 | `deadline_exceeded` | The request passed its 600-second limit. |

[Errors](errors.md) lists every code with what to do.

# Chat Completions

`POST https://api.llm.buzz/api/v1/chat/completions`

Send a conversation, get the next message. OpenAI's Chat Completions API: the OpenAI and OpenRouter SDKs call it unchanged. Set `stream: true` for server-sent events, which end with a usage chunk and `data: [DONE]`.

Authentication: an inference key, as `Authorization: Bearer KEY`.

## Parameters

- `Idempotency-Key` (header string): A retry with the same key and the same body never runs the request twice: it returns the first answer, or 409 `idempotency_replay_unavailable` when that answer is not stored where the retry arrived. Send the same key again only when the first call got no answer, or after an answer carrying `x-gateway-admission-refused: true`; after any other error answer send a new key. A retry without the key, or with a new one, runs and bills the request again. Reusing a key with a different body answers 409 `idempotency_conflict`.
- `x-session-id` (header string): Names the conversation, as `session_id` does, so its turns stay on the provider that holds its cache.

## Body

- `model` (string, required): A model's name from `GET /api/v1/models`, such as `deepseek/deepseek-v4.1-flash`. A suffix changes one request: `:floor` tries the cheapest provider first and `:nitro` the fastest. On Chat Completions and Responses `:online` asks for a web search, which llm.buzz does not run today: the request is served without one and `x-sgr-ignored-parameters` names `web_search->dropped(search_unavailable)`, or for an organization whose requests llm.buzz keeps in the EU it answers 400 `web_search_unavailable_in_region`. On Messages `:online` does nothing and is named as `model:online`.
- `messages` (object[], required): The conversation, as OpenAI's message objects: `system`, `user`, `assistant` and `tool` roles. A user message may carry `image_url` parts for models that read images, and a `text` part may carry `prompt_cache_breakpoint`, OpenAI's explicit cache mark.
- `stream` (boolean): Stream the answer as server-sent events.
- `max_tokens` (integer): The most tokens the answer may use. Also bounds what the request holds of your balance while it runs.
- `temperature` (number): Sampling temperature. A model that takes none drops it and names it in `x-sgr-ignored-parameters`.
- `tools` (object[]): Functions the model can call, in OpenAI's shape. `{"type": "openrouter:tool_search"}` declares a tool search, and a tool with `defer_loading: true` beside `function` waits for it.
- `tool_choice` (string or object): `auto`, `none`, `required`, or an object naming one function.
- `response_format` (object): `{"type": "json_schema", ...}` asks for an answer in a JSON schema; `{"type": "json_object"}` for any JSON object.
- `reasoning` (object): Reasoning controls, translated for each provider.
  - `effort` (string): How hard the model thinks. An effort the model does not take becomes the nearest one it does, named in `x-sgr-ignored-parameters`.
  - `max_tokens` (integer): A reasoning budget in tokens.
  - `enabled` (boolean): Turn reasoning on or off.
  - `exclude` (boolean): Keep the reasoning text out of the answer while the model still reasons.
- `include_reasoning` (boolean): OpenRouter's older switch: `true` is `reasoning: {}`, `false` is `reasoning: {exclude: true}`.
- `provider` (object): Steers one request across the model's providers. With the OpenAI SDK, send it in `extra_body`. Every field accepts `null`. A name matches a provider's display name, its tag or the tag's first part, ignoring case: `llm.buzz` or `llm-buzz` names llm.buzz's own provider, and `llm-buzz/eu` only its capacity in the EU.
  - `order` (string[]): Try these providers first, in this order, then the others. At most 64 names.
  - `only` (string[]): Use only these providers.
  - `ignore` (string[]): Never use these providers.
  - `allow_fallbacks` (boolean): `false` stops moving to other providers.
  - `sort` (string, one of `price`, `throughput`, `latency`, `smart`): The order to try providers in. Ignored when `order` is set.
  - `zdr` (boolean): `true` keeps the request on providers that declare 0 days of retention.
  - `data_collection` (string, one of `allow`, `deny`): `deny` keeps the request on providers that declare no training and 0 days of retention.
- `session_id` (string): A name for the conversation, 1 to 256 characters, so its turns stay on the provider that holds its cache.
- `cache_control` (object): Anthropic's prompt-cache marker, `{"type": "ephemeral"}` with an optional `ttl` of `"5m"` or `"1h"`. llm.buzz passes it on and never adds one. Claude models cache only where a request marks it.
- `prompt_cache_options` (object): OpenAI's explicit caching, for GPT-5.6 and later models: `mode` `implicit` (OpenAI's default) or `explicit` (OpenAI writes the cache only at a `prompt_cache_breakpoint`), and `ttl` `30m`.
- `prompt_cache_retention` (string, one of `in_memory`, `24h`): OpenAI's cache retention. `24h` with `provider.zdr`, `provider.data_collection: "deny"` or your organization's no-retention policy is a 400 `invalid_request`.
- `web_search_options` (object): Asks for a web search, as the `:online` suffix does. llm.buzz runs none today: the request is served without a search, nothing is billed for one, and `x-sgr-ignored-parameters` names it as `web_search->dropped(search_unavailable)`. An organization whose requests llm.buzz keeps in the EU gets 400 `web_search_unavailable_in_region` instead.

## Responses

- 200: The answer. `provider` names the provider that answered, and `usage.cost` is the request's cost in US dollars when every part of it is known.
- 400: A field is missing or malformed (`invalid_request`, with `metadata.param`), or the model cannot do what the request asks (`unsupported_capability`).
- 401: No key, or the key is unknown, expired, revoked or disabled.
- 402: Your balance is spent and the model is not free, or the request could cost more than your balance.
- 404: No model has this name, or your key may not call it.
- 409: The `Idempotency-Key` was used before. `idempotency_replay_unavailable`: the first request was accepted, and its answer is not stored where this retry arrived, so it was not run again; retry with the same key after a short backoff first, when the first call got no answer, and send a new key to run the request again (it is billed again) or after an error answer. `idempotency_conflict`: the key came with a different body; use a new key for each distinct request. Neither runs the request again.
- 429: Providers are throttling, or the model is at capacity (`unavailable_route`): retry after `Retry-After` seconds. Or a free offer's allowance is used (`free_allowance_exceeded`): your organization's tokens, or your key's requests per minute. Choose a paid model, or for the per-minute limit retry after `Retry-After` seconds.

## Example (curl)

```bash
curl -X POST https://api.llm.buzz/api/v1/chat/completions \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "deepseek/deepseek-v4.1-flash",
  "messages": [
    {
      "role": "user",
      "content": "Say hello in one word."
    }
  ]
}'
```

## Example (Python)

```python
import os
from openai import OpenAI

client = OpenAI(base_url="https://api.llm.buzz/api/v1", api_key=os.environ["LLM_BUZZ_KEY"])
reply = client.chat.completions.create(
    model="deepseek/deepseek-v4.1-flash",
    messages=[{"role": "user", "content": "Say hello in one word."}],
)
print(reply.choices[0].message.content)
```

## Example (TypeScript)

```ts
import OpenAI from "openai";

const client = new OpenAI({ baseURL: "https://api.llm.buzz/api/v1", apiKey: process.env.LLM_BUZZ_KEY });
const reply = await client.chat.completions.create({
  model: "deepseek/deepseek-v4.1-flash",
  messages: [{ role: "user", content: "Say hello in one word." }],
});
console.log(reply.choices[0].message.content);
```

## Example response

```json
{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "model": "deepseek/deepseek-v4.1-flash",
  "provider": "Prime Intellect",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello!"
      },
      "finish_reason": "stop",
      "native_finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 12,
    "completion_tokens": 3,
    "total_tokens": 15,
    "cost": 0.0000072
  },
  "system_fingerprint": null
}
```

# Responses WebSocket

`GET https://api.llm.buzz/api/v1/responses`

The same Responses API over a WebSocket, the transport the Codex CLI uses. Send `response.create` frames; each Responses stream event arrives as one text frame, and a request that fails arrives as an `{"type": "error", ...}` frame. The key is checked before the upgrade is accepted. A `GET` without an upgrade answers 426.

Authentication: an inference key, as `Authorization: Bearer KEY`.

## Parameters

- `Upgrade` (header string, required): `websocket`.

## Responses

- 101: The connection switches to the WebSocket.
- 401: No key, or the key is unknown, expired, revoked or disabled.
- 426: A `GET` without a well-formed upgrade. Clients such as Codex fall back to `POST /api/v1/responses`.

# Responses

`POST https://api.llm.buzz/api/v1/responses`

OpenAI's Responses API. Continue a conversation with `previous_response_id`. Set `stream: true` for Responses stream events.

Authentication: an inference key, as `Authorization: Bearer KEY`.

## Parameters

- `Idempotency-Key` (header string): A retry with the same key and the same body never runs the request twice: it returns the first answer, or 409 `idempotency_replay_unavailable` when that answer is not stored where the retry arrived. Send the same key again only when the first call got no answer, or after an answer carrying `x-gateway-admission-refused: true`; after any other error answer send a new key. A retry without the key, or with a new one, runs and bills the request again. Reusing a key with a different body answers 409 `idempotency_conflict`.
- `x-session-id` (header string): Names the conversation, as `session_id` does, so its turns stay on the provider that holds its cache.

## Body

- `model` (string, required): A model's name from `GET /api/v1/models`, such as `deepseek/deepseek-v4.1-flash`. A suffix changes one request: `:floor` tries the cheapest provider first and `:nitro` the fastest. On Chat Completions and Responses `:online` asks for a web search, which llm.buzz does not run today: the request is served without one and `x-sgr-ignored-parameters` names `web_search->dropped(search_unavailable)`, or for an organization whose requests llm.buzz keeps in the EU it answers 400 `web_search_unavailable_in_region`. On Messages `:online` does nothing and is named as `model:online`.
- `input` (string or object[], required): A string, or a list of input items in OpenAI's shape. An `input_text` part may carry `prompt_cache_breakpoint`, OpenAI's explicit cache mark.
- `instructions` (string): A system message for this turn.
- `previous_response_id` (string): Continue from an earlier response.
- `stream` (boolean): Stream Responses events.
- `reasoning` (object): `effort`, translated for each provider.
- `tools` (object[]): Function tools in the Responses shape; `{"type": "tool_search"}` declares a tool search over tools marked `defer_loading: true`.
- `provider` (object): Steers one request across the model's providers. With the OpenAI SDK, send it in `extra_body`. Every field accepts `null`. A name matches a provider's display name, its tag or the tag's first part, ignoring case: `llm.buzz` or `llm-buzz` names llm.buzz's own provider, and `llm-buzz/eu` only its capacity in the EU.
  - `order` (string[]): Try these providers first, in this order, then the others. At most 64 names.
  - `only` (string[]): Use only these providers.
  - `ignore` (string[]): Never use these providers.
  - `allow_fallbacks` (boolean): `false` stops moving to other providers.
  - `sort` (string, one of `price`, `throughput`, `latency`, `smart`): The order to try providers in. Ignored when `order` is set.
  - `zdr` (boolean): `true` keeps the request on providers that declare 0 days of retention.
  - `data_collection` (string, one of `allow`, `deny`): `deny` keeps the request on providers that declare no training and 0 days of retention.
- `session_id` (string): A name for the conversation, 1 to 256 characters, so its turns stay on the provider that holds its cache.
- `cache_control` (object): Anthropic's prompt-cache marker, `{"type": "ephemeral"}` with an optional `ttl` of `"5m"` or `"1h"`. llm.buzz passes it on and never adds one. Claude models cache only where a request marks it.
- `prompt_cache_options` (object): OpenAI's explicit caching, for GPT-5.6 and later models: `mode` `implicit` (OpenAI's default) or `explicit` (OpenAI writes the cache only at a `prompt_cache_breakpoint`), and `ttl` `30m`.
- `prompt_cache_retention` (string, one of `in_memory`, `24h`): OpenAI's cache retention. `24h` with `provider.zdr`, `provider.data_collection: "deny"` or your organization's no-retention policy is a 400 `invalid_request`.

## Responses

- 200: A response object. Cache reads show in `usage.input_tokens_details.cached_tokens` and writes in its `cache_write_tokens`; readable reasoning arrives as a reasoning item's `summary_text`.
- 400: A field is missing or malformed (`invalid_request`, with `metadata.param`), or the model cannot do what the request asks (`unsupported_capability`).
- 401: No key, or the key is unknown, expired, revoked or disabled.
- 402: Your balance is spent and the model is not free, or the request could cost more than your balance.
- 404: No model has this name, or your key may not call it.
- 409: The `Idempotency-Key` was used before. `idempotency_replay_unavailable`: the first request was accepted, and its answer is not stored where this retry arrived, so it was not run again; retry with the same key after a short backoff first, when the first call got no answer, and send a new key to run the request again (it is billed again) or after an error answer. `idempotency_conflict`: the key came with a different body; use a new key for each distinct request. Neither runs the request again.
- 429: Providers are throttling, or the model is at capacity (`unavailable_route`): retry after `Retry-After` seconds. Or a free offer's allowance is used (`free_allowance_exceeded`): your organization's tokens, or your key's requests per minute. Choose a paid model, or for the per-minute limit retry after `Retry-After` seconds.

## Example (curl)

```bash
curl -X POST https://api.llm.buzz/api/v1/responses \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "deepseek/deepseek-v4.1-flash",
  "input": "Say hello in one word."
}'
```

## Example (Python)

```python
import os
from openai import OpenAI

client = OpenAI(base_url="https://api.llm.buzz/api/v1", api_key=os.environ["LLM_BUZZ_KEY"])
response = client.responses.create(model="deepseek/deepseek-v4.1-flash", input="Say hello in one word.")
print(response.output_text)
```

## Example response

```json
{
  "id": "resp_...",
  "object": "response",
  "model": "deepseek/deepseek-v4.1-flash",
  "status": "completed",
  "output": [
    {
      "type": "message",
      "role": "assistant",
      "content": [
        {
          "type": "output_text",
          "text": "Hello!"
        }
      ]
    }
  ],
  "usage": {
    "input_tokens": 12,
    "output_tokens": 3,
    "total_tokens": 15,
    "input_tokens_details": {
      "cached_tokens": 0,
      "cache_write_tokens": 0
    },
    "cost": 0.0000072
  }
}
```

# Messages

`POST https://api.llm.buzz/api/v1/messages`

Anthropic's Messages API, for the Anthropic SDK and the tools built on it. The key may also come in `x-api-key`. Errors keep Anthropic's error body.

Authentication: an inference key, as `Authorization: Bearer KEY` or in `x-api-key`.

## Parameters

- `x-session-id` (header string): Names the conversation, as `session_id` does, so its turns stay on the provider that holds its cache.

## Body

- `model` (string, required): A model's name from `GET /api/v1/models`, such as `deepseek/deepseek-v4.1-flash`. A suffix changes one request: `:floor` tries the cheapest provider first and `:nitro` the fastest. On Chat Completions and Responses `:online` asks for a web search, which llm.buzz does not run today: the request is served without one and `x-sgr-ignored-parameters` names `web_search->dropped(search_unavailable)`, or for an organization whose requests llm.buzz keeps in the EU it answers 400 `web_search_unavailable_in_region`. On Messages `:online` does nothing and is named as `model:online`.
- `max_tokens` (integer, required): The most tokens the answer may use.
- `messages` (object[], required): The conversation, as Anthropic's message objects.
- `system` (string or object[]): The system prompt: a string, or a list of text blocks, which may carry `cache_control`.
- `stream` (boolean): Stream Anthropic's server-sent events.
- `tools` (object[]): Tools in Anthropic's shape. `{"type": "tool_search_tool_bm25"}` or `{"type": "tool_search_tool_regex"}` declares a tool search over tools marked `defer_loading: true`.
- `reasoning` (object): Reasoning controls, translated for each provider.
  - `effort` (string): How hard the model thinks. An effort the model does not take becomes the nearest one it does, named in `x-sgr-ignored-parameters`.
  - `max_tokens` (integer): A reasoning budget in tokens.
  - `enabled` (boolean): Turn reasoning on or off.
  - `exclude` (boolean): Keep the reasoning text out of the answer while the model still reasons.
- `provider` (object): Steers one request across the model's providers. With the OpenAI SDK, send it in `extra_body`. Every field accepts `null`. A name matches a provider's display name, its tag or the tag's first part, ignoring case: `llm.buzz` or `llm-buzz` names llm.buzz's own provider, and `llm-buzz/eu` only its capacity in the EU.
  - `order` (string[]): Try these providers first, in this order, then the others. At most 64 names.
  - `only` (string[]): Use only these providers.
  - `ignore` (string[]): Never use these providers.
  - `allow_fallbacks` (boolean): `false` stops moving to other providers.
  - `sort` (string, one of `price`, `throughput`, `latency`, `smart`): The order to try providers in. Ignored when `order` is set.
  - `zdr` (boolean): `true` keeps the request on providers that declare 0 days of retention.
  - `data_collection` (string, one of `allow`, `deny`): `deny` keeps the request on providers that declare no training and 0 days of retention.
- `session_id` (string): A name for the conversation, 1 to 256 characters, so its turns stay on the provider that holds its cache.

## Responses

- 200: Anthropic's message object. `usage` always carries `cache_creation_input_tokens` and `cache_read_input_tokens`, and `cost` in US dollars; readable reasoning arrives as a `thinking` block.
- 400: A field is missing or malformed, or the model cannot do what the request asks, in Anthropic's error body as an `invalid_request_error`; the message names the field.
- 401: Messages keeps Anthropic's error body: a 401 is an `authentication_error`, a 402 a `billing_error`, a 403 a `permission_error` and a 429 a `rate_limit_error`.

## Example (curl)

```bash
curl -X POST https://api.llm.buzz/api/v1/messages \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "anthropic/claude-haiku-5.5",
  "max_tokens": 256,
  "messages": [
    {
      "role": "user",
      "content": "Say hello in one word."
    }
  ]
}'
```

## Example (Python)

```python
import os
from anthropic import Anthropic

client = Anthropic(base_url="https://api.llm.buzz/api", api_key=os.environ["LLM_BUZZ_KEY"])
message = client.messages.create(
    model="anthropic/claude-haiku-5.5",
    max_tokens=256,
    messages=[{"role": "user", "content": "Say hello in one word."}],
)
print(message.content[0].text)
```

## Example (TypeScript)

```ts
import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({ baseURL: "https://api.llm.buzz/api", apiKey: process.env.LLM_BUZZ_KEY });
const message = await client.messages.create({
  model: "anthropic/claude-haiku-5.5",
  max_tokens: 256,
  messages: [{ role: "user", content: "Say hello in one word." }],
});
```

## Example response

```json
{
  "id": "msg_...",
  "type": "message",
  "role": "assistant",
  "model": "anthropic/claude-haiku-5.5",
  "content": [
    {
      "type": "text",
      "text": "Hello!"
    }
  ],
  "stop_reason": "end_turn",
  "usage": {
    "input_tokens": 14,
    "output_tokens": 4,
    "cache_creation_input_tokens": 0,
    "cache_read_input_tokens": 0,
    "cost": 0.0000034
  }
}
```

# Count tokens

`POST https://api.llm.buzz/api/v1/messages/count_tokens`

An estimate of a Messages request's input tokens, from llm.buzz's own tokenizer, never the provider's. It costs nothing and is not refused for a key's spending limit. The body names the estimate in `x-sgr-ignored-parameters` as `input_tokens->estimated(gateway_tokenizer)`.

Authentication: an inference key, as `Authorization: Bearer KEY` or in `x-api-key`.

## Body

- `model` (string, required): A model's name from `GET /api/v1/models`, such as `deepseek/deepseek-v4.1-flash`. A suffix changes one request: `:floor` tries the cheapest provider first and `:nitro` the fastest. On Chat Completions and Responses `:online` asks for a web search, which llm.buzz does not run today: the request is served without one and `x-sgr-ignored-parameters` names `web_search->dropped(search_unavailable)`, or for an organization whose requests llm.buzz keeps in the EU it answers 400 `web_search_unavailable_in_region`. On Messages `:online` does nothing and is named as `model:online`.
- `messages` (object[], required): The conversation, as on Messages.
- `system` (string or object[]): The system prompt: a string, or a list of text blocks, which may carry `cache_control`.
- `tools` (object[]): Tools, as on Messages.

## Responses

- 200: The estimate.
- 401: Messages keeps Anthropic's error body: a 401 is an `authentication_error`, a 402 a `billing_error`, a 403 a `permission_error` and a 429 a `rate_limit_error`.

## Example (curl)

```bash
curl -X POST https://api.llm.buzz/api/v1/messages/count_tokens \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "anthropic/claude-haiku-5.5",
  "messages": [
    {
      "role": "user",
      "content": "Say hello in one word."
    }
  ]
}'
```

## Example response

```json
{
  "input_tokens": 14,
  "x-sgr-ignored-parameters": [
    "input_tokens->estimated(gateway_tokenizer)"
  ]
}
```

# Embeddings

`POST https://api.llm.buzz/api/v1/embeddings`

OpenAI's Embeddings API, for embeddings models: vectors for one or more inputs, never streamed. A call bills the input tokens the provider reports at that provider's input price, with no output. A request without `dimensions` gets vectors of the model's declared dimensions. Send no `stream`, or `false`; unknown fields are dropped. There is no answer store: `Idempotency-Key` is ignored, so a retry is a new, billed call. `GET /api/v1/embeddings/models` lists the embeddings models your key can call.

Authentication: an inference key, as `Authorization: Bearer KEY`.

## Body

- `model` (string, required): An embeddings model from `GET /api/v1/embeddings/models`, such as `qwen/qwen3-embedding-8b`.
- `input` (string or string[] or integer[] or integer[][], required): A string, a list of strings, a list of token IDs, or a list of such lists. Token IDs must match the model's tokenizer.
- `encoding_format` (string, one of `float`, `base64`): How the vectors come back.
- `dimensions` (integer): The vectors' size, sent to the provider as given.
- `provider` (object): Steers one request across the model's providers. With the OpenAI SDK, send it in `extra_body`. Every field accepts `null`. A name matches a provider's display name, its tag or the tag's first part, ignoring case: `llm.buzz` or `llm-buzz` names llm.buzz's own provider, and `llm-buzz/eu` only its capacity in the EU.
  - `order` (string[]): Try these providers first, in this order, then the others. At most 64 names.
  - `only` (string[]): Use only these providers.
  - `ignore` (string[]): Never use these providers.
  - `allow_fallbacks` (boolean): `false` stops moving to other providers.
  - `sort` (string, one of `price`, `throughput`, `latency`, `smart`): The order to try providers in. Ignored when `order` is set.
  - `zdr` (boolean): `true` keeps the request on providers that declare 0 days of retention.
  - `data_collection` (string, one of `allow`, `deny`): `deny` keeps the request on providers that declare no training and 0 days of retention.
- `user` (string): Your end user's label, recorded and never forwarded.

## Responses

- 200: One vector per input, exactly as the provider returned it, with `provider` and `usage`, whose `cost` is in US dollars.
- 400: A field is missing or malformed (`invalid_request`, with `metadata.param`), or the model cannot do what the request asks (`unsupported_capability`).
- 401: No key, or the key is unknown, expired, revoked or disabled.
- 402: Your balance is spent and the model is not free, or the request could cost more than your balance.
- 404: No model has this name, or your key may not call it.
- 429: Providers are throttling, or the model is at capacity (`unavailable_route`): retry after `Retry-After` seconds. Or a free offer's allowance is used (`free_allowance_exceeded`): your organization's tokens, or your key's requests per minute. Choose a paid model, or for the per-minute limit retry after `Retry-After` seconds.

## Example (curl)

```bash
curl -X POST https://api.llm.buzz/api/v1/embeddings \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "qwen/qwen3-embedding-8b",
  "input": "The quick brown fox"
}'
```

## Example response

```json
{
  "object": "list",
  "data": [
    {
      "object": "embedding",
      "index": 0,
      "embedding": [
        0.0123,
        -0.0456
      ]
    }
  ],
  "model": "qwen/qwen3-embedding-8b",
  "provider": "DeepInfra",
  "usage": {
    "prompt_tokens": 4,
    "total_tokens": 4,
    "cost": 4e-8
  }
}
```

# Images

`POST https://api.llm.buzz/api/v1/images`

OpenRouter's Image API: a text prompt in, images out, never streamed. The request resolves to one size the model declares: an explicit `size`; else `resolution` keeps the sizes of that tier and `aspect_ratio` picks the first of them with that ratio; with neither, the model's first size. Only the parameters the model declares reach the provider. A provider may return fewer images than `n`, and you pay for the images returned: a model priced per image bills each image, one priced per token the tokens the provider reports. A high-quality image can take minutes, so allow a long client timeout. `GET /api/v1/images/models` lists the image models your key can call and the values each takes.

Authentication: an inference key, as `Authorization: Bearer KEY`.

## Body

- `model` (string, required): An image model from `GET /api/v1/images/models`, such as `qwen/qwen-image-3`; a chat or embeddings model is a 400 on `model`. The `:floor` and `:nitro` suffixes work as on chat.
- `prompt` (string, required): What to draw, 1 to 32,000 characters.
- `n` (integer): How many images, 1 to 10 and at most the model's most images per request.
- `size` (string): A size the model declares, `WIDTHxHEIGHT` or `auto`, or a resolution tier such as `1K` or `2K`, read as `resolution`.
- `aspect_ratio` (string): A ratio the model declares, such as `16:9`: the first declared size with that ratio.
- `resolution` (string): A tier the model declares, such as `1K` or `2K`.
- `quality` (string): A value the model declares; a model that declares none ignores the field.
- `background` (string): A value the model declares; a model that declares none ignores the field.
- `output_format` (string, one of `png`, `jpeg`, `webp`): A format the model declares; a model that declares none ignores the field.
- `output_compression` (integer): 0 to 100, forwarded only to a model that takes it.
- `seed` (integer): Forwarded only to a model that takes it.
- `provider` (object): Steers one request across the model's providers. With the OpenAI SDK, send it in `extra_body`. Every field accepts `null`. A name matches a provider's display name, its tag or the tag's first part, ignoring case: `llm.buzz` or `llm-buzz` names llm.buzz's own provider, and `llm-buzz/eu` only its capacity in the EU.
  - `order` (string[]): Try these providers first, in this order, then the others. At most 64 names.
  - `only` (string[]): Use only these providers.
  - `ignore` (string[]): Never use these providers.
  - `allow_fallbacks` (boolean): `false` stops moving to other providers.
  - `sort` (string, one of `price`, `throughput`, `latency`, `smart`): The order to try providers in. Ignored when `order` is set.
  - `zdr` (boolean): `true` keeps the request on providers that declare 0 days of retention.
  - `data_collection` (string, one of `allow`, `deny`): `deny` keeps the request on providers that declare no training and 0 days of retention.
- `user` (string): Your end user's label, recorded and never forwarded.

## Responses

- 200: The images, each as `b64_json` with the `media_type` its bytes show (PNG, JPEG or WebP), and `usage`, whose `cost` is in US dollars. The `X-Generation-Id` header names the call's generation record, `GET /api/v1/generation`.
- 400: Before any provider call: a field is missing or malformed, or names a size, ratio, tier or value the model does not declare, or `n` is above its most images. `metadata.param` names the field, and the message what to send. A chat or embeddings model, `input_references`, `stream: true`, `output_format: "svg"` and `response_format: "url"` are refused the same way.
- 401: No key, or the key is unknown, expired, revoked or disabled.
- 402: Your balance is spent and the model is not free, or the request could cost more than your balance.
- 404: No model has this name, or your key may not call it.
- 429: Providers are throttling, or the model is at capacity (`unavailable_route`): retry after `Retry-After` seconds. Or a free offer's allowance is used (`free_allowance_exceeded`): your organization's tokens, or your key's requests per minute. Choose a paid model, or for the per-minute limit retry after `Retry-After` seconds.

## Example (curl)

```bash
curl -X POST https://api.llm.buzz/api/v1/images \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "qwen/qwen-image-3",
  "prompt": "A honeycomb engraved on a copper plate",
  "aspect_ratio": "16:9"
}'
```

## Example response

```json
{
  "created": 1792900000,
  "data": [
    {
      "b64_json": "iVBORw0...",
      "media_type": "image/png"
    }
  ],
  "usage": {
    "prompt_tokens": 0,
    "completion_tokens": 4175,
    "total_tokens": 4175,
    "cost": 0.03
  }
}
```

# Image generation

`POST https://api.llm.buzz/api/v1/images/generations`

OpenAI's Images API, generations only, for the OpenAI SDK: a prompt in, images out as `b64_json`, never streamed. It serves the same models and checks the same declarations as `POST /api/v1/images`: `model` and `prompt` suffice, and the model's first declared size is the default. `style` is accepted and never forwarded, the SDK's `standard` and `hd` qualities match no declared quality, and unknown fields are dropped. `GET /api/v1/images/models` lists the image models your key can call.

Authentication: an inference key, as `Authorization: Bearer KEY`.

## Body

- `model` (string, required): An image model from `GET /api/v1/images/models`, such as `qwen/qwen-image-3`; a chat or embeddings model is a 400 on `model`. The `:floor` and `:nitro` suffixes work as on chat.
- `prompt` (string, required): What to draw, 1 to 32,000 characters.
- `n` (integer): How many images, 1 to 10 and at most the model's most images per request.
- `size` (string): A size the model declares, `WIDTHxHEIGHT` or `auto`.
- `quality` (string): A value the model declares; a model that declares none ignores the field.
- `background` (string): A value the model declares; a model that declares none ignores the field.
- `output_format` (string, one of `png`, `jpeg`, `webp`): A format the model declares; a model that declares none ignores the field.
- `output_compression` (integer): 0 to 100, forwarded only to a model that takes it.
- `moderation` (string): Forwarded only to a model that takes it.
- `response_format` (string, one of `b64_json`): Every image is answered as `b64_json`; `url` is a 400.
- `provider` (object): Steers one request across the model's providers. With the OpenAI SDK, send it in `extra_body`. Every field accepts `null`. A name matches a provider's display name, its tag or the tag's first part, ignoring case: `llm.buzz` or `llm-buzz` names llm.buzz's own provider, and `llm-buzz/eu` only its capacity in the EU.
  - `order` (string[]): Try these providers first, in this order, then the others. At most 64 names.
  - `only` (string[]): Use only these providers.
  - `ignore` (string[]): Never use these providers.
  - `allow_fallbacks` (boolean): `false` stops moving to other providers.
  - `sort` (string, one of `price`, `throughput`, `latency`, `smart`): The order to try providers in. Ignored when `order` is set.
  - `zdr` (boolean): `true` keeps the request on providers that declare 0 days of retention.
  - `data_collection` (string, one of `allow`, `deny`): `deny` keeps the request on providers that declare no training and 0 days of retention.
- `user` (string): Your end user's label, recorded and never forwarded.

## Responses

- 200: The images as `b64_json` (with any `revised_prompt`), the rendering facts the provider echoes (`background`, `output_format`, `quality`, `size`), and `usage` in OpenAI's names, whose `cost` is in US dollars. The `X-Generation-Id` header names the call's generation record, `GET /api/v1/generation`.
- 400: Before any provider call: a field is missing or malformed, or names a size, ratio, tier or value the model does not declare, or `n` is above its most images. `metadata.param` names the field, and the message what to send. A chat or embeddings model, `input_references`, `stream: true`, `output_format: "svg"` and `response_format: "url"` are refused the same way.
- 401: No key, or the key is unknown, expired, revoked or disabled.
- 402: Your balance is spent and the model is not free, or the request could cost more than your balance.
- 404: No model has this name, or your key may not call it.
- 429: Providers are throttling, or the model is at capacity (`unavailable_route`): retry after `Retry-After` seconds. Or a free offer's allowance is used (`free_allowance_exceeded`): your organization's tokens, or your key's requests per minute. Choose a paid model, or for the per-minute limit retry after `Retry-After` seconds.

## Example (curl)

```bash
curl -X POST https://api.llm.buzz/api/v1/images/generations \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "qwen/qwen-image-3",
  "prompt": "A honeycomb engraved on a copper plate"
}'
```

## Example response

```json
{
  "created": 1792900000,
  "data": [
    {
      "b64_json": "iVBORw0..."
    }
  ],
  "usage": {
    "input_tokens": 0,
    "output_tokens": 4175,
    "total_tokens": 4175,
    "input_tokens_details": {
      "image_tokens": 0,
      "text_tokens": 0
    },
    "cost": 0.03
  }
}
```

# Speech to text

`POST https://api.llm.buzz/api/v1/audio/transcriptions`

OpenRouter's speech-to-text API: one audio file in, its text out, never streamed. Send the audio as base64 in `input_audio`, or as an OpenAI multipart upload with the audio in a `file` part and every other field as a text part. The file is at most 25 MiB in `wav`, `mp3`, `flac`, `ogg`, `webm`, `m4a`, `aac` or `pcm`; its duration is measured before any provider is called, and a file that cannot be read is a 400, never billed. A call bills the seconds of audio. `GET /api/v1/models?output_modalities=transcription` lists the models your key can call, each with `verbose_json`.

Authentication: an inference key, as `Authorization: Bearer KEY`.

## Body

- `model` (string, required): A speech-to-text model from `GET /api/v1/models?output_modalities=transcription`, such as `openai/whisper-large-v3-turbo`.
- `input_audio` (object, required): `data` (the audio in base64) and `format`. A `url` is a 400: send the bytes.
- `response_format` (string, one of `json`, `verbose_json`): `json` (the default: `text` and `usage`) or `verbose_json`, which adds `task`, `language`, `duration`, `segments` and `words`, on a model whose `verbose_json` is `true`.
- `language` (string): An ISO-639-1 code such as `en`; omitted, the provider detects it.
- `temperature` (number): 0 to 1.
- `timestamp_granularities` (string[]): Sent with `verbose_json` only.
- `provider` (object): The audio routes read only the data policy fields; `order`, `only`, `ignore`, `allow_fallbacks` and `sort` are accepted and ignored. With the OpenAI SDK, send it in `extra_body`.
  - `zdr` (boolean): `true` keeps the request on providers that declare 0 days of retention.
  - `data_collection` (string, one of `allow`, `deny`): `deny` keeps the request on providers that declare no training and 0 days of retention.

## Responses

- 200: The text, with `usage.seconds` (what the call billed) and `usage.cost` in US dollars. The `X-Generation-Id` header names the call's generation record.
- 400: A field is missing or malformed (`invalid_request`, with `metadata.param`), or the model cannot do what the request asks (`unsupported_capability`).
- 401: No key, or the key is unknown, expired, revoked or disabled.
- 402: Your balance is spent and the model is not free, or the request could cost more than your balance.
- 404: No model has this name, or your key may not call it.
- 413: The file is over 25 MiB (`request_too_large`). Send a shorter file.

## Example (curl)

```bash
curl -X POST https://api.llm.buzz/api/v1/audio/transcriptions \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "openai/whisper-large-v3-turbo",
  "input_audio": {
    "data": "BASE64_AUDIO",
    "format": "mp3"
  }
}'
```

## Example response

```json
{
  "text": "Thanks for calling.",
  "usage": {
    "seconds": 12.48,
    "cost": 0.000041558
  }
}
```

# Text to speech

`POST https://api.llm.buzz/api/v1/audio/speech`

OpenRouter's text-to-speech API: one text in, the whole audio out as MP3 or raw PCM, never streamed. A call bills the input's characters, and settles before the first byte is sent. The answer has no JSON body, so its cost is in the generation record that `X-Generation-Id` names. `GET /api/v1/models?output_modalities=speech` lists the models your key can call, each with its `supported_voices`, `default_voice` and `speech_speed`.

Authentication: an inference key, as `Authorization: Bearer KEY`.

## Body

- `model` (string, required): A text-to-speech model from `GET /api/v1/models?output_modalities=speech`, such as `hexgrad/kokoro-82m`.
- `input` (string, required): 1 to 4,096 characters. An array of turns (several speakers) is a 400.
- `voice` (string): One of the model's `supported_voices`; omitted, its `default_voice`.
- `response_format` (string, one of `pcm`, `mp3`): `pcm` (the default: signed 16-bit little-endian mono) or `mp3`.
- `speed` (number): 0.25 to 4.0, on a model whose `speech_speed` is `true`.
- `provider` (object): The audio routes read only the data policy fields; `order`, `only`, `ignore`, `allow_fallbacks` and `sort` are accepted and ignored. With the OpenAI SDK, send it in `extra_body`.
  - `zdr` (boolean): `true` keeps the request on providers that declare 0 days of retention.
  - `data_collection` (string, one of `allow`, `deny`): `deny` keeps the request on providers that declare no training and 0 days of retention.

## Responses

- 200: The audio: `audio/mpeg`, or `audio/pcm` at the provider's rate, with `Cache-Control: no-store` and the `X-Generation-Id` header.
- 400: A field is missing or malformed (`invalid_request`, with `metadata.param`), or the model cannot do what the request asks (`unsupported_capability`).
- 401: No key, or the key is unknown, expired, revoked or disabled.
- 402: Your balance is spent and the model is not free, or the request could cost more than your balance.
- 404: No model has this name, or your key may not call it.

## Example (curl)

```bash
curl -X POST https://api.llm.buzz/api/v1/audio/speech \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "hexgrad/kokoro-82m",
  "input": "Your order has shipped.",
  "voice": "af_heart",
  "response_format": "mp3"
}'
```

# Get a generation

`GET https://api.llm.buzz/api/v1/generation`

One finished request's record and cost, by its generation ID: the `X-Generation-Id` an image or audio answer carries, or for any request `gen-` and the 32 hex digits of its `x-request-id`, which is `request-` and the same digits. It reads any finished request of your key's organization. `total_cost` and `usage` are what you were charged in US dollars; `api_type` is `completions` (Chat Completions, Responses and Messages), `embeddings`, `image`, `stt` or `tts`. Never cached.

Authentication: an inference key, as `Authorization: Bearer KEY`.

## Parameters

- `id` (query string, required): The generation ID, such as `gen-` and 32 hex digits.

## Responses

- 200: OpenRouter's `GenerationResponse`, as `{"data": ...}`, with `model`, `provider_name`, `latency` and `generation_time` in milliseconds, and `data_region` (`europe` for a provider in the EU, else `global`).
- 400: No `id` (`invalid_parameter` on `id`).
- 401: No key, or the key is unknown, expired, revoked or disabled.
- 404: Another organization's ID, a malformed one, or one whose request has not finished.

## Example (curl)

```bash
curl https://api.llm.buzz/api/v1/generation \
  -H "Authorization: Bearer $LLM_BUZZ_KEY"
```

## Example response

```json
{
  "data": {
    "id": "gen-...",
    "model": "hexgrad/kokoro-82m",
    "api_type": "tts",
    "provider_name": "DeepInfra",
    "total_cost": 0.00002277,
    "is_byok": false,
    "upstream_inference_cost": null
  }
}
```

# List models

`GET https://api.llm.buzz/api/v1/models`

The models your key can call, in OpenRouter's shape. Prices are decimal strings in US dollars per token. An audio model's `prompt` price is per second of audio or per character of input; an image model lists its output price in `image_output`, and a model priced per image adds `image_token`. With a key, a model carries `reasoning_efforts` where its provider lists them. A speech-to-text model carries `verbose_json`, and a text-to-speech model `supported_voices`, `default_voice` and `speech_speed`. Without an `Authorization` header it lists the public models, priced at each model's first-ranked public provider.

Authentication: optional; without a key it answers the public catalog.

## Parameters

- `output_modalities` (query string): `text` by default; `all` lists every model, and a comma list such as `text,embeddings` lists models that produce any of them. The values are `text`, `image`, `embeddings`, `transcription` (speech to text) and `speech` (text to speech).
- `limit` (query integer): Page size.
- `offset` (query integer): Rows to skip.
- `region` (query string, one of `eu`): `eu` lists only models with an EU provider, priced over those providers. It changes the listing, never how a call routes.

## Responses

- 200: `data`, `total_count` (the matches before paging) and `links`.
- 400: A field is missing or malformed (`invalid_request`, with `metadata.param`), or the model cannot do what the request asks (`unsupported_capability`).

## Example (curl)

```bash
curl https://api.llm.buzz/api/v1/models \
  -H "Authorization: Bearer $LLM_BUZZ_KEY"
```

## Example response

```json
{
  "object": "list",
  "data": [
    {
      "id": "deepseek/deepseek-v4.1-flash",
      "canonical_slug": "deepseek/deepseek-v4.1-flash",
      "name": "DeepSeek: DeepSeek V4.1 Flash",
      "description": "DeepSeek V4.1 Flash is a sparse mixture-of-experts model from DeepSeek...",
      "created": 1792497912,
      "context_length": 1048576,
      "architecture": {
        "input_modalities": [
          "text",
          "image"
        ],
        "output_modalities": [
          "text"
        ],
        "modality": "text+image->text"
      },
      "pricing": {
        "prompt": "0.0000002",
        "completion": "0.0000006",
        "input_cache_read": "0.000000006"
      },
      "top_provider": {
        "is_moderated": false,
        "context_length": 1048576,
        "max_completion_tokens": 131072
      },
      "per_request_limits": null,
      "supported_parameters": [
        "max_tokens",
        "response_format",
        "structured_outputs",
        "temperature",
        "tool_choice",
        "tools"
      ],
      "links": {
        "details": "/api/v1/models/deepseek/deepseek-v4.1-flash/endpoints"
      }
    }
  ],
  "total_count": 54,
  "links": {
    "next": null
  }
}
```

# Get a model

`GET https://api.llm.buzz/api/v1/models/{author}/{slug}`

One model, as the bare model object. A model whose name has no `/` is `GET /api/v1/models/{model_id}`, and `GET /api/v1/model/{author}/{slug}` answers the same model wrapped as `{"data": ...}`. Works without a key for a public model.

Authentication: optional; without a key it answers the public catalog.

## Parameters

- `author` (path string, required): The part of the model's name before `/`, such as `deepseek`.
- `slug` (path string, required): The part after `/`, such as `deepseek-v4.1-flash`.

## Responses

- 200: The model.
- 404: No model has this name, or your key may not call it.

## Example (curl)

```bash
curl https://api.llm.buzz/api/v1/models/deepseek/deepseek-v4.1-flash \
  -H "Authorization: Bearer $LLM_BUZZ_KEY"
```

## Example response

```json
{
  "id": "deepseek/deepseek-v4.1-flash",
  "canonical_slug": "deepseek/deepseek-v4.1-flash",
  "name": "DeepSeek: DeepSeek V4.1 Flash",
  "description": "DeepSeek V4.1 Flash is a sparse mixture-of-experts model from DeepSeek...",
  "created": 1792497912,
  "context_length": 1048576,
  "architecture": {
    "input_modalities": [
      "text",
      "image"
    ],
    "output_modalities": [
      "text"
    ],
    "modality": "text+image->text"
  },
  "pricing": {
    "prompt": "0.0000002",
    "completion": "0.0000006",
    "input_cache_read": "0.000000006"
  },
  "top_provider": {
    "is_moderated": false,
    "context_length": 1048576,
    "max_completion_tokens": 131072
  },
  "per_request_limits": null,
  "supported_parameters": [
    "max_tokens",
    "response_format",
    "structured_outputs",
    "temperature",
    "tool_choice",
    "tools"
  ],
  "links": {
    "details": "/api/v1/models/deepseek/deepseek-v4.1-flash/endpoints"
  }
}
```

# List a model's endpoints

`GET https://api.llm.buzz/api/v1/models/{author}/{slug}/endpoints`

One endpoint per provider that may serve the model to you, each with its own prices, context length and output limit, its `tag` for `provider.order` and `provider.only`, and its recent uptime, latency and throughput. `pricing.discount` is the fraction a running provider discount takes off, with the prices already discounted. llm.buzz's own provider is one endpoint, `provider_name` `llm.buzz` and `tag` `llm-buzz`, priced at the lowest live direct price of each component. `GET /api/v1/models/{model_id}/endpoints` for a name with no `/`.

Authentication: optional; without a key it answers the public catalog.

## Parameters

- `author` (path string, required): The part of the model's name before `/`, such as `deepseek`.
- `slug` (path string, required): The part after `/`, such as `deepseek-v4.1-flash`.

## Responses

- 200: The model and its endpoints, as `{"data": ...}`.
- 404: No model has this name, or your key may not call it.

## Example (curl)

```bash
curl https://api.llm.buzz/api/v1/models/deepseek/deepseek-v4.1-flash/endpoints \
  -H "Authorization: Bearer $LLM_BUZZ_KEY"
```

## Example response

```json
{
  "data": {
    "id": "deepseek/deepseek-v4.1-flash",
    "name": "DeepSeek: DeepSeek V4.1 Flash",
    "endpoints": [
      {
        "name": "DeepInfra | DeepSeek: DeepSeek V4.1 Flash",
        "provider_name": "DeepInfra",
        "tag": "deepinfra",
        "context_length": 1048576,
        "max_completion_tokens": 131072,
        "quantization": null,
        "supports_implicit_caching": true,
        "status": 0,
        "uptime_last_1d": 100,
        "latency_last_30m": null,
        "throughput_last_30m": null,
        "pricing": {
          "prompt": "0.0000002",
          "completion": "0.0000006",
          "input_cache_read": "0.000000006",
          "discount": 0
        }
      }
    ]
  }
}
```

# List embeddings models

`GET https://api.llm.buzz/api/v1/embeddings/models`

The model list with `output_modalities=embeddings` fixed, in the shape of `GET /api/v1/models`: with a key, the embeddings models it can call; without one, the public ones. Each lists as `text->embeddings` with its context window and `pricing` `{prompt, completion: "0"}` in US dollars per token, no cache prices and empty `supported_parameters`.

Authentication: optional; without a key it answers the public catalog.

## Parameters

- `limit` (query integer): Page size.
- `offset` (query integer): Rows to skip.

## Responses

- 200: `data`, `total_count` (the matches before paging) and `links`.
- 400: A field is missing or malformed (`invalid_request`, with `metadata.param`), or the model cannot do what the request asks (`unsupported_capability`).

## Example (curl)

```bash
curl https://api.llm.buzz/api/v1/embeddings/models \
  -H "Authorization: Bearer $LLM_BUZZ_KEY"
```

## Example response

```json
{
  "object": "list",
  "data": [
    {
      "id": "qwen/qwen3-embedding-8b",
      "name": "Qwen: Qwen3 Embedding 8B",
      "context_length": 32768,
      "architecture": {
        "input_modalities": [
          "text"
        ],
        "output_modalities": [
          "embeddings"
        ],
        "modality": "text->embeddings"
      },
      "pricing": {
        "prompt": "0.00000001",
        "completion": "0"
      },
      "supported_parameters": []
    }
  ],
  "total_count": 18,
  "links": {
    "next": null
  }
}
```

# List image models

`GET https://api.llm.buzz/api/v1/images/models`

OpenRouter's image model list: one entry per image model your key may call, with `architecture` (`text` in, `image` out), `supported_parameters`, `supports_streaming` (always `false`) and `endpoints`, the relative URL of its endpoints listing. `supported_parameters` names each field the model takes with its values: `resolution`, `aspect_ratio`, `quality`, `output_format`, `background` and `size` as enums of the declared values, `n` and `output_compression` as ranges, `seed` as a boolean. OpenRouter lists `size` as a boolean; here its enum says which sizes the model accepts. A key is required, and the answer is never cached.

Authentication: an inference key, as `Authorization: Bearer KEY`.

## Responses

- 200: The image models, as `{"data": [...]}`.
- 401: No key, or the key is unknown, expired, revoked or disabled.

## Example (curl)

```bash
curl https://api.llm.buzz/api/v1/images/models \
  -H "Authorization: Bearer $LLM_BUZZ_KEY"
```

## Example response

```json
{
  "data": [
    {
      "id": "qwen/qwen-image-3",
      "name": "Qwen: Qwen Image 3",
      "description": "...",
      "created": 1792900000,
      "architecture": {
        "input_modalities": [
          "text"
        ],
        "output_modalities": [
          "image"
        ]
      },
      "supported_parameters": {
        "aspect_ratio": {
          "type": "enum",
          "values": [
            "1:1",
            "4:3",
            "3:4",
            "16:9",
            "9:16",
            "3:2",
            "2:3"
          ]
        },
        "n": {
          "type": "range",
          "min": 1,
          "max": 6
        }
      },
      "supports_streaming": false,
      "endpoints": "/api/v1/images/models/qwen/qwen-image-3/endpoints"
    }
  ]
}
```

# List an image model's endpoints

`GET https://api.llm.buzz/api/v1/images/models/{author}/{slug}/endpoints`

One endpoint per provider that may serve the image model to you: `provider_name`, `provider_slug`, `provider_tag` (with `/eu` for an EU provider), `supported_parameters` as on the image model list, `allowed_passthrough_parameters` (always `[]`, since every field is checked against the model), `supports_streaming` (`false`) and `pricing` at your rate: an `input_text` and an `output_image` line in US dollars per token for a model priced per token, one `output_image` line with the price of one image for a model priced per image. `GET /api/v1/images/models/{model_id}/endpoints` for a name with no `/`. A key is required, and the answer is never cached.

Authentication: an inference key, as `Authorization: Bearer KEY`.

## Parameters

- `author` (path string, required): The part of the model's name before `/`, such as `qwen`.
- `slug` (path string, required): The part after `/`, such as `qwen-image-3`.

## Responses

- 200: The model's `id` and its `endpoints`.
- 401: No key, or the key is unknown, expired, revoked or disabled.
- 404: No model has this name or your key may not call it; a model that makes no images answers a 404 naming `GET /api/v1/images/models`.

## Example (curl)

```bash
curl https://api.llm.buzz/api/v1/images/models/qwen/qwen-image-3/endpoints \
  -H "Authorization: Bearer $LLM_BUZZ_KEY"
```

## Example response

```json
{
  "id": "qwen/qwen-image-3",
  "endpoints": [
    {
      "provider_name": "Alibaba",
      "provider_slug": "alibaba",
      "provider_tag": "alibaba",
      "supported_parameters": {
        "aspect_ratio": {
          "type": "enum",
          "values": [
            "1:1",
            "4:3",
            "3:4",
            "16:9",
            "9:16",
            "3:2",
            "2:3"
          ]
        },
        "n": {
          "type": "range",
          "min": 1,
          "max": 6
        }
      },
      "allowed_passthrough_parameters": [],
      "supports_streaming": false,
      "pricing": [
        {
          "billable": "output_image",
          "unit": "image",
          "cost_usd": 0.03
        }
      ]
    }
  ]
}
```

# Get the current key

`GET https://api.llm.buzz/api/v1/key`

The key you send, inference or management, in OpenRouter's current-key shape: its `label`, `limit`, `limit_remaining` and `limit_reset`, its spend over its life (`usage`) and in the current UTC day, week and month, `expires_at`, `is_management_key`, and `is_free_tier`, `true` until your organization has bought credit. A key with a limit has one entry in `budgets`, whose `resets_at` says when its period ends (`null` for a lifetime limit). USD amounts are JSON numbers.

Authentication: any key, inference or management, as `Authorization: Bearer KEY`.

## Responses

- 200: The key, as `{"data": ...}`.
- 401: No `Authorization` header (`missing_authentication`), or the key is unknown, expired, revoked or disabled (`invalid_key`).

## Example (curl)

```bash
curl https://api.llm.buzz/api/v1/key \
  -H "Authorization: Bearer $LLM_BUZZ_KEY"
```

## Example response

```json
{
  "data": {
    "label": "CI key",
    "limit": 5,
    "limit_remaining": 3.75,
    "limit_reset": "monthly",
    "include_byok_in_limit": false,
    "usage": 1.25,
    "usage_daily": 0.5,
    "usage_weekly": 1.25,
    "usage_monthly": 1.25,
    "is_free_tier": false,
    "is_management_key": false,
    "is_provisioning_key": false,
    "expires_at": null,
    "budgets": [
      {
        "scope": "key_limit",
        "limit_usd": 5,
        "spend_usd": 1.25,
        "remaining_usd": 3.75,
        "reset_interval": "monthly",
        "resets_at": "2026-11-01T00:00:00Z"
      }
    ],
    "workspace_id": "default",
    "allowed_data_regions": [
      "global",
      "europe",
      "us"
    ]
  }
}
```

# Get credits

`GET https://api.llm.buzz/api/v1/credits`

Your organization's prepaid credit: `total_credits`, the credit it has added, and `total_usage`, what it has spent, in US dollars. Zeros before any credit.

Authentication: a management key, created on the console's Keys page, as `Authorization: Bearer KEY`; the samples read it from `$LLM_BUZZ_MANAGEMENT_KEY`.

## Responses

- 200: The credit, as `{"data": ...}`.
- 401: No `Authorization` header (`missing_authentication`), or the key is unknown, expired, revoked or disabled (`invalid_key`).
- 403: An inference key called a route that takes a management key. Create one on the console's Keys page.

## Example (curl)

```bash
curl https://api.llm.buzz/api/v1/credits \
  -H "Authorization: Bearer $LLM_BUZZ_MANAGEMENT_KEY"
```

## Example response

```json
{
  "data": {
    "total_credits": 25,
    "total_usage": 3.25
  }
}
```

# List keys

`GET https://api.llm.buzz/api/v1/keys`

Your unrevoked inference keys, newest first, 100 per page; never a management key.

Authentication: a management key, created on the console's Keys page, as `Authorization: Bearer KEY`; the samples read it from `$LLM_BUZZ_MANAGEMENT_KEY`.

## Parameters

- `offset` (query integer): Keys to skip.
- `include_disabled` (query boolean): `true` adds disabled keys.
- `include_expired` (query boolean): `true` adds expired keys.

## Responses

- 200: The keys, as `{"data": [...]}`.
- 400: The body is not JSON (`invalid_json`), a field or query value is invalid (`invalid_parameter`, with `metadata.param`), or a `PATCH` sets no field (`invalid_request`).
- 401: No `Authorization` header (`missing_authentication`), or the key is unknown, expired, revoked or disabled (`invalid_key`).
- 403: An inference key called a route that takes a management key. Create one on the console's Keys page.

## Example (curl)

```bash
curl https://api.llm.buzz/api/v1/keys \
  -H "Authorization: Bearer $LLM_BUZZ_MANAGEMENT_KEY"
```

## Example response

```json
{
  "data": [
    {
      "hash": "...",
      "name": "CI key",
      "label": "CI key",
      "disabled": false,
      "created_at": "...",
      "updated_at": null,
      "expires_at": null,
      "last_used_at": null,
      "limit": 5,
      "limit_remaining": 5,
      "limit_reset": "monthly",
      "include_byok_in_limit": false,
      "usage": 0,
      "usage_daily": 0,
      "usage_weekly": 0,
      "usage_monthly": 0,
      "byok_usage": 0,
      "byok_usage_daily": 0,
      "byok_usage_weekly": 0,
      "byok_usage_monthly": 0,
      "creator_user_id": "...",
      "external_user": null,
      "workspace_id": "default"
    }
  ]
}
```

# Create a key

`POST https://api.llm.buzz/api/v1/keys`

Creates an inference key of your own. The answer, 201, carries the key object in `data` and the key itself in `key`, which no later answer repeats. OpenRouter's other create fields, such as `include_byok_in_limit` and `workspace_id`, are accepted and ignored, except `external.api_key`, which is a 400: llm.buzz issues its own keys.

Authentication: a management key, created on the console's Keys page, as `Authorization: Bearer KEY`; the samples read it from `$LLM_BUZZ_MANAGEMENT_KEY`.

## Body

- `name` (string, required): 1 to 64 characters.
- `limit` (number): A spending limit in US dollars, at most nine decimal places; `null` for none.
- `limit_reset` (string, one of `daily`, `weekly`, `monthly`): The limit's period in UTC, a week starting on Monday; `null` counts the key's whole life.
- `expires_at` (string): When the key expires: ISO 8601 with seconds and an offset, in the future, such as `2027-12-31T23:59:59Z`. Omitted, it never expires.

## Responses

- 201: The key object and the key.
- 400: The body is not JSON (`invalid_json`), a field or query value is invalid (`invalid_parameter`, with `metadata.param`), or a `PATCH` sets no field (`invalid_request`).
- 401: No `Authorization` header (`missing_authentication`), or the key is unknown, expired, revoked or disabled (`invalid_key`).
- 403: An inference key called a route that takes a management key. Create one on the console's Keys page.

## Example (curl)

```bash
curl -X POST https://api.llm.buzz/api/v1/keys \
  -H "Authorization: Bearer $LLM_BUZZ_MANAGEMENT_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "name": "CI key",
  "limit": 5,
  "limit_reset": "monthly"
}'
```

## Example response

```json
{
  "data": {
    "hash": "...",
    "name": "CI key",
    "label": "CI key",
    "disabled": false,
    "created_at": "...",
    "updated_at": null,
    "expires_at": null,
    "last_used_at": null,
    "limit": 5,
    "limit_remaining": 5,
    "limit_reset": "monthly",
    "include_byok_in_limit": false,
    "usage": 0,
    "usage_daily": 0,
    "usage_weekly": 0,
    "usage_monthly": 0,
    "byok_usage": 0,
    "byok_usage_daily": 0,
    "byok_usage_weekly": 0,
    "byok_usage_monthly": 0,
    "creator_user_id": "...",
    "external_user": null,
    "workspace_id": "default"
  },
  "key": "exp_vk_..."
}
```

# Get a key

`GET https://api.llm.buzz/api/v1/keys/{hash}`

One of your inference keys, by its `hash`.

Authentication: a management key, created on the console's Keys page, as `Authorization: Bearer KEY`; the samples read it from `$LLM_BUZZ_MANAGEMENT_KEY`.

## Parameters

- `hash` (path string, required): The key's `hash`, as `GET /api/v1/keys` lists it.

## Responses

- 200: The key, as `{"data": ...}`.
- 401: No `Authorization` header (`missing_authentication`), or the key is unknown, expired, revoked or disabled (`invalid_key`).
- 403: An inference key called a route that takes a management key. Create one on the console's Keys page.
- 404: The hash names none of your inference keys.

## Example (curl)

```bash
curl https://api.llm.buzz/api/v1/keys/{hash} \
  -H "Authorization: Bearer $LLM_BUZZ_MANAGEMENT_KEY"
```

## Example response

```json
{
  "data": {
    "hash": "...",
    "name": "CI key",
    "label": "CI key",
    "disabled": false,
    "created_at": "...",
    "updated_at": null,
    "expires_at": null,
    "last_used_at": null,
    "limit": 5,
    "limit_remaining": 5,
    "limit_reset": "monthly",
    "include_byok_in_limit": false,
    "usage": 0,
    "usage_daily": 0,
    "usage_weekly": 0,
    "usage_monthly": 0,
    "byok_usage": 0,
    "byok_usage_daily": 0,
    "byok_usage_weekly": 0,
    "byok_usage_monthly": 0,
    "creator_user_id": "...",
    "external_user": null,
    "workspace_id": "default"
  }
}
```

# Update a key

`PATCH https://api.llm.buzz/api/v1/keys/{hash}`

Changes a key's `name`, `disabled`, `limit` or `limit_reset`. A body that sets none of them answers 400 `invalid_request`; `include_byok_in_limit` and unknown fields are ignored.

Authentication: a management key, created on the console's Keys page, as `Authorization: Bearer KEY`; the samples read it from `$LLM_BUZZ_MANAGEMENT_KEY`.

## Parameters

- `hash` (path string, required): The key's `hash`, as `GET /api/v1/keys` lists it.

## Body

- `name` (string): 1 to 64 characters.
- `disabled` (boolean): `true` disables the key, `false` enables it again.
- `limit` (number): A spending limit in US dollars, at most nine decimal places; `null` for none.
- `limit_reset` (string, one of `daily`, `weekly`, `monthly`): The limit's period in UTC, a week starting on Monday; `null` counts the key's whole life.

## Responses

- 200: The changed key, as `{"data": ...}`.
- 400: The body is not JSON (`invalid_json`), a field or query value is invalid (`invalid_parameter`, with `metadata.param`), or a `PATCH` sets no field (`invalid_request`).
- 401: No `Authorization` header (`missing_authentication`), or the key is unknown, expired, revoked or disabled (`invalid_key`).
- 403: An inference key called a route that takes a management key. Create one on the console's Keys page.
- 404: The hash names none of your inference keys.

## Example (curl)

```bash
curl -X PATCH https://api.llm.buzz/api/v1/keys/{hash} \
  -H "Authorization: Bearer $LLM_BUZZ_MANAGEMENT_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "limit": 10,
  "limit_reset": "weekly"
}'
```

## Example response

```json
{
  "data": {
    "hash": "...",
    "name": "CI key",
    "label": "CI key",
    "disabled": false,
    "created_at": "...",
    "updated_at": null,
    "expires_at": null,
    "last_used_at": null,
    "limit": 5,
    "limit_remaining": 5,
    "limit_reset": "monthly",
    "include_byok_in_limit": false,
    "usage": 0,
    "usage_daily": 0,
    "usage_weekly": 0,
    "usage_monthly": 0,
    "byok_usage": 0,
    "byok_usage_daily": 0,
    "byok_usage_weekly": 0,
    "byok_usage_monthly": 0,
    "creator_user_id": "...",
    "external_user": null,
    "workspace_id": "default"
  }
}
```

# Revoke a key

`DELETE https://api.llm.buzz/api/v1/keys/{hash}`

Revokes the key for good. The answer is `{"deleted": true}`, also when another request revoked it first.

Authentication: a management key, created on the console's Keys page, as `Authorization: Bearer KEY`; the samples read it from `$LLM_BUZZ_MANAGEMENT_KEY`.

## Parameters

- `hash` (path string, required): The key's `hash`, as `GET /api/v1/keys` lists it.

## Responses

- 200: The key is revoked.
- 401: No `Authorization` header (`missing_authentication`), or the key is unknown, expired, revoked or disabled (`invalid_key`).
- 403: An inference key called a route that takes a management key. Create one on the console's Keys page.
- 404: The hash names none of your inference keys.

## Example (curl)

```bash
curl -X DELETE https://api.llm.buzz/api/v1/keys/{hash} \
  -H "Authorization: Bearer $LLM_BUZZ_MANAGEMENT_KEY"
```

## Example response

```json
{
  "deleted": true
}
```

# Site facts

`GET https://api.llm.buzz/public/v1/site`

Counts of public models and providers, the standard rate, and the billing facts: the card fee, the smallest and largest top-up, auto top-up and the refund window. Amounts are in US cents and rates in basis points. It is the website's data, not a stable API.

Authentication: none, the route is keyless.

## Responses

- 200: The site document.
- 400: A query string other than `revision`, a whole number. The body is `{"error": {"code", "message"}}`.
- 503: The public data is busy, or the request pinned a `revision` this server has not published yet. Retry after a second, as `Retry-After: 1` says.

## Example (curl)

```bash
curl https://api.llm.buzz/public/v1/site
```

## Example response

```json
{
  "schema_version": 1,
  "generated_at": "2026-10-26T03:02:52Z",
  "revision": 915,
  "counts": {
    "models": 85,
    "providers": 11,
    "independent_providers": 1,
    "eu_providers": 3,
    "tokens_4w": null
  },
  "standard_rate_bps": 0,
  "provider_cut_bps": 0,
  "billing": {
    "enforced": true,
    "purchases": true,
    "fee_bps": 550,
    "fee_minimum_cents": 80,
    "minimum_cents": 500,
    "maximum_cents": 2500000,
    "auto_topup": true,
    "refund_window_hours": 24
  },
  "minimum_organizations": 5,
  "provider_performance": true,
  "provider_usage": true
}
```

# Public models

`GET https://api.llm.buzz/public/v1/models`

Every public model, at most 1,000, by name, with its price, features, providers, EU and data-policy flags, and published metrics. Prices are decimal strings in US dollars per million units of `price_unit`: `token`, `second` of audio, `character` of input text, or `image`. They are those of the model's first-ranked live provider. It is the website's data, not a stable API.

Authentication: none, the route is keyless.

## Responses

- 200: The model list.
- 400: A query string other than `revision`, a whole number. The body is `{"error": {"code", "message"}}`.
- 503: The public data is busy, or the request pinned a `revision` this server has not published yet. Retry after a second, as `Retry-After: 1` says.

## Example (curl)

```bash
curl https://api.llm.buzz/public/v1/models
```

## Example response

```json
{
  "schema_version": 1,
  "generated_at": "2026-10-26T03:02:52Z",
  "revision": 915,
  "truncated": false,
  "models": [
    {
      "name": "deepseek/deepseek-v4.1-flash",
      "display_name": "DeepSeek: DeepSeek V4.1 Flash",
      "author": "deepseek",
      "author_name": "DeepSeek",
      "input_modalities": [
        "text",
        "image"
      ],
      "output_modalities": [
        "text"
      ],
      "context_length": 1048576,
      "max_output_tokens": 131072,
      "features": [
        "tools",
        "structured_output",
        "caching"
      ],
      "open_weights": true,
      "providers": {
        "live": 3,
        "paused": 0
      },
      "provider_names": [
        "DeepInfra",
        "Scaleway",
        "Prime Intellect"
      ],
      "eu_available": true,
      "free": false,
      "available": true,
      "price_varies": true,
      "price_unit": "token",
      "price": {
        "input": "0.2",
        "output": "0.6",
        "cache_read": "0.006",
        "cache_write": "0",
        "cache_write_1h": "0",
        "reasoning": "0.6"
      },
      "price_discount": null
    }
  ]
}
```

# Public model

`GET https://api.llm.buzz/public/v1/models/{author}/{slug}`

One public model with its providers in rank order, each with its prices, limits, residency, data policy and metrics, plus the model's uptime, usage, questions and similar models. `auto_price` is `true` on the row of llm.buzz's own provider, whose prices are the lowest live direct price of each component (the Auto price). A model whose name has no `/` is `GET /public/v1/models/{name}`. Send the name's `/` as it is: an encoded `%2F` is a 404. It is the website's data, not a stable API.

Authentication: none, the route is keyless.

## Parameters

- `author` (path string, required): The part of the model's name before `/`, such as `deepseek`.
- `slug` (path string, required): The part after `/`, such as `deepseek-v4.1-flash`.

## Responses

- 200: The model document.
- 400: A query string other than `revision`, a whole number. The body is `{"error": {"code", "message"}}`.
- 404: No public model has this name: `no public model named 'acme/nope'; see /public/v1/models`.
- 503: The public data is busy, or the request pinned a `revision` this server has not published yet. Retry after a second, as `Retry-After: 1` says.

## Example (curl)

```bash
curl https://api.llm.buzz/public/v1/models/deepseek/deepseek-v4.1-flash
```

## Example response

```json
{
  "schema_version": 1,
  "model": {
    "name": "deepseek/deepseek-v4.1-flash",
    "display_name": "DeepSeek: DeepSeek V4.1 Flash",
    "author": "deepseek",
    "author_name": "DeepSeek",
    "input_modalities": [
      "text",
      "image"
    ],
    "output_modalities": [
      "text"
    ],
    "context_length": 1048576,
    "max_output_tokens": 131072,
    "features": [
      "tools",
      "structured_output",
      "caching"
    ],
    "open_weights": true,
    "providers": {
      "live": 3,
      "paused": 0
    },
    "provider_names": [
      "DeepInfra",
      "Scaleway",
      "Prime Intellect"
    ],
    "eu_available": true,
    "free": false,
    "available": true,
    "price_varies": true,
    "price_unit": "token",
    "price": {
      "input": "0.2",
      "output": "0.6",
      "cache_read": "0.006",
      "cache_write": "0",
      "cache_write_1h": "0",
      "reasoning": "0.6"
    },
    "price_discount": null
  },
  "providers": [
    {
      "rank": 1,
      "slug": "deepinfra",
      "tag": "deepinfra",
      "display_name": "DeepInfra",
      "type": "cloud",
      "residency": "non-eu",
      "state": "live",
      "context_length": 1048576,
      "max_output_tokens": 131072,
      "prices": {
        "input": "0.2",
        "output": "0.6",
        "cache_read": "0.006",
        "cache_write": "0",
        "cache_write_1h": "0",
        "reasoning": "0.6"
      },
      "discount": null,
      "auto_price": false,
      "cache_capable": true,
      "data_policy": null
    }
  ]
}
```

# Public providers

`GET https://api.llm.buzz/public/v1/providers`

Every public provider: its slug, display name, description, logo, type (`cloud` or `independent`), residency (`eu` or `non-eu`), declared data policy, the number of models it serves live and when it first appeared. `llm-buzz` is llm.buzz's own provider, priced at the lowest live direct price of each component of each model. It is the website's data, not a stable API.

Authentication: none, the route is keyless.

## Responses

- 200: The provider list.
- 400: A query string other than `revision`, a whole number. The body is `{"error": {"code", "message"}}`.
- 503: The public data is busy, or the request pinned a `revision` this server has not published yet. Retry after a second, as `Retry-After: 1` says.

## Example (curl)

```bash
curl https://api.llm.buzz/public/v1/providers
```

## Example response

```json
{
  "schema_version": 1,
  "generated_at": "2026-10-26T03:02:52Z",
  "revision": 915,
  "providers": [
    {
      "slug": "llm-buzz",
      "display_name": "llm.buzz",
      "description": "llm.buzz's own routes to a model. They bill each model at its best direct provider's price, shown with the Auto tag, and take part in routing and failover like any other provider.",
      "logo_url": "/public/v1/logos/bbda16277217d4e6bcb4e654d96d6e5a6efd6e4edc7fdd403e1b68d104270c66.svg",
      "type": "cloud",
      "residency": "non-eu",
      "eu": false,
      "data_policy": null,
      "models": 36,
      "first_seen_at": null
    },
    {
      "slug": "scaleway",
      "display_name": "Scaleway",
      "description": "The French cloud provider, part of the iliad group. Its Generative APIs serve open-weight models from data centers in France.",
      "logo_url": "/public/v1/logos/b8f6af777a33df860fb3629e6247044b9de981f63a85ce1b0a340310f5d3332d.png",
      "type": "cloud",
      "residency": "eu",
      "eu": true,
      "data_policy": null,
      "models": 10,
      "first_seen_at": "2026-10-21T09:23:58Z"
    }
  ]
}
```

## Models

Prices are the first-ranked provider's, in USD per million tokens unless the price names another unit (an audio model's input, per million seconds of audio or characters; an image model's output, per image), a running discount applied and noted after the price. A model with several providers may cost more on another.

| Model | Name | Context | Max output | Input | Output | Cache read | Modalities | Features | Providers | EU |
|---|---|---|---|---|---|---|---|---|---|---|
| `anthropic/claude-fable-5` | Anthropic: Claude Fable 5 | 1,000,000 | 128,000 | $7.50 | $37.50 (25% off until 2026-11-06 00:00 UTC) | $0.75 | text+image to text | tools, structured_output, caching | 2 | no |
| `anthropic/claude-fable-5.1` | Anthropic: Claude Fable 5.1 | 1,000,000 | 128,000 | $7.50 | $37.50 (25% off until 2026-11-06 00:00 UTC) | $0.1875 | text+image to text | tools, structured_output, caching | 2 | no |
| `anthropic/claude-haiku-5.5` | Anthropic: Claude Haiku 5.5 | 1,000,000 | 128,000 | $0.10 | $0.50 | $0.01 | text+image to text | tools, structured_output, caching | 2 | no |
| `anthropic/claude-opus-4.7` | Anthropic: Claude Opus 4.7 | 1,000,000 | 128,000 | $3.75 | $18.75 (25% off until 2026-11-06 00:00 UTC) | $0.375 | text+image to text | tools, structured_output, caching | 2 | no |
| `anthropic/claude-opus-4.8` | Anthropic: Claude Opus 4.8 | 1,000,000 | 128,000 | $3.75 | $18.75 (25% off until 2026-11-06 00:00 UTC) | $0.375 | text+image to text | tools, structured_output, caching | 2 | no |
| `anthropic/claude-opus-5` | Anthropic: Claude Opus 5 | 1,000,000 | 128,000 | $3.75 | $18.75 (25% off until 2026-11-06 00:00 UTC) | $0.375 | text+image to text | tools, structured_output, caching | 2 | no |
| `anthropic/claude-opus-5.5` | Anthropic: Claude Opus 5.5 | 1,000,000 | 128,000 | $3.00 | $15.00 (25% off until 2026-11-06 00:00 UTC) | $0.15 | text+image to text | tools, structured_output, caching | 2 | no |
| `anthropic/claude-sonnet-5` | Anthropic: Claude Sonnet 5 | 1,000,000 | 128,000 | $1.50 | $7.50 (25% off until 2026-11-06 00:00 UTC) | $0.15 | text+image to text | tools, structured_output, caching | 2 | no |
| `anthropic/claude-sonnet-5.5` | Anthropic: Claude Sonnet 5.5 | 1,000,000 | 128,000 | $2.00 | $10.00 | $0.10 | text+image to text | tools, structured_output, caching | 2 | no |
| `baai/bge-base-en-v1.5` | BAAI: bge-base-en-v1.5 | 512 | n/a | $0.005 | n/a | n/a | text to embeddings | none | 1 | no |
| `baai/bge-large-en-v1.5` | BAAI: bge-large-en-v1.5 | 512 | n/a | $0.01 | n/a | n/a | text to embeddings | none | 1 | no |
| `baai/bge-m3` | BAAI: bge-m3 | 8,192 | n/a | $0.01 | n/a | n/a | text to embeddings | none | 1 | no |
| `black-forest-labs/flux.2-klein-4b` | Black Forest Labs: FLUX.2 Klein 4B | n/a | n/a | n/a | $0.014/image | n/a | text to image | none | 1 | no |
| `black-forest-labs/flux.2-max` | Black Forest Labs: FLUX.2 Max | n/a | n/a | n/a | $0.07/image | n/a | text to image | none | 1 | no |
| `black-forest-labs/flux.2-pro` | Black Forest Labs: FLUX.2 Pro | n/a | n/a | n/a | $0.03/image | n/a | text to image | none | 1 | no |
| `deepseek/deepseek-v4-flash-0731` | DeepSeek: DeepSeek V4 Flash 0731 | 1,048,576 | 384,000 | $0.461 | $0.9219 | $0.0922 | text to text | tools, structured_output, caching | 3 | yes |
| `deepseek/deepseek-v4-flash-vision-exp` | DeepSeek: DeepSeek V4 Flash Vision Exp | 1,048,576 | 262,144 | $0.33 | $0.99 (25% off until 2026-11-06 00:00 UTC) | $0.0105 | text+image to text | tools, structured_output, caching | 2 | no |
| `deepseek/deepseek-v4-pro-0813` | DeepSeek: DeepSeek V4 Pro 0813 | 1,048,576 | 16,384 | $1.30 | $2.60 | $0.10 | text to text | tools, structured_output, caching | 2 | no |
| `deepseek/deepseek-v4.1-flash` | DeepSeek: DeepSeek V4.1 Flash | 1,048,576 | 131,072 | $0.30 | $1.20 | $0.006 | text+image to text | tools, structured_output, caching | 3 | yes |
| `deepseek/deepseek-v4.1-flash-1017` | DeepSeek: DeepSeek V4.1 Flash 1017 | 1,048,576 | 131,072 | $0.15 | $0.45 (25% off until 2026-11-06 00:00 UTC) | $0.0045 | text+image to text | tools, structured_output, caching | 3 | no |
| `google/gemini-3-pro-image` | Google: Nano Banana Pro \(Gemini 3 Pro Image\) | n/a | n/a | n/a | $0.1344/image | n/a | text to image | none | 1 | no |
| `google/gemini-3.1-flash-image` | Google: Nano Banana 2 \(Gemini 3.1 Flash Image\) | n/a | n/a | n/a | $0.0672/image | n/a | text to image | none | 1 | no |
| `google/gemini-3.1-flash-lite-image` | Google: Nano Banana 2 Lite \(Gemini 3.1 Flash Lite Image\) | n/a | n/a | n/a | $0.0336/image | n/a | text to image | none | 1 | no |
| `google/gemma-4-26b-a4b-it` | Google: Gemma 4 26B A4B | 262,144 | 16,384 | $0.07 | $0.34 | n/a | text+image to text | tools, structured_output | 3 | yes |
| `google/gemma-4-31b-it` | Google: Gemma 4 31B | 262,144 | 16,384 | $0.15 | $0.30 (25% off until 2026-11-06 00:00 UTC) | n/a | text+image to text | tools, structured_output | 2 | no |
| `hexgrad/kokoro-82m` | hexgrad: Kokoro 82M | n/a | n/a | $0.99/M characters | n/a | n/a | text to speech | none | 1 | no |
| `intfloat/e5-base-v2` | Intfloat: E5-Base-v2 | 512 | n/a | $0.005 | n/a | n/a | text to embeddings | none | 1 | no |
| `intfloat/e5-large-v2` | Intfloat: E5-Large-v2 | 512 | n/a | $0.01 | n/a | n/a | text to embeddings | none | 1 | no |
| `intfloat/multilingual-e5-large` | Intfloat: Multilingual-E5-Large | 512 | n/a | $0.01 | n/a | n/a | text to embeddings | none | 1 | no |
| `mistralai/mistral-medium-3-5` | Mistral: Mistral Medium 3.5 | 180,000 | 16,384 | $1.7286 | $8.6428 | n/a | text+image to text | tools, structured_output | 1 | yes |
| `mistralai/mistral-small-3.2-24b-instruct` | Mistral: Mistral Small 3.2 24B | 128,000 | 32,768 | $0.129675 | $0.30255 (25% off until 2026-11-06 00:00 UTC) | n/a | text+image to text | tools, structured_output | 2 | yes |
| `moondream/moondream3-preview` | Moondream: Moondream 3 Preview | 32,000 | 4,096 | $0.30 | $2.50 | n/a | text+image to text | none | 1 | no |
| `moondream/moondream3.1-9b-a2b` | Moondream: Moondream 3.1 | 32,000 | 4,096 | $0.30 | $2.50 | n/a | text+image to text | none | 1 | no |
| `moonshotai/kimi-k3` | MoonshotAI: Kimi K3 | 1,048,576 | 943,718 | $2.565 | $12.825 (10% off until 2026-11-06 00:00 UTC) | $0.2565 | text+image to text | tools, structured_output, caching | 6 | no |
| `moonshotai/kimi-k3.1-pro` | MoonshotAI: Kimi K3.1 Pro | 1,048,576 | 943,718 | $2.00 | $10.00 | $0.20 | text+image to text | tools, structured_output, caching | 1 | no |
| `nvidia/nemotron-3-super-120b-a12b` | NVIDIA: Nemotron 3 Super | 262,144 | 16,384 | $0.085 | $0.40 | n/a | text to text | tools, structured_output | 1 | no |
| `openai/gpt-4.1` | OpenAI: GPT-4.1 | 1,047,576 | 32,768 | $2.00 | $8.00 | $0.50 | text+image to text | tools, structured_output, caching | 2 | no |
| `openai/gpt-4.1-mini` | OpenAI: GPT-4.1 Mini | 1,047,576 | 32,768 | $0.36 | $1.44 (10% off until 2026-11-06 00:00 UTC) | $0.09 | text+image to text | tools, structured_output, caching | 2 | no |
| `openai/gpt-4o-mini` | OpenAI: GPT-4o-mini | 128,000 | 16,384 | $0.15 | $0.60 | $0.075 | text+image to text | tools, structured_output, caching | 2 | no |
| `openai/gpt-5-mini` | OpenAI: GPT-5 Mini | 400,000 | 128,000 | $0.25 | $2.00 | $0.025 | text+image to text | tools, structured_output, caching | 2 | no |
| `openai/gpt-5-nano` | OpenAI: GPT-5 Nano | 400,000 | 128,000 | $0.045 | $0.36 (10% off until 2026-11-06 00:00 UTC) | $0.0045 | text+image to text | tools, structured_output, caching | 2 | no |
| `openai/gpt-5.4` | OpenAI: GPT-5.4 | 1,050,000 | 128,000 | $2.50 | $15.00 | $0.25 | text+image to text | tools, structured_output, caching | 2 | no |
| `openai/gpt-5.4-mini` | OpenAI: GPT-5.4 Mini | 400,000 | 128,000 | $0.75 | $4.50 | $0.075 | text+image to text | tools, structured_output, caching | 2 | no |
| `openai/gpt-5.4-nano` | OpenAI: GPT-5.4 Nano | 400,000 | 128,000 | $0.20 | $1.25 | $0.02 | text+image to text | tools, structured_output, caching | 2 | no |
| `openai/gpt-5.5` | OpenAI: GPT-5.5 | 1,050,000 | 128,000 | $5.00 | $30.00 | $0.50 | text+image to text | tools, structured_output, caching | 2 | no |
| `openai/gpt-5.6-luna` | OpenAI: GPT-5.6 Luna | 1,050,000 | 128,000 | $0.20 | $1.20 | $0.02 | text+image to text | tools, structured_output, caching | 2 | no |
| `openai/gpt-5.6-sol` | OpenAI: GPT-5.6 Sol | 1,050,000 | 128,000 | $4.00 | $20.00 | $0.40 | text+image to text | tools, structured_output, caching | 2 | no |
| `openai/gpt-5.6-terra` | OpenAI: GPT-5.6 Terra | 1,050,000 | 128,000 | $1.80 | $10.80 (10% off until 2026-11-06 00:00 UTC) | $0.18 | text+image to text | tools, structured_output, caching | 2 | no |
| `openai/gpt-6-astra` | OpenAI: GPT-6 Astra | 1,050,000 | 128,000 | $10.00 | $50.00 | $1.00 | text+image to text | tools, structured_output, caching | 2 | no |
| `openai/gpt-6-luna` | OpenAI: GPT-6 Luna | 1,050,000 | 128,000 | $0.10 | $0.50 | $0.01 | text+image to text | tools, structured_output, caching | 2 | no |
| `openai/gpt-6-sol` | OpenAI: GPT-6 Sol | 1,050,000 | 128,000 | $2.00 | $10.00 | $0.20 | text+image to text | tools, structured_output, caching | 2 | no |
| `openai/gpt-6.1-sol` | OpenAI: GPT-6.1 Sol | 1,050,000 | 128,000 | $2.00 | $10.00 | $0.10 | text+image to text | tools, structured_output, caching | 2 | no |
| `openai/gpt-image-2` | OpenAI: GPT Image 2 | n/a | n/a | $5.00 | $30.00 | n/a | text to image | none | 1 | no |
| `openai/gpt-image-2.5-flare` | OpenAI: GPT Image 2.5 Flare | n/a | n/a | $5.00 | $30.00 | n/a | text to image | none | 1 | no |
| `openai/gpt-image-2.5-sunburst` | OpenAI: GPT Image 2.5 Sunburst | n/a | n/a | $5.00 | $30.00 | n/a | text to image | none | 1 | no |
| `openai/gpt-oss-120b` | OpenAI: gpt-oss-120b | 131,072 | 32,768 | $0.05 | $0.25 | n/a | text to text | tools, structured_output | 3 | yes |
| `openai/text-embedding-3-large` | OpenAI: Text Embedding 3 Large | 8,192 | n/a | $0.13 | n/a | n/a | text to embeddings | none | 1 | no |
| `openai/text-embedding-3-small` | OpenAI: Text Embedding 3 Small | 8,192 | n/a | $0.02 | n/a | n/a | text to embeddings | none | 1 | no |
| `openai/text-embedding-ada-002` | OpenAI: Text Embedding Ada 002 | 8,192 | n/a | $0.10 | n/a | n/a | text to embeddings | none | 1 | no |
| `openai/whisper-large-v3` | OpenAI: Whisper Large V3 | n/a | n/a | $7.50/M seconds | n/a | n/a | audio to transcription | none | 2 | yes |
| `openai/whisper-large-v3-turbo` | OpenAI: Whisper Large V3 Turbo | n/a | n/a | $3.33/M seconds | n/a | n/a | audio to transcription | none | 1 | no |
| `qwen/qwen-image-3` | Qwen: Qwen Image 3 | n/a | n/a | n/a | $0.03/image | n/a | text to image | none | 1 | no |
| `qwen/qwen3-embedding-4b` | Qwen: Qwen3 Embedding 4B | 32,768 | n/a | $0.02 | n/a | n/a | text to embeddings | none | 1 | no |
| `qwen/qwen3-embedding-8b` | Qwen: Qwen3 Embedding 8B | 32,768 | n/a | $0.01 | n/a | n/a | text to embeddings | none | 1 | no |
| `qwen/qwen3.5-0.8b` | Qwen: Qwen3.5-0.8B | 65,536 | 32,768 | $0.04 | $0.08 | n/a | text+image to text | tools, structured_output | 0 | no |
| `qwen/qwen3.5-2b` | Qwen: Qwen3.5-2B | 65,536 | 32,768 | $0.06 | $0.18 | n/a | text+image to text | tools, structured_output | 0 | no |
| `qwen/qwen3.5-4b` | Qwen: Qwen3.5-4B | 65,536 | 32,768 | $0.10 | $0.30 | n/a | text+image to text | tools, structured_output | 0 | no |
| `qwen/qwen3.5-9b` | Qwen: Qwen3.5-9B | 65,536 | 32,768 | $0.18 | $0.54 | n/a | text+image to text | tools, structured_output | 0 | no |
| `qwen/qwen3.6-35b-a3b` | Qwen: Qwen3.6 35B A3B | 262,144 | 65,536 | $0.25 | $1.25 | n/a | text+image to text | tools, structured_output | 3 | yes |
| `qwen/qwen3.8-27b` | Qwen: Qwen3.8 27B | 262,144 | 235,929 | $0.20 | $2.50 | $0.05 | text+image to text | tools, structured_output, caching | 4 | yes |
| `qwen/qwen3.8-flash` | Qwen: Qwen3.8 Flash | 1,000,000 | 131,072 | $0.1125 | $0.3525 (25% off until 2026-11-06 00:00 UTC) | $0.012 | text+image to text | tools, structured_output, caching | 4 | no |
| `qwen/qwen3.8-max-0902` | Qwen: Qwen3.8 Max \(0902\) | 1,000,000 | 131,072 | $2.00 | $6.00 | $0.25 | text+image to text | tools, structured_output, caching | 1 | no |
| `qwen/qwen3.8-omni-flash` | Qwen: Qwen3.8 Omni Flash | 1,000,000 | 131,072 | $0.15 | $0.47 | $0.016 | text+image to text | tools, structured_output, caching | 1 | no |
| `sentence-transformers/all-minilm-l12-v2` | Sentence Transformers: all-MiniLM-L12-v2 | 512 | n/a | $0.005 | n/a | n/a | text to embeddings | none | 1 | no |
| `sentence-transformers/all-minilm-l6-v2` | Sentence Transformers: all-MiniLM-L6-v2 | 512 | n/a | $0.005 | n/a | n/a | text to embeddings | none | 1 | no |
| `sentence-transformers/all-mpnet-base-v2` | Sentence Transformers: all-mpnet-base-v2 | 512 | n/a | $0.005 | n/a | n/a | text to embeddings | none | 1 | no |
| `sentence-transformers/multi-qa-mpnet-base-dot-v1` | Sentence Transformers: multi-qa-mpnet-base-dot-v1 | 512 | n/a | $0.005 | n/a | n/a | text to embeddings | none | 1 | no |
| `sentence-transformers/paraphrase-minilm-l6-v2` | Sentence Transformers: paraphrase-MiniLM-L6-v2 | 512 | n/a | $0.005 | n/a | n/a | text to embeddings | none | 1 | no |
| `surogate/surogate3.7-flash` | Surogate: Surogate 3.7 Flash | 1,048,576 | 32,768 | $0.10 | $0.75 | $0.05 | text+image to text | tools, structured_output, caching | 2 | yes |
| `thenlper/gte-base` | Thenlper: GTE-Base | 512 | n/a | $0.005 | n/a | n/a | text to embeddings | none | 1 | no |
| `thenlper/gte-large` | Thenlper: GTE-Large | 512 | n/a | $0.01 | n/a | n/a | text to embeddings | none | 1 | no |
| `xiaomi/mimo-v2.6-flash` | Xiaomi: MiMo-V2.6-Flash | 1,048,576 | 943,718 | $0.14 | $0.28 | $0.0028 | text+image to text | tools, structured_output, caching | 1 | no |
| `xiaomi/mimo-v2.6-pro` | Xiaomi: MiMo-V2.6-Pro | 1,048,576 | 943,718 | $0.43 | $0.87 | $0.0036 | text+image to text | tools, structured_output, caching | 2 | no |
| `z-ai/glm-5.3` | Z.ai: GLM 5.3 | 1,000,000 | 131,072 | $0.98 | $3.08 | $0.182 | text to text | tools, structured_output, caching | 5 | yes |
| `z-ai/glm-5.3-flash` | Z.ai: GLM 5.3 Flash | 1,048,576 | 943,718 | $0.15 | $0.50 | $0.03 | text+image to text | tools, structured_output, caching | 4 | no |