# Chat Completions

Chat Completions is OpenAI's conversation API, the one most SDKs and coding agents speak. llm.buzz serves it at `POST /chat/completions` under `https://api.llm.buzz/api/v1` for every text model, with OpenRouter's extensions, so the OpenAI and OpenRouter SDKs call it with the base URL changed.

## Send a request

Send the whole conversation in `messages` on every turn: Chat Completions has no continuation, so a follow-up question carries the turns before it.

```python tab="Python"
from openai import OpenAI
import os

client = OpenAI(
    base_url="https://api.llm.buzz/api/v1",
    api_key=os.environ["LLM_BUZZ_KEY"],
)
reply = client.chat.completions.create(
    model="deepseek/deepseek-v4.1-flash",
    messages=[
        {"role": "system", "content": "You answer in one short sentence."},
        {"role": "user", "content": "Which port does HTTPS use?"},
        {"role": "assistant", "content": "HTTPS uses port 443."},
        {"role": "user", "content": "And plain HTTP?"},
    ],
)
print(reply.choices[0].message.content)
```

```ts tab="TypeScript"
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llm.buzz/api/v1",
  apiKey: process.env.LLM_BUZZ_KEY,
});
const reply = await client.chat.completions.create({
  model: "deepseek/deepseek-v4.1-flash",
  messages: [
    { role: "system", content: "You answer in one short sentence." },
    { role: "user", content: "Which port does HTTPS use?" },
    { role: "assistant", content: "HTTPS uses port 443." },
    { role: "user", content: "And plain HTTP?" },
  ],
});
console.log(reply.choices[0].message.content);
```

```bash tab="curl"
curl https://api.llm.buzz/api/v1/chat/completions \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek/deepseek-v4.1-flash",
    "messages": [
      {"role": "system", "content": "You answer in one short sentence."},
      {"role": "user", "content": "Which port does HTTPS use?"},
      {"role": "assistant", "content": "HTTPS uses port 443."},
      {"role": "user", "content": "And plain HTTP?"}
    ]
  }'
```

The answer is OpenAI's chat completion object with the additions [below](#the-answer). The model name, and the models you can call, come from `GET https://api.llm.buzz/api/v1/models` ([Choosing an API](choosing-an-api.md#find-a-model-for-an-api)).

## The messages array

`messages` is the conversation in order, with the roles `system`, `developer`, `user`, `assistant` and `tool`.

- A `developer` message is accepted for every model. A provider whose API has no developer role receives it as a system message, so the two roles read the same.
- A `system` or `developer` message that comes after the conversation has started stays in place when the provider takes one there. Otherwise its text moves into the user turn at the same place, and the body's `x-sgr-ignored-parameters` names the move, for example `messages.system->folded(system_messages_leading_only)`.
- Message content is a string or a list of parts. The parts are `text` and `image_url`; an image needs a model that lists `image` among its `architecture.input_modalities`. A `tool` message may carry `image_url` parts beside its text. [Images in requests](images-in-requests.md) has the rules.
- A tool call's `id` is an opaque string of 1 to 65,536 characters. Send each one back complete, unchanged, as the `tool_call_id` of its `tool` message. [Tools and structured output](tools-and-structured-output.md) covers the loop.

## Request fields

A field llm.buzz deliberately refuses is a 400 before any provider is called, and a value that fails validation is a 400 `invalid_parameter` with `metadata.param` naming the field. A field llm.buzz drops is named in the body's `x-sgr-ignored-parameters`.

| Field | What llm.buzz does |
|---|---|
| `model` | The model's name, `author/slug`. Four suffixes change one request: `:floor` tries the cheapest provider first, `:nitro` the fastest, `:exacto` is accepted and ignored, and `:online` is accepted and searches nothing (see `web_search_options`). [Models and routing](models-and-routing.md#suffixes). |
| `messages` | Required: a request without it is 400 `invalid_parameter`. [The messages array](#the-messages-array), above. |
| `max_tokens`, `max_completion_tokens`, `max_output_tokens` | The most output tokens, which may include reasoning. Send one of the three: two together are 400 `invalid_parameter`. Without one, llm.buzz does not choose a smaller limit for you. A provider that needs a larger minimum than your limit is skipped, and when none is left the request is refused. |
| `temperature`, `top_p` | Sent to the provider; an out-of-range `temperature` is 400 `invalid_parameter`. A model that takes none, or takes them only at reasoning effort `none`, has the value dropped and named: [Sampling settings a model does not take](api-reference.md#sampling-settings-a-model-does-not-take). |
| `top_k` | Sent to providers that take it. Dropped and named as `top_k->dropped(unsupported_by_provider)` when none does. |
| `frequency_penalty`, `presence_penalty` | Sent. Dropped and named, as `frequency_penalty->dropped(unsupported_by_provider)`, where the model takes none. |
| `seed`, `logit_bias` | Refused: 400 `unsupported_parameter` naming the field. |
| `stop` | Sent to providers that take it. Where a provider's API has no stop field, llm.buzz cuts the answer at the first match itself and ends it with `finish_reason: stop`. |
| `n` | Only `1`. Any other value is 400 `invalid_parameter`. |
| `store` | Only `false`. `true` is 400 `invalid_parameter`. |
| `stream` | `true` streams the answer as server-sent events: [Stream](#stream). |
| `stream_options` | Dropped and named when the request does not stream. On a stream, `include_usage` changes nothing: the last chunk carries usage whenever the counts are known. |
| `tools`, `tool_choice` | Sent, in OpenAI's shape. Some models cannot be forced to call a tool, and a provider that cannot enforce `strict` gets the tool without it. [Tools and structured output](tools-and-structured-output.md) says what happens to each. |
| `parallel_tool_calls` | Honored on every provider. Where a provider has no such control, `true` is its default behavior and is named `parallel_tool_calls->dropped(provider_default)`, and `false` is enforced by llm.buzz, which returns only the first tool call of a turn and names it `parallel_tool_calls->emulated(serialized_by_gateway)`. |
| `response_format` | `json_schema` asks for an answer in a JSON schema, `json_object` for any JSON object. Where a provider has no JSON mode, llm.buzz adds an instruction asking for JSON and names it `response_format->instruction(json_object)`. Use `json_schema` when the shape must be enforced. [Tools and structured output](tools-and-structured-output.md). |
| `verbosity` | `low`, `medium` or `high`, as a hint. Sent to providers that take it and omitted, named `verbosity`, elsewhere. Another value is 400 `invalid_parameter` naming `verbosity`. |
| `logprobs`, `top_logprobs` | `logprobs: true` returns each token's log probability in `choices[0].logprobs.content` (and `.refusal`); `top_logprobs`, 0 to 20, needs `logprobs: true`; a count above 20, or without `logprobs`, is 400 `invalid_parameter`. Sent to providers that return them; a request no provider of the model can serve is refused. A numeric `thinking_budget`, and a stop sequence llm.buzz applies itself, cannot be combined with them. |
| `reasoning`, `reasoning_effort`, `include_reasoning` | How hard the model thinks, and whether you see it. `reasoning` takes `effort` or `max_tokens`, plus `enabled` and `exclude`; an effort tier or a budget beside `enabled: false`, or `effort: "none"` beside `enabled: true`, is 400 `invalid_parameter` naming `reasoning.enabled`. `include_reasoning: true` is `reasoning: {}` and `false` is `reasoning: {exclude: true}`. Each provider receives the setting in its own terms. [Reasoning](reasoning.md). |
| `thinking`, `thinking_budget`, `enable_thinking`, `chat_template_kwargs.enable_thinking` | Other spellings of the same controls, translated for the provider. An enabled `thinking.budget_tokens` under 1,024 is 400 `invalid_parameter`. [Reasoning](reasoning.md). |
| `cache_control` | Anthropic's prompt-cache marker, `{"type": "ephemeral"}` with an optional `ttl` of `"5m"` or `"1h"`. Accepted at the top level, on function tools, messages, text parts and assistant tool calls. Passed on or translated, never added by llm.buzz. [Prompt caching](prompt-caching.md). |
| `prompt_cache_breakpoint` | OpenAI's explicit cache mark, `{"mode": "explicit"}`, on a `text` part of a `system`, `developer`, `user` or `tool` message; at the top level it is dropped and named. Sent to OpenAI as given. For Claude it becomes a five-minute `cache_control` marker, named `prompt_cache_breakpoint->translated(cache_control)`. On another kind of part, or on an assistant message, it is a 400 that names where it was. [Prompt caching](prompt-caching.md#openais-explicit-caching). |
| `prompt_cache_options`, `prompt_cache_retention` | OpenAI's cache mode and retention. Sent to OpenAI only; no other provider receives them, and where no provider of the model takes them, `x-sgr-ignored-parameters` names them. `prompt_cache_retention: "24h"` on a request that demands zero retention is a 400 naming it. [Prompt caching](prompt-caching.md#openais-explicit-caching). |
| `prompt_cache_key` | Names the conversation for caching; `promptCacheKey` works too. It names the session when `session_id` and `x-session-id` are absent. |
| `session_id` | A name for the conversation, 1 to 256 characters, so its turns stay on the provider that holds its cache. An empty or whitespace-only value counts as absent; any other value that is not a string of 1 to 256 characters, such as one over 256 characters, is 400 `invalid_parameter` naming `session_id`. The `x-session-id` header names it too. |
| `provider` | Steers one request: `order`, `only`, `ignore`, `allow_fallbacks`, `sort`, `zdr` and `data_collection`. Other keys, such as `require_parameters`, `quantizations`, `max_price` and `sort.partition`, are accepted and do not change which provider serves. [The provider object](api-reference.md#the-provider-object). |
| `gateway` | llm.buzz's own per-request bounds, `retry` and `routing.allow_fallbacks`. Never sent to a provider. [Retries and fallbacks per request](api-reference.md#retries-and-fallbacks-per-request). |
| `service_tier` | `auto` and `default` are served at the standard tier and not sent to the provider. `flex`, `priority` and `fast` are 400 `unsupported_capability` naming `service_tier`, before any provider is called: llm.buzz sells no priority tier, and no model has a price for one today. |
| `web_search_options`, the `web` plugin | Accepted, and nothing is searched: llm.buzz runs no web search. The request is served without one, nothing is billed for one, and `x-sgr-ignored-parameters` names `web_search->dropped(search_unavailable)`. For an organization whose requests llm.buzz keeps in the EU, the request is 400 `web_search_unavailable_in_region`. |
| other `plugins` entries | Dropped and named by id, as `plugins.response-healing`. |
| `models`, `route`, `transforms`, `usage`, `trace`, `debug` | Dropped and named. A model's own providers fail over through its routing preset, in place of `models`. Usage counts and `usage.cost` come back whether or not you send `usage`. |
| `audio`, `modalities`, `functions`, `function_call`, `prediction`, `moderation` | Refused: 400 `unsupported_parameter` naming the field. Send `tools` and `tool_choice` in place of `functions` and `function_call`. |
| `defer_loading` on a tool | Declare a tool search with `{"type": "openrouter:tool_search"}`, and a tool with `defer_loading: true` beside `function` stays out of the prompt until a search finds it. [Tool search](api-reference.md#tool-search). |
| Any other field | Dropped, and named in `x-sgr-ignored-parameters`, as `previous_response_id` is: it belongs to Responses. |

## Headers

You send:

| Header | Use |
|---|---|
| `Authorization` | `Bearer` and your inference key. [Authentication](authentication.md). |
| `Idempotency-Key` | Makes a retry safe: [Retries](#retries). |
| `x-session-id` | Names the conversation, as `session_id` does. |
| `X-Title`, `HTTP-Referer` | Name your app. [App attribution](api-reference.md#app-attribution). |
| `x-client-request-id` | Your own correlation ID, echoed on the answer. It also names the cache session when nothing else does, so send one stable value per conversation, or omit it: a value that changes on every request gives every request its own cache key. |

Every served answer carries:

| Header | Value |
|---|---|
| `x-request-id` | The request's ID. Its generation ID is `gen-` and the same 32 hex digits: [Cost of a finished request](api-reference.md#cost-of-a-finished-request). |
| `x-gateway-alias` | The model name that served. |
| `x-gateway-canonical-model` | The exact model that produced the answer. |
| `x-gateway-provider` | The display name of the provider that answered, `llm.buzz` for llm.buzz's own provider. |
| `x-gateway-route-depth`, `x-gateway-route-reason` | The answering provider's position in the order tried, and why the route was chosen. |

An answer that asks you to wait, a 429 and some 402 and 503 answers, carries `Retry-After`: wait that many seconds before you retry.

## The answer

A non-streamed answer is OpenAI's chat completion object. llm.buzz adds or fixes these parts:

- `choices[0].message` holds `content`, and where they apply `reasoning` (the model's readable reasoning, left out under `reasoning.exclude`), `tool_calls`, and `images`. A model that makes images returns each as a complete base64 image in `images[].image_url.url`.
- `finish_reason` is `tool_calls` when the turn ends with tool calls, `length` when it hit the output limit, and `stop` otherwise, a matched stop sequence included. A stream that fails after it opened finishes with `error`. `native_finish_reason` sits beside it.
- `system_fingerprint` is always `null`.
- `provider` is the display name of the provider that answered, after any failover; `llm.buzz` for llm.buzz's own provider.
- `usage` is described [below](#usage-and-cost). It is left out, never `null`, when the provider reported no counts.
- `x-sgr-ignored-parameters` is a key in the body, not a header. It lists every field llm.buzz dropped and every value it changed, with the reason where there is one, such as `temperature->dropped(unsupported_by_provider)`; an unknown field is named bare. Check it when a setting seems to have no effect.

## Stream

Set `stream: true` and the answer arrives as server-sent events: `data:` lines, each one `chat.completion.chunk`, ending with `data: [DONE]`.

| Chunk | What it carries |
|---|---|
| Every chunk | `provider`, the provider that is answering. |
| Content chunks | Deltas: `delta.content`, `delta.reasoning`, `delta.tool_calls` and `delta.images`, each image complete. |
| Content and finish chunks | `x-sgr-ignored-parameters`, when the request had a dropped field. The usage and error chunks do not carry it. |
| The last chunk | `usage`, with the token counts and `cost`, whenever the counts are known, whatever `stream_options.include_usage` says. |
| A failure after output | One chunk with an `error` object and `finish_reason: "error"`, then `data: [DONE]`. A failure before the stream opens is an ordinary HTTP error. |

While the model is silent, comment lines beginning with `:` keep the connection active; clients skip them. Closing a stream sent without an `Idempotency-Key` stops the request, and the prompt and the output produced up to then are billed. [Streaming](streaming.md) covers the event shapes of all three text APIs.

## Usage and cost

| Field | Value |
|---|---|
| `prompt_tokens`, `completion_tokens`, `total_tokens` | The counts the provider reported for the attempt that answered. |
| `prompt_tokens_details.cached_tokens` | Prompt tokens read from the provider's cache. |
| `prompt_tokens_details.cache_write_tokens` | Prompt tokens written to it. |
| `prompt_tokens_details.cache_write_1h_tokens` | The one-hour part of the writes, only when the provider reports the split. |
| `completion_tokens_details.reasoning_tokens` | Reasoning tokens, where the provider reports them. A stream's usage chunk always has it: `0` when the provider reported no count, and then `usage.unreported_token_details` lists `reasoning_tokens`. |
| `cost` | The request's total cost in US dollars at your rate. |

Cache reads and writes are separate parts of `prompt_tokens`. A cache field appears only when the provider reported it, except that a stream's usage chunk shows `cached_tokens: 0` when the count is unknown. `cost` is a JSON number that includes any billed tool-search rounds, so it can exceed the counts times the listed prices. A failed attempt that another provider replaced adds 0. It is left out when any part of the cost is unknown; a model with no price reports `0`. A replay with the same `Idempotency-Key` returns the original cost. [Usage and limits](usage-and-limits.md) covers the rest, and [Prompt caching](prompt-caching.md#read-the-cache-in-usage) the cache fields.

## Retries

Send an `Idempotency-Key` header and a retry with the same key and the same body never runs the request twice: it returns the first answer with its original cost, or 409 `idempotency_replay_unavailable` when that answer is not stored where the retry arrived. That 409 means the first request ran or is still running, so retry with the same key after a short backoff. The exception is a first call that ended in an error answer: its key answers 409 for good, so send a new key. [What to send next](retries-and-idempotency.md#what-to-send-next) tells the cases apart.

- The same key with another body is 409 `idempotency_conflict`. A changed suffix or `provider` hint counts as another body; a changed `stream_options.include_usage` does not.
- A retry without the key, or with a new one, runs the request again and bills it again.
- A stream you drop while it carries a key keeps running, and a retry with the key joins it instead of starting a second generation. Without a key, two requests with the same prompt are two requests.
- A provider that fails or times out before output reaches you is replaced by the next one, and the failed attempt costs nothing; a provider's own 400 is not tried on another provider. Once output has reached you, the request stays on the provider that sent it.

[Retries and idempotency](retries-and-idempotency.md) explains the rest.

## Errors

Errors come in OpenRouter's envelope. `metadata.gateway_code` is the code to branch on, and `metadata.param` names the field at fault.

```json
{"error": {"code": 400, "message": "...", "metadata": {"error_type": "invalid_request", "gateway_code": "unsupported_parameter", "param": "seed"}}}
```

| Status | Code | When |
|---|---|---|
| 400 | `invalid_parameter` | A field fails validation and `metadata.param` names it, or `body` when the fault spans fields: a missing `messages`, an out-of-range `temperature`, `top_logprobs` above 20 or without `logprobs`, an unknown `verbosity`, a `session_id` over 256 characters, two of `max_tokens`, `max_completion_tokens` and `max_output_tokens`, a `thinking.budget_tokens` under 1,024, `n` other than 1, or `store: true`. Fix the field. |
| 400 | `invalid_json` | The body is not JSON. |
| 400 | `unsupported_parameter` | You sent a refused field: `seed`, `logit_bias`, `audio`, `modalities`, `functions`, `function_call`, `prediction` or `moderation`. Remove it. |
| 400 | `unsupported_capability` | `service_tier` is `flex`, `priority` or `fast`, or no provider of the model offers what the request asks. `metadata.param` names it. |
| 400 | `invalid_request` | A refusal after llm.buzz read the body: the provider's own 400, whose sentence follows `provider rejected the request:`, or another admission refusal. The message says which. |
| 400 | `context_length_exceeded` | The prompt fits no provider's context window. |
| 400 | `web_search_unavailable_in_region` | The request asked for a web search and your organization's requests stay in the EU. |
| 409 | `idempotency_conflict` | The `Idempotency-Key` was used with another body. |
| 409 | `idempotency_replay_unavailable` | The request ran or is running, and its answer is not stored where this retry arrived. Retry with the same key after a short backoff. If the first call ended in an error answer, the key answers 409 for good: send a new key. |
| 413 | `request_too_large` | The body is over 64 MiB. |

A stream that fails after output ends with the error chunk described under [Stream](#stream). [Errors](errors.md) lists every code, including the key, balance, model and provider errors this API shares with the others.

## Limits

- One completion per request: `n` is 1.
- A model's context window is `context_length` on its entry in `GET https://api.llm.buzz/api/v1/models`, and `top_provider.max_completion_tokens` is its output ceiling.
- A request may run for 600 seconds. A stream still generating then ends with a timeout error chunk and `data: [DONE]`, and is billed for the prompt and the output you received. A request that is not streaming answers 504 `deadline_exceeded`.
- A request body over 64 MiB answers 413 `request_too_large`.
- A request makes at most 8 provider calls, every fallback included; `gateway.retry` lowers the ceiling.

[Usage and limits](usage-and-limits.md#limits) collects the limits of every API.

## Related

- [Tools and structured output](tools-and-structured-output.md), [Streaming](streaming.md), [Reasoning](reasoning.md), [Images in requests](images-in-requests.md) and [Prompt caching](prompt-caching.md) cover what the three text APIs share.
- [Responses](responses.md) and [Messages](messages.md) are the other two text APIs; [Choosing an API](choosing-an-api.md) compares them.
- [Coming from OpenRouter](from-openrouter.md) lists every difference from OpenRouter in one place.