# Streaming

With `stream: true` the answer arrives as server-sent events while the model writes it. Chat Completions, Responses and Messages all stream, each with its own events.

## Stream a request

The SDKs read the events for you. This loop prints each piece of text as it arrives, and the last chunk carries the usage and its `cost`. You do not need `stream_options`: the usage chunk comes whenever the counts are known.

```python tab="Python"
from openai import OpenAI
import os

client = OpenAI(
    base_url="https://api.llm.buzz/api/v1",
    api_key=os.environ["LLM_BUZZ_KEY"],
)
stream = client.chat.completions.create(
    model="deepseek/deepseek-v4.1-flash",
    messages=[{"role": "user", "content": "Explain a database index in two sentences."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:
        print(f"\n{chunk.usage.total_tokens} tokens, cost {chunk.usage.model_extra.get('cost')}")
```

```ts tab="TypeScript"
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llm.buzz/api/v1",
  apiKey: process.env.LLM_BUZZ_KEY,
});
const stream = await client.chat.completions.create({
  model: "deepseek/deepseek-v4.1-flash",
  messages: [{ role: "user", content: "Explain a database index in two sentences." }],
  stream: true,
});
for await (const chunk of stream) {
  const text = chunk.choices[0]?.delta?.content;
  if (text) process.stdout.write(text);
  if (chunk.usage) {
    const { cost } = chunk.usage as { cost?: number };
    console.log(`\n${chunk.usage.total_tokens} tokens, cost ${cost}`);
  }
}
```

```bash tab="curl"
curl -N https://api.llm.buzz/api/v1/chat/completions \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek/deepseek-v4.1-flash",
    "stream": true,
    "messages": [{"role": "user", "content": "Explain a database index in two sentences."}]
  }'
```

`-N` turns off curl's own buffering, so the lines print as they arrive. Responses and Messages take the same `stream: true` and send their own events, described next.

## Events in each API

| | Chat Completions | Responses | Messages |
|---|---|---|---|
| Opens with | A chunk with the assistant role | `response.created`, then `response.in_progress` | `message_start`, with the message so far |
| Content | `data:` lines, each a `chat.completion.chunk` with `delta.content`, `delta.reasoning`, `delta.tool_calls` or `delta.images`. Every chunk carries `provider`. | OpenAI's output events for text, function-call arguments and reasoning summaries, such as `response.output_text.delta` | `content_block_start`, `content_block_delta` and `content_block_stop` for text, tool input and thinking. One `ping` follows `message_start`. |
| Stop reason | `finish_reason` on the finish chunk, whose `delta` is empty; the usage chunk repeats it | `status` of the final response | `stop_reason` on `message_delta` |
| Usage and cost | A last chunk with `usage`, `cost` inside it | `usage`, `cost` inside it, on the final response | `usage`, `cost` inside it, on `message_delta`. `message_start` carries an early `usage` and never `cost`. |
| Ignored fields | `x-sgr-ignored-parameters` on content and finish chunks, not on the usage or error chunk | On the response object in the events | On `message_start` |
| Ends with | `data: [DONE]` | Exactly one terminal event, which carries the final response | `message_stop` |
| A failure after output | One chunk with an `error` object and `finish_reason: "error"`, then `data: [DONE]` | The terminal event reports the failure | One `error` event in place of `message_stop` |

`x-sgr-ignored-parameters` is a key in the body of an event, not a header. `cost` is left out of `usage` when any part of the cost is unknown. The usage in `message_start` is an early figure: Anthropic's own start counts when Anthropic serves the request, and llm.buzz's estimate of the prompt for any other provider. `message_delta` carries the final counts: [Usage and cost](messages.md#usage-and-cost). The response object in Responses events also echoes `instructions`, `metadata` and `tools`: [The answer](responses.md#the-answer).

## Comments

While the model is silent, llm.buzz sends comment lines, which begin with `:`, to keep the connection active. Any server-sent events client ignores them; ignore them in a parser of your own too. They carry no data, and they do not extend the 600-second limit below.

## Failures

- **Before output reaches you,** a provider that fails or times out is replaced by the next provider of the model, unless you turned fallbacks off, and the failed attempt costs nothing; a provider's own 400 is not tried on another provider. Your stream opens from the provider that answered. When every provider fails before it opens, you get an ordinary HTTP error: [Errors](errors.md).
- **Once output has reached you,** the request stays on the provider that sent it. The first text, tool call or refusal that reaches you fixes the provider, and llm.buzz never splices another provider's output into the stream. A refusal is not retried on another provider.
- **A later failure** ends the stream with an error in your API's form, shown in the table above. The HTTP status was already 200 when the stream opened, so read the last event of every stream before you treat the answer as complete.
- **A provider that stops after output,** closing its stream without a terminal event, is not an error. llm.buzz finishes the answer from what was served: with the finish the provider had already declared, when it answers in OpenAI's Chat Completions form, or otherwise as incomplete (`finish_reason: "length"` on Chat Completions, `status: "incomplete"` on Responses, `stop_reason: "max_tokens"` on Messages), with open items closed and a tool call cut in the middle dropped. A provider that stops before any output is replaced by the next provider.
- **At 600 seconds** a stream still generating ends with a timeout error, followed on Chat Completions by `data: [DONE]`, and is billed for the prompt and the output you received. The request time limit is shared by every provider the request tried.
- **A balance that runs out** while a stream runs does not cut it off. llm.buzz charges the stream when it completes.

## Stopping a stream

On Chat Completions, Responses and Messages, closing the connection makes llm.buzz stop reading from the provider, provided the request carried no `Idempotency-Key`. Messages has no such key, so closing a Messages stream always stops it. A provider may still finish work it had begun.

- You are billed for the prompt and for the output you received. When the provider has not reported the counts, llm.buzz counts them itself.
- A stream you close before the provider accepted the request is charged what llm.buzz reserved for it before calling the provider, the most it could cost.
- A Chat Completions or Responses stream sent with an `Idempotency-Key` keeps running after you leave, within the limit of what llm.buzz keeps for a replay, and a retry with the same key joins it or returns its answer instead of starting a second generation. The request is billed once: [Retries and idempotency](retries-and-idempotency.md).

## What never streams

Embeddings and Images answer 400 to `stream: true`, Audio returns its whole result at once, and Decisions are buffered and refuse a `stream` field. Responses also has a WebSocket transport that carries the same events, one JSON event in each frame: [WebSocket](responses.md#websocket).

## Related

- [Chat Completions](chat-completions.md#stream), [Responses](responses.md#stream) and [Messages](messages.md#stream) list what each API adds to a stream.
- [Tools and structured output](tools-and-structured-output.md) covers streamed tool arguments and what a cut-off call looks like.