Inference

Responses

OpenAI's Responses API. Continue a conversation with previous_response_id. Set stream: true for Responses stream events.

POST/api/v1/responsesTry it

Your key

Request

curl -X POST https://api.llm.buzz/api/v1/responses \
  -H "Authorization: Bearer $LLM_BUZZ_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "deepseek/deepseek-v4.1-flash",
  "input": "Say hello in one word."
}'

import os
from openai import OpenAI

client = OpenAI(base_url="https://api.llm.buzz/api/v1", api_key=os.environ["LLM_BUZZ_KEY"])
response = client.responses.create(model="deepseek/deepseek-v4.1-flash", input="Say hello in one word.")
print(response.output_text)

Response

{
  "id": "resp_...",
  "object": "response",
  "model": "deepseek/deepseek-v4.1-flash",
  "status": "completed",
  "output": [
    {
      "type": "message",
      "role": "assistant",
      "content": [
        {
          "type": "output_text",
          "text": "Hello!"
        }
      ]
    }
  ],
  "usage": {
    "input_tokens": 12,
    "output_tokens": 3,
    "total_tokens": 15,
    "input_tokens_details": {
      "cached_tokens": 0,
      "cache_write_tokens": 0
    },
    "cost": 0.0000072
  }
}

{
  "error": {
    "code": 400,
    "message": "...",
    "metadata": {
      "error_type": "invalid_request",
      "gateway_code": "invalid_request",
      "param": "messages"
    }
  }
}

{
  "error": {
    "code": 401,
    "message": "A valid gateway Bearer key is required. Send the virtual key as 'Authorization: Bearer <key>'.",
    "metadata": {
      "error_type": "authentication",
      "gateway_code": "invalid_key"
    }
  }
}

{
  "error": {
    "code": 402,
    "message": "...",
    "metadata": {
      "error_type": "payment_required",
      "gateway_code": "insufficient_credits"
    }
  }
}

{
  "error": {
    "code": 404,
    "message": "The requested model does not exist or is not granted to this key. GET /api/v1/models lists the model aliases available to this key.",
    "metadata": {
      "error_type": "not_found",
      "gateway_code": "model_not_found",
      "param": "model"
    }
  }
}

{
  "error": {
    "code": 409,
    "message": "...",
    "metadata": {
      "error_type": "invalid_request",
      "gateway_code": "idempotency_replay_unavailable",
      "param": "Idempotency-Key"
    }
  }
}

{
  "error": {
    "code": 429,
    "message": "...",
    "metadata": {
      "error_type": "rate_limit_exceeded",
      "gateway_code": "unavailable_route"
    }
  }
}

Authorization

An inference key, as Authorization: Bearer KEY.

Keys and limits

Parameters

Idempotency-Keystringheader
A retry with the same key and the same body never runs the request twice: it returns the first answer, or 409 idempotency_replay_unavailable when that answer is not stored where the retry arrived. Send the same key again only when the first call got no answer, or after an answer carrying x-gateway-admission-refused: true; after any other error answer send a new key. A retry without the key, or with a new one, runs and bills the request again. Reusing a key with a different body answers 409 idempotency_conflict.
x-session-idstringheader
Names the conversation, as session_id does, so its turns stay on the provider that holds its cache.

Body application/json

modelstringrequired
A model's name from GET /api/v1/models, such as deepseek/deepseek-v4.1-flash. A suffix changes one request: :floor tries the cheapest provider first and :nitro the fastest. On Chat Completions and Responses :online asks for a web search, which llm.buzz does not run today: the request is served without one and x-sgr-ignored-parameters names web_search->dropped(search_unavailable), or for an organization whose requests llm.buzz keeps in the EU it answers 400 web_search_unavailable_in_region. On Messages :online does nothing and is named as model:online.
inputstring or object[]required
A string, or a list of input items in OpenAI's shape. An input_text part may carry prompt_cache_breakpoint, OpenAI's explicit cache mark.
instructionsstring
A system message for this turn.
previous_response_idstring
Continue from an earlier response.
streambooleandefault false
Stream Responses events.
reasoningobject
effort, translated for each provider.
toolsobject[]
Function tools in the Responses shape; {"type": "tool_search"} declares a tool search over tools marked defer_loading: true.
providerobject
Steers one request across the model's providers. With the OpenAI SDK, send it in extra_body. Every field accepts null. A name matches a provider's display name, its tag or the tag's first part, ignoring case: llm.buzz or llm-buzz names llm.buzz's own provider, and llm-buzz/eu only its capacity in the EU.
Show 7 child fields
- orderstring[]
  Try these providers first, in this order, then the others. At most 64 names.
- onlystring[]
  Use only these providers.
- ignorestring[]
  Never use these providers.
- allow_fallbacksboolean
  false stops moving to other providers.
- sortstring
  The order to try providers in. Ignored when order is set.
  
  pricethroughputlatencysmart
- zdrboolean
  true keeps the request on providers that declare 0 days of retention.
- data_collectionstring
  deny keeps the request on providers that declare no training and 0 days of retention.
  
  allowdeny
session_idstring
A name for the conversation, 1 to 256 characters, so its turns stay on the provider that holds its cache.
cache_controlobject
Anthropic's prompt-cache marker, {"type": "ephemeral"} with an optional ttl of "5m" or "1h". llm.buzz passes it on and never adds one. Claude models cache only where a request marks it.
prompt_cache_optionsobject
OpenAI's explicit caching, for GPT-5.6 and later models: mode implicit (OpenAI's default) or explicit (OpenAI writes the cache only at a prompt_cache_breakpoint), and ttl 30m.
prompt_cache_retentionstring
OpenAI's cache retention. 24h with provider.zdr, provider.data_collection: "deny" or your organization's no-retention policy is a 400 invalid_request.

in_memory24h

Response 200

A response object. Cache reads show in usage.input_tokens_details.cached_tokens and writes in its cache_write_tokens; readable reasoning arrives as a reasoning item's summary_text.

Errors

400
A field is missing or malformed (invalid_request, with metadata.param), or the model cannot do what the request asks (unsupported_capability).
401
No key, or the key is unknown, expired, revoked or disabled.
402
Your balance is spent and the model is not free, or the request could cost more than your balance.
404
No model has this name, or your key may not call it.
409
The Idempotency-Key was used before. idempotency_replay_unavailable: the first request was accepted, and its answer is not stored where this retry arrived, so it was not run again; retry with the same key after a short backoff first, when the first call got no answer, and send a new key to run the request again (it is billed again) or after an error answer. idempotency_conflict: the key came with a different body; use a new key for each distinct request. Neither runs the request again.
429
Providers are throttling, or the model is at capacity (unavailable_route): retry after Retry-After seconds. Or a free offer's allowance is used (free_allowance_exceeded): your organization's tokens, or your key's requests per minute. Choose a paid model, or for the per-minute limit retry after Retry-After seconds.

Every error code