# Tools and structured output

Tools let a model ask your code to run a function; structured output makes it answer in JSON that matches your schema. Both work on Chat Completions, Responses and Messages, each in its own shape.

## Which models

A model takes tools and structured output when its entry in `GET https://api.llm.buzz/api/v1/models` lists `tools`, `tool_choice`, `response_format` and `structured_outputs` in `supported_parameters`. Most text models do today. Before a provider serves a model, an automated check sends it real requests for tool calls and structured output where the model declares them, and a provider that fails stays out of routing: [Which providers serve](models-and-routing.md#which-providers-serve).

## Tools in each API

You declare the tools, the model asks for calls, and you run them and send the results back in the next request. Tool descriptions reach the provider unchanged and have no length limit of their own; the 64 MiB request limit and the model's context window still apply.

| | Chat Completions | Responses | Messages |
|---|---|---|---|
| You declare | `tools`, each `{"type": "function", "function": {...}}` with `name`, `description` and `parameters` | `tools`, each `{"type": "function", ...}` with `name`, `description`, `parameters` and `strict` | `tools`, each with `name`, `description` and `input_schema` |
| The model asks | `message.tool_calls`, each with an `id` and `function.arguments`, a JSON string; `finish_reason` is `tool_calls` | `function_call` items in `output`, each with a `call_id` and `arguments` | `tool_use` blocks in `content`, each with an `id` and `input`, an object; `stop_reason` is `tool_use` |
| You answer | A `tool` message with the call's `tool_call_id` | A `function_call_output` item with the call's `call_id` | A `tool_result` block with the call's `tool_use_id`, in a user message |

A Chat Completions loop on `deepseek/deepseek-v4.1-flash` asks, runs what the model asked for, and asks again until the model answers in text:

```python tab="Python"
import json
import os

from openai import OpenAI

client = OpenAI(
    base_url="https://api.llm.buzz/api/v1",
    api_key=os.environ["LLM_BUZZ_KEY"],
)
stock_schema = {
    "type": "object",
    "properties": {"sku": {"type": "string"}},
    "required": ["sku"],
    "additionalProperties": False,
}
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_stock_level",
            "description": "Return how many units of one SKU are in the warehouse.",
            "parameters": stock_schema,
        },
    }
]


def get_stock_level(sku: str) -> str:
    """Return the stock level of one SKU."""
    return {"KB-204": "37 units"}.get(sku, "unknown SKU")


messages = [{"role": "user", "content": "How many units of KB-204 are in stock?"}]
while True:
    reply = client.chat.completions.create(
        model="deepseek/deepseek-v4.1-flash", messages=messages, tools=tools
    )
    turn = reply.choices[0].message
    if not turn.tool_calls:
        print(turn.content)
        break
    messages.append(
        {
            "role": "assistant",
            "content": turn.content,
            "tool_calls": [call.model_dump() for call in turn.tool_calls],
        }
    )
    for call in turn.tool_calls:
        result = get_stock_level(**json.loads(call.function.arguments))
        messages.append({"role": "tool", "tool_call_id": call.id, "content": result})
```

```ts tab="TypeScript"
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llm.buzz/api/v1",
  apiKey: process.env.LLM_BUZZ_KEY,
});
const tools: OpenAI.Chat.Completions.ChatCompletionTool[] = [
  {
    type: "function",
    function: {
      name: "get_stock_level",
      description: "Return how many units of one SKU are in the warehouse.",
      parameters: {
        type: "object",
        properties: { sku: { type: "string" } },
        required: ["sku"],
        additionalProperties: false,
      },
    },
  },
];
const stock: Record<string, string> = { "KB-204": "37 units" };

const messages: OpenAI.Chat.Completions.ChatCompletionMessageParam[] = [
  { role: "user", content: "How many units of KB-204 are in stock?" },
];
while (true) {
  const reply = await client.chat.completions.create({
    model: "deepseek/deepseek-v4.1-flash",
    messages,
    tools,
  });
  const turn = reply.choices[0].message;
  if (!turn.tool_calls?.length) {
    console.log(turn.content);
    break;
  }
  messages.push({ role: "assistant", content: turn.content, tool_calls: turn.tool_calls });
  for (const call of turn.tool_calls) {
    if (call.type !== "function") continue;
    const { sku } = JSON.parse(call.function.arguments);
    messages.push({ role: "tool", tool_call_id: call.id, content: stock[sku] ?? "unknown SKU" });
  }
}
```

The same exchange on the other two APIs, with the `client`, `stock_schema` and `get_stock_level` above. Responses sends the results with `previous_response_id`, so the follow-up carries only them. Each snippet sends the follow-up only when the model asked for a call, because a request with no results to send is a 400:

```python tab="Responses"
tools = [
    {
        "type": "function",
        "name": "get_stock_level",
        "description": "Return how many units of one SKU are in the warehouse.",
        "parameters": stock_schema,
        "strict": True,
    }
]
response = client.responses.create(
    model="openai/gpt-5.6-luna", input="How many units of KB-204 are in stock?", tools=tools
)
results = [
    {
        "type": "function_call_output",
        "call_id": item.call_id,
        "output": get_stock_level(**json.loads(item.arguments)),
    }
    for item in response.output
    if item.type == "function_call"
]
if results:
    response = client.responses.create(
        model="openai/gpt-5.6-luna",
        previous_response_id=response.id,
        input=results,
        tools=tools,
    )
print(response.output_text)
```

```python tab="Messages"
from anthropic import Anthropic

claude = Anthropic(
    base_url="https://api.llm.buzz/api",
    api_key=os.environ["LLM_BUZZ_KEY"],
)
tools = [
    {
        "name": "get_stock_level",
        "description": "Return how many units of one SKU are in the warehouse.",
        "input_schema": stock_schema,
    }
]
messages = [{"role": "user", "content": "How many units of KB-204 are in stock?"}]
reply = claude.messages.create(
    model="anthropic/claude-sonnet-5.5", max_tokens=2048, tools=tools, messages=messages
)
results = [
    {
        "type": "tool_result",
        "tool_use_id": block.id,
        "content": get_stock_level(**block.input),
    }
    for block in reply.content
    if block.type == "tool_use"
]
if results:
    messages.append({"role": "assistant", "content": reply.content})
    messages.append({"role": "user", "content": results})
    reply = claude.messages.create(
        model="anthropic/claude-sonnet-5.5", max_tokens=2048, tools=tools, messages=messages
    )
print("".join(block.text for block in reply.content if block.type == "text"))
```

A Messages assistant turn goes back with every block it came with, `thinking` blocks included ([Messages](messages.md#the-answer)). A Responses follow-up that does not use `previous_response_id` sends the earlier items in `input` instead ([Input items](responses.md#input-items)). The Chat Completions loop above sends back only `content` and `tool_calls`. A reasoning model's tool turn may also carry `reasoning_content`, and sending it back keeps the model's thinking across the call: [Keep reasoning in the conversation](reasoning.md#keep-reasoning-in-the-conversation).

## Choosing the tool

`tool_choice` takes `auto`, which lets the model decide; `none`, which allows no call; `required`, which makes it call some tool; or one named function, which makes it call that one. Each API spells them its own way:

| | Chat Completions | Responses | Messages |
|---|---|---|---|
| Let the model decide | `"auto"` | `"auto"` | `{"type": "auto"}` |
| No call | `"none"` | `"none"` | `{"type": "none"}` |
| Some tool | `"required"` | `"required"` | `{"type": "any"}` |
| A named function | `{"type": "function", "function": {"name": "get_stock_level"}}` | `{"type": "function", "name": "get_stock_level"}` | `{"type": "tool", "name": "get_stock_level"}` |

Some models cannot be forced:

- **Claude Opus 5.5, Sonnet 5.5 and Fable 5.1** take no forced choice from any provider. llm.buzz serves `required` or a named function as `auto` and names it `tool_choice->auto` in the body's `x-sgr-ignored-parameters`, and the model may then answer without calling a tool. When the result must fit a schema, use `auto` with strict tools or structured output.
- **Moonshot AI's own API** takes no named function. llm.buzz sends a named choice to another provider of the model when one takes it, and otherwise serves the request as `auto`, named `tool_choice->auto`. `required` reaches Moonshot AI as sent and still forces a call, so with one tool that must be called, send `required`.

## Parallel calls

A model may ask for several tools in one turn. `parallel_tool_calls: false` on Chat Completions and Responses asks for one at a time, and llm.buzz honors it on every provider:

- A provider that has the switch receives it.
- On a provider that has none, llm.buzz enforces `false` itself. It returns the first call of a turn, drops the later ones, and names it `parallel_tool_calls->emulated(serialized_by_gateway)`. The model sees one result and asks for the remaining calls on the next turn.
- On such a provider `true` is the provider's own behavior, and `x-sgr-ignored-parameters` names it `parallel_tool_calls->dropped(provider_default)`.

Messages has no `parallel_tool_calls` field: one you send is dropped and named.

## Strict tools

`strict: true` asks the provider to hold the model to your schema: `function.strict` on Chat Completions and `strict` on a Responses function tool or a Messages tool. llm.buzz sends it to providers that enforce it. When no provider that may serve the request enforces it, llm.buzz serves the tool without `strict`, your schema unchanged, and names it `tools.strict->false`.

On Responses, a function tool sent without `strict` is not strict: llm.buzz sends it to the provider as `strict: false`, where OpenAI's own Responses API defaults to `true`. Send `strict: true` for strict schema adherence, as the Responses example above does.

- Claude's strict mode cannot express every JSON Schema keyword, among them `maxItems`, `oneOf`, `minimum`, some `format` values and a recursive `$ref`. A strict tool that uses one goes to a provider that can enforce it. When none can, llm.buzz drops `strict` and keeps the schema as you wrote it, never removing a keyword, and names the drop `tools.strict->false`.
- Claude requires `additionalProperties: false` on every object of a strict schema. llm.buzz closes the objects of a schema sent to Claude and names it `tools.parameters.additionalProperties->false`.

## IDs, arguments and tool results

- **IDs.** On Chat Completions and Responses a tool call's ID is an opaque string of 1 to 65,536 characters. Send each one back complete, signature suffix included, in the assistant call and in its result. On Messages the result names the call by the `id` of its `tool_use` block.
- **Streamed arguments.** The fragments of a streamed call add up, in order, to one JSON object. Join them and parse the result once the call is complete. On a Chat Completions stream llm.buzz stops relaying a call's arguments where that object closes, so whatever a provider sends after it never reaches you.
- **Arguments cut off.** If the output limit ends a call's arguments in the middle, llm.buzz drops that call and ends the turn as cut by the limit: `finish_reason: "length"` on Chat Completions, `status: "incomplete"` on Responses, `stop_reason: "max_tokens"` on Messages. Raise the output limit and ask again. llm.buzz does not repair the arguments or retry the turn, and you must not run a partial call from an answer the limit ended. On a stream, the fragments you already received end with the cut outcome: do not run them.
- **Check what you receive.** A finished call is JSON, but llm.buzz does not promise that it satisfies every constraint of your schema. Validate the arguments before you run the function.
- **Images in results.** A tool can return an image, such as a screenshot. A Chat Completions `tool` message takes `image_url` parts beside its text, a Responses `function_call_output` takes a list of text and image parts, and a Messages `tool_result` holds text and image blocks. All three APIs refuse any other kind of content in a tool result, a Messages `document` block included, with a 400 that names the message, item or block. Where a provider's tool results take only text, llm.buzz moves the images into one user message that follows the results and names it `messages.content.tool_result.image->following_user_message`. A provider that takes no image input at all gets placeholder text where the image was, named `messages.content.tool_result.image->placeholder`. [Images in requests](images-in-requests.md) covers images in the rest of a request.

## Structured output

Ask for JSON in the field your API has for it:

| API | How to ask |
|---|---|
| Chat Completions | `response_format` with `type: "json_schema"` for your schema, or `type: "json_object"` for any JSON object. |
| Responses | `text.format` with `type: "json_schema"` for your schema. `type: "json_object"` is a 400 `invalid_parameter` naming `text.format.type`: use `json_schema`. |
| Messages | Anthropic's `output_config`, with its `format` for a schema. An `output_config` that holds more than `effort` is sent as you wrote it when every provider that may serve the request takes Anthropic's API. Otherwise llm.buzz drops it and names `output_config` in `x-sgr-ignored-parameters`, and an `effort` in it still applies. Chat's `response_format` is not applied on Messages: it is dropped and named. |

The examples below reuse `client` and `json` from the tool loop.

```python tab="Chat Completions"
schema = {
    "type": "object",
    "properties": {"city": {"type": "string"}, "year": {"type": "integer"}},
    "required": ["city", "year"],
    "additionalProperties": False,
}
reply = client.chat.completions.create(
    model="deepseek/deepseek-v4.1-flash",
    messages=[{"role": "user", "content": "Lisbon hosted the world expo in 1998. Extract the city and the year."}],
    response_format={
        "type": "json_schema",
        "json_schema": {"name": "place_and_year", "strict": True, "schema": schema},
    },
)
data = json.loads(reply.choices[0].message.content)
```

```python tab="Responses"
response = client.responses.create(
    model="deepseek/deepseek-v4.1-flash",
    input="Lisbon hosted the world expo in 1998. Extract the city and the year.",
    text={
        "format": {
            "type": "json_schema",
            "name": "place_and_year",
            "strict": True,
            "schema": schema,
        }
    },
)
data = json.loads(response.output_text)
```

- **`json_schema` is the one to use when the shape matters.** `json_object` asks for valid JSON and nothing more. Each provider uses its own JSON mode and also gets a short instruction asking for JSON. Claude has no JSON mode and gets the instruction alone, which `x-sgr-ignored-parameters` names `response_format->instruction(json_object)`. llm.buzz sends no empty schema in its place.
- **Claude closes the objects.** Claude requires `additionalProperties: false` on every object of a schema. On Chat Completions and Responses, llm.buzz closes the objects of a strict schema sent to Claude, and `x-sgr-ignored-parameters` says so. On Messages, `output_config` is not rewritten: it reaches Anthropic as you wrote it, under the rule in the table above. Write the schema closed and the two stay the same.
- **A cut answer is cut.** An answer that hits the output limit ends with `finish_reason: "length"` on Chat Completions, `status: "incomplete"` on Responses or `stop_reason: "max_tokens"` on Messages, and may be unfinished JSON. Raise the limit and ask again. llm.buzz does not run OpenRouter's `response-healing` plugin: on Chat Completions and Responses it is dropped and named `plugins.response-healing`, and Messages drops and names `plugins` whole.

## Tools llm.buzz runs

llm.buzz runs one kind of tool for you, **tool search**. Mark most tools `defer_loading: true` and declare a tool search, and the model sees only the other tools and a search. The tools a search finds join the request. [Tool search](api-reference.md#tool-search) has each API's spelling, and `cost` includes the billed search rounds.

Web search is not available on llm.buzz today. Chat Completions and Responses accept `:online` and the `web` plugin, and Chat Completions also `web_search_options`: the request is served without a search, with nothing billed for one, and `x-sgr-ignored-parameters` names `web_search->dropped(search_unavailable)`. For an organization whose requests llm.buzz keeps in the EU such a request answers 400 `web_search_unavailable_in_region`. Messages drops `:online`, `plugins` and `web_search_options` and names them. [Coming from OpenRouter](from-openrouter.md) lists the other tools OpenRouter runs that llm.buzz does not.

## Related

- [Chat Completions](chat-completions.md), [Responses](responses.md) and [Messages](messages.md) list every request field of their API.
- [Streaming](streaming.md) lists the events of a stream, the ones that carry tool calls included.