Tools and structured output

Which models

A model takes tools and structured output when its entry in GET https://api.llm.buzz/api/v1/models lists tools, tool_choice, response_format and structured_outputs in supported_parameters. Most text models do today. Before a provider serves a model, an automated check sends it real requests for tool calls and structured output where the model declares them, and a provider that fails stays out of routing: Which providers serve.

Tools in each API

You declare the tools, the model asks for calls, and you run them and send the results back in the next request. Tool descriptions reach the provider unchanged and have no length limit of their own; the 64 MiB request limit and the model's context window still apply.

A Chat Completions loop on deepseek/deepseek-v4.1-flash asks, runs what the model asked for, and asks again until the model answers in text:

import json
import os

from openai import OpenAI

client = OpenAI(
    base_url="https://api.llm.buzz/api/v1",
    api_key=os.environ["LLM_BUZZ_KEY"],
)
stock_schema = {
    "type": "object",
    "properties": {"sku": {"type": "string"}},
    "required": ["sku"],
    "additionalProperties": False,
}
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_stock_level",
            "description": "Return how many units of one SKU are in the warehouse.",
            "parameters": stock_schema,
        },
    }
]


def get_stock_level(sku: str) -> str:
    """Return the stock level of one SKU."""
    return {"KB-204": "37 units"}.get(sku, "unknown SKU")


messages = [{"role": "user", "content": "How many units of KB-204 are in stock?"}]
while True:
    reply = client.chat.completions.create(
        model="deepseek/deepseek-v4.1-flash", messages=messages, tools=tools
    )
    turn = reply.choices[0].message
    if not turn.tool_calls:
        print(turn.content)
        break
    messages.append(
        {
            "role": "assistant",
            "content": turn.content,
            "tool_calls": [call.model_dump() for call in turn.tool_calls],
        }
    )
    for call in turn.tool_calls:
        result = get_stock_level(**json.loads(call.function.arguments))
        messages.append({"role": "tool", "tool_call_id": call.id, "content": result})

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llm.buzz/api/v1",
  apiKey: process.env.LLM_BUZZ_KEY,
});
const tools: OpenAI.Chat.Completions.ChatCompletionTool[] = [
  {
    type: "function",
    function: {
      name: "get_stock_level",
      description: "Return how many units of one SKU are in the warehouse.",
      parameters: {
        type: "object",
        properties: { sku: { type: "string" } },
        required: ["sku"],
        additionalProperties: false,
      },
    },
  },
];
const stock: Record<string, string> = { "KB-204": "37 units" };

const messages: OpenAI.Chat.Completions.ChatCompletionMessageParam[] = [
  { role: "user", content: "How many units of KB-204 are in stock?" },
];
while (true) {
  const reply = await client.chat.completions.create({
    model: "deepseek/deepseek-v4.1-flash",
    messages,
    tools,
  });
  const turn = reply.choices[0].message;
  if (!turn.tool_calls?.length) {
    console.log(turn.content);
    break;
  }
  messages.push({ role: "assistant", content: turn.content, tool_calls: turn.tool_calls });
  for (const call of turn.tool_calls) {
    if (call.type !== "function") continue;
    const { sku } = JSON.parse(call.function.arguments);
    messages.push({ role: "tool", tool_call_id: call.id, content: stock[sku] ?? "unknown SKU" });
  }
}

The same exchange on the other two APIs, with the client, stock_schema and get_stock_level above. Responses sends the results with previous_response_id, so the follow-up carries only them. Each snippet sends the follow-up only when the model asked for a call, because a request with no results to send is a 400:

tools = [
    {
        "type": "function",
        "name": "get_stock_level",
        "description": "Return how many units of one SKU are in the warehouse.",
        "parameters": stock_schema,
        "strict": True,
    }
]
response = client.responses.create(
    model="openai/gpt-5.6-luna", input="How many units of KB-204 are in stock?", tools=tools
)
results = [
    {
        "type": "function_call_output",
        "call_id": item.call_id,
        "output": get_stock_level(**json.loads(item.arguments)),
    }
    for item in response.output
    if item.type == "function_call"
]
if results:
    response = client.responses.create(
        model="openai/gpt-5.6-luna",
        previous_response_id=response.id,
        input=results,
        tools=tools,
    )
print(response.output_text)

from anthropic import Anthropic

claude = Anthropic(
    base_url="https://api.llm.buzz/api",
    api_key=os.environ["LLM_BUZZ_KEY"],
)
tools = [
    {
        "name": "get_stock_level",
        "description": "Return how many units of one SKU are in the warehouse.",
        "input_schema": stock_schema,
    }
]
messages = [{"role": "user", "content": "How many units of KB-204 are in stock?"}]
reply = claude.messages.create(
    model="anthropic/claude-sonnet-5.5", max_tokens=2048, tools=tools, messages=messages
)
results = [
    {
        "type": "tool_result",
        "tool_use_id": block.id,
        "content": get_stock_level(**block.input),
    }
    for block in reply.content
    if block.type == "tool_use"
]
if results:
    messages.append({"role": "assistant", "content": reply.content})
    messages.append({"role": "user", "content": results})
    reply = claude.messages.create(
        model="anthropic/claude-sonnet-5.5", max_tokens=2048, tools=tools, messages=messages
    )
print("".join(block.text for block in reply.content if block.type == "text"))

A Messages assistant turn goes back with every block it came with, thinking blocks included (Messages). A Responses follow-up that does not use previous_response_id sends the earlier items in input instead (Input items). The Chat Completions loop above sends back only content and tool_calls. A reasoning model's tool turn may also carry reasoning_content, and sending it back keeps the model's thinking across the call: Keep reasoning in the conversation.

Choosing the tool

tool_choice takes auto, which lets the model decide; none, which allows no call; required, which makes it call some tool; or one named function, which makes it call that one. Each API spells them its own way:

Some models cannot be forced:

• Claude Opus 5.5, Sonnet 5.5 and Fable 5.1 take no forced choice from any provider. llm.buzz serves required or a named function as auto and names it tool_choice->auto in the body's x-sgr-ignored-parameters, and the model may then answer without calling a tool. When the result must fit a schema, use auto with strict tools or structured output.
• Moonshot AI's own API takes no named function. llm.buzz sends a named choice to another provider of the model when one takes it, and otherwise serves the request as auto, named tool_choice->auto. required reaches Moonshot AI as sent and still forces a call, so with one tool that must be called, send required.

Parallel calls

A model may ask for several tools in one turn. parallel_tool_calls: false on Chat Completions and Responses asks for one at a time, and llm.buzz honors it on every provider:

• A provider that has the switch receives it.
• On a provider that has none, llm.buzz enforces false itself. It returns the first call of a turn, drops the later ones, and names it parallel_tool_calls->emulated(serialized_by_gateway). The model sees one result and asks for the remaining calls on the next turn.
• On such a provider true is the provider's own behavior, and x-sgr-ignored-parameters names it parallel_tool_calls->dropped(provider_default).

Messages has no parallel_tool_calls field: one you send is dropped and named.

Strict tools

strict: true asks the provider to hold the model to your schema: function.strict on Chat Completions and strict on a Responses function tool or a Messages tool. llm.buzz sends it to providers that enforce it. When no provider that may serve the request enforces it, llm.buzz serves the tool without strict, your schema unchanged, and names it tools.strict->false.

On Responses, a function tool sent without strict is not strict: llm.buzz sends it to the provider as strict: false, where OpenAI's own Responses API defaults to true. Send strict: true for strict schema adherence, as the Responses example above does.

• Claude's strict mode cannot express every JSON Schema keyword, among them maxItems, oneOf, minimum, some format values and a recursive $ref. A strict tool that uses one goes to a provider that can enforce it. When none can, llm.buzz drops strict and keeps the schema as you wrote it, never removing a keyword, and names the drop tools.strict->false.
• Claude requires additionalProperties: false on every object of a strict schema. llm.buzz closes the objects of a schema sent to Claude and names it tools.parameters.additionalProperties->false.

IDs, arguments and tool results

• IDs. On Chat Completions and Responses a tool call's ID is an opaque string of 1 to 65,536 characters. Send each one back complete, signature suffix included, in the assistant call and in its result. On Messages the result names the call by the id of its tool_use block.
• Streamed arguments. The fragments of a streamed call add up, in order, to one JSON object. Join them and parse the result once the call is complete. On a Chat Completions stream llm.buzz stops relaying a call's arguments where that object closes, so whatever a provider sends after it never reaches you.
• Arguments cut off. If the output limit ends a call's arguments in the middle, llm.buzz drops that call and ends the turn as cut by the limit: finish_reason: "length" on Chat Completions, status: "incomplete" on Responses, stop_reason: "max_tokens" on Messages. Raise the output limit and ask again. llm.buzz does not repair the arguments or retry the turn, and you must not run a partial call from an answer the limit ended. On a stream, the fragments you already received end with the cut outcome: do not run them.
• Check what you receive. A finished call is JSON, but llm.buzz does not promise that it satisfies every constraint of your schema. Validate the arguments before you run the function.
• Images in results. A tool can return an image, such as a screenshot. A Chat Completions tool message takes image_url parts beside its text, a Responses function_call_output takes a list of text and image parts, and a Messages tool_result holds text and image blocks. All three APIs refuse any other kind of content in a tool result, a Messages document block included, with a 400 that names the message, item or block. Where a provider's tool results take only text, llm.buzz moves the images into one user message that follows the results and names it messages.content.tool_result.image->following_user_message. A provider that takes no image input at all gets placeholder text where the image was, named messages.content.tool_result.image->placeholder. Images in requests covers images in the rest of a request.

Structured output

Ask for JSON in the field your API has for it:

The examples below reuse client and json from the tool loop.

schema = {
    "type": "object",
    "properties": {"city": {"type": "string"}, "year": {"type": "integer"}},
    "required": ["city", "year"],
    "additionalProperties": False,
}
reply = client.chat.completions.create(
    model="deepseek/deepseek-v4.1-flash",
    messages=[{"role": "user", "content": "Lisbon hosted the world expo in 1998. Extract the city and the year."}],
    response_format={
        "type": "json_schema",
        "json_schema": {"name": "place_and_year", "strict": True, "schema": schema},
    },
)
data = json.loads(reply.choices[0].message.content)

response = client.responses.create(
    model="deepseek/deepseek-v4.1-flash",
    input="Lisbon hosted the world expo in 1998. Extract the city and the year.",
    text={
        "format": {
            "type": "json_schema",
            "name": "place_and_year",
            "strict": True,
            "schema": schema,
        }
    },
)
data = json.loads(response.output_text)

• json_schema is the one to use when the shape matters. json_object asks for valid JSON and nothing more. Each provider uses its own JSON mode and also gets a short instruction asking for JSON. Claude has no JSON mode and gets the instruction alone, which x-sgr-ignored-parameters names response_format->instruction(json_object). llm.buzz sends no empty schema in its place.
• Claude closes the objects. Claude requires additionalProperties: false on every object of a schema. On Chat Completions and Responses, llm.buzz closes the objects of a strict schema sent to Claude, and x-sgr-ignored-parameters says so. On Messages, output_config is not rewritten: it reaches Anthropic as you wrote it, under the rule in the table above. Write the schema closed and the two stay the same.
• A cut answer is cut. An answer that hits the output limit ends with finish_reason: "length" on Chat Completions, status: "incomplete" on Responses or stop_reason: "max_tokens" on Messages, and may be unfinished JSON. Raise the limit and ask again. llm.buzz does not run OpenRouter's response-healing plugin: on Chat Completions and Responses it is dropped and named plugins.response-healing, and Messages drops and names plugins whole.

Tools llm.buzz runs

llm.buzz runs one kind of tool for you, tool search. Mark most tools defer_loading: true and declare a tool search, and the model sees only the other tools and a search. The tools a search finds join the request. Tool search has each API's spelling, and cost includes the billed search rounds.

Web search is not available on llm.buzz today. Chat Completions and Responses accept :online and the web plugin, and Chat Completions also web_search_options: the request is served without a search, with nothing billed for one, and x-sgr-ignored-parameters names web_search->dropped(search_unavailable). For an organization whose requests llm.buzz keeps in the EU such a request answers 400 web_search_unavailable_in_region. Messages drops :online, plugins and web_search_options and names them. Coming from OpenRouter lists the other tools OpenRouter runs that llm.buzz does not.

Related

• Chat Completions, Responses and Messages list every request field of their API.
• Streaming lists the events of a stream, the ones that carry tool calls included.