A market, not a gateway
A gateway gives you one key and one bill for many models. That part is useful, and llm.buzz does it too. What a gateway usually hides is who actually runs the model. You send deepseek/deepseek-v4.1-flash, someone serves it, and the price is whatever the gateway decided it is.
llm.buzz works the other way round. A model name is a slot that several providers can fill. Each provider posts its own prices for that model, the router compares them on every request, and the whole contest is on the model page for anyone to read. We call it a market because that is what it is: providers compete for the same request, in the open.
Why let providers compete
Open-weights models are the same model wherever they run. The weights are identical; what differs is the price, the speed, how often the endpoint is up, where it serves from and what the company does with your prompts. Those are exactly the things a buyer cares about, and they change from week to week.
When one provider sits behind a name, none of that is visible and none of it is negotiable. When several do, a provider that lowers its price or serves faster wins more requests the moment it does, without anyone renegotiating a contract. You keep calling the same model name. The router does the shopping.
How one request is ranked
Each model has a routing preset that orders its providers:
| Preset | Order |
|---|---|
smart | By price and time to first token together. Every new model starts on it. |
cheapest | By the estimated cost of this request at each provider’s price. |
fastest | By recent time to first token. |
order | In the order Surogate set. |
The cost is estimated per request, not per model: the input estimate times the provider’s input price, plus the output bound times its output price. A long prompt with a short answer can therefore rank providers differently from a short prompt with a long answer.
Speed is measured, not declared. Time to first token comes from real traffic, as a weighted mean per provider with a five-minute half-life, so a provider that slows down falls back in the order within minutes and climbs again when it recovers. Under smart, cost and time to first token are each scaled between the cheapest and the dearest, the fastest and the slowest, and averaged; the lowest score goes first. A provider with no fresh measurement takes the median of the measured ones, so it is neither punished nor favoured for being new.
The request goes to the first provider in that order. If it errors or times out, the next one answers, and you are billed at the price of the provider that served you. A model whose preset is order switches to smart when its first independent provider goes live, so the market starts working on a model as soon as there is a second seller in it.
Asking for a different trade
The preset is a default, not a rule. A suffix on the model name changes one request:
reply = client.chat.completions.create(
model="deepseek/deepseek-v4.1-flash:floor",
messages=[{"role": "user", "content": "Summarize this contract"}],
):floor puts the cheapest provider first and :nitro the fastest, by recent time to first token. The same choice is available as a field, which is easier when the model name comes from configuration:
extra_body={"provider": {"sort": "price"}}"price" works as :floor, "latency" and "throughput" as :nitro, and "smart" ranks by price and speed together.
Who the providers are
Some providers are cloud companies Surogate connects to. Others are independent providers: inference companies that sign up, apply once with their company details and the endpoints they serve from, and are approved by Surogate. For each endpoint they state whether it serves from the EU and declare their data policy, whether they train on prompts and how many days they keep them. The form has no default for either answer, because routing and customers rely on both.
An approved provider lists models from its endpoints, and each listing goes live by itself when an automated check of the endpoint passes. A live listing is one more provider of a public model, for every account, at the provider’s own prices. A listing that fails its check, or fails under real traffic, leaves routing by itself until it passes again. Providers never see customer data, and customers never see the providers’ own pages; what they share is the model page.
If you run inference and want to sell it here, the provider page explains how to apply.
What the model page shows
Every model page lists its providers side by side, one row each:
- the input, output and cache read prices, per million tokens;
- latency and throughput, and uptime over three days;
- whether the provider serves from the EU, and its declared precision when it is independent;
- its data policy in one line, such as “Trains on prompts: no; retention: 0 days”, or “Data policy not declared”.
The figures come from the public API, each with its window. When there is not enough traffic to publish a figure honestly, the page says so instead of showing a number. A provider’s policy is marked as declared by the provider or declared by Surogate, so you know whose statement you are reading.
The same rows are available to your code. GET https://api.llm.buzz/api/v1/models/{author}/{slug}/endpoints lists one model’s providers with their prices, limits and tag, the name you use to steer a request toward or away from a provider.
What this does not change
Your code does not change. You call one base URL with the OpenAI or Anthropic SDK and name a model. The market happens behind that name, and each request is billed at the price of the provider that actually served it.
The part we care about most is that the contest is public. The models list shows every model with its providers, and anyone can check that the cheapest row is really the cheapest, or that the fastest one is really fast, without an account.