Skip to main content
A model on Kyma can usually be served more than one way. This page states which one is chosen, in what order, and what each costs you — because a gateway that does not say this is asking you to trust an average.

​
The rule, in one line

What you pay is derived from the specific infrastructure that served your request — not from an average, and not from a table we maintain. usage.cost on every response is the exact amount charged for that request. There is no second number anywhere.

​
Why a model has more than one price

The same model is often available from several operators, at different service tiers, in different regions. Those are genuinely different products with genuinely different costs, and the spread between the cheapest and dearest way to serve one model can be several-fold at identical context length. So “the price of a model” is not one number, and a gateway that publishes one is publishing an estimate. What we publish is the rate of the path that normally serves — the typical price, not a maximum.
Prices are computed from real serving costs. They can fall, and never rise without notice.

​
The order routes are tried

  1. The primary route, then its configured alternates.
  2. Within those, cheapest first. A route that costs more is tried later, not removed — it is what keeps the model answering when the cheap one is down.
  3. A route with no known rate sorts last. Unknown is not the same as cheap.
  4. Routes that would certainly fail this particular request are skipped entirely — for example a route without tool support when your request carries tools.
Requests that fail are not billed. You pay for the attempt that completed.

​
What a failover costs you

If the usual route is unavailable and a dearer one answers, that request costs more, and usage.cost says so. We used to cap the charge at the published price. It sounds generous and it was not: keeping that cap meant the router had to refuse the dearer routes, which left most multi-route models with a single supplier. A price ceiling that removes your failover is a worse deal than a request that occasionally costs more. If you would rather fail than pay above a limit, say so per request — see max_price below.

​
Guarding against a wrong number

A single request is never billed more than 3× the published price, whatever an upstream reports. It exists because a usage figure can arrive wrong — misreported upstream, or misread by us. Three times the published price sits far above any real difference between routes and far below a decimal-point error, so it clips mistakes without clipping an expensive but genuine one. A request that hits that stop is charged less than it cost to serve, and the model is investigated the same day.

​
Choosing your own limit

max_price is in design and not yet accepted by the API. It is documented here because the routing rules above are what it will build on.
The intent: a per-request ceiling you set, and when no route can serve under it the request fails with a clear error rather than being served at a price you did not agree to. Cost-sensitive batch work sets it; latency-sensitive interactive work does not. The choice is yours per request rather than ours for everyone.

​
What we do not do

  • We never substitute a different model. If you ask for a model, you get that model or an error. An answer from something else, under the name you asked for, is worse than a failure you can see.
  • We never hide which is which. usage.cost is the amount charged, always.