Prompt Caching

Prompt caching reuses context a model has already processed. When you send requests with the same system prompt, tool definitions, or conversation history, the repeated part usually costs less than fresh input and the model starts answering sooner.

See real-time cache stats across all models on the Rankings page.

How It Works

First request: Full prompt is processed and cached by the provider
Subsequent requests: the cached prefix is not processed again — it is billed at that model’s cached rate, and the model starts answering sooner

Not every model caches, and the rules come from each model’s creator. Most models that cache do it automatically once a prompt prefix repeats; Claude models cache the prefix you mark with cache_control. Check supports_caching and pricing.cached on /v1/models for the model you use. Live cache hit rates per model are on kymaapi.com/rankings?tab=cache.

Automatic Caching

On models that cache automatically, place static content (system prompt, tool definitions) at the beginning so the repeated prefix is identical on every request:

from openai import OpenAI

client = OpenAI(
    base_url="https://api.kymaapi.com/v1",
    api_key="kyma-your-api-key"
)

response = client.chat.completions.create(
    model="deepseek-v4-flash",  # supports caching and publishes a cached rate
    messages=[
        # Static content (can be cached after the first request)
        {"role": "system", "content": "Your long system prompt here..."},
        # Dynamic content (never cached)
        {"role": "user", "content": "User's question"}
    ]
)

# Cache stats in the response
details = response.usage.prompt_tokens_details
print(f"Cached: {details.cached_tokens if details and details.cached_tokens else 0}")
print(f"Cost: ${response.usage.cost}")
print(f"Saved: ${response.usage.cache_discount}")

Marking a prefix with `cache_control`

Claude models cache what you mark. Send the system prompt as content blocks and put cache_control on the last block of the stable prefix. Kyma forwards the marker to Claude models and removes it on routes that do not accept it, so sending it to another model does no harm.

response = client.chat.completions.create(
    model="claude-sonnet-5",
    messages=[
        {
            "role": "system",
            "content": [
                {"type": "text", "text": "Your long system prompt here...",
                 "cache_control": {"type": "ephemeral"}}
            ],
        },
        {"role": "user", "content": "User's question"},
    ],
)

The same cache_control field works on system blocks in /v1/messages, which is how the Anthropic SDK and Claude Code send it. Claude models are outside the free tier and need a top-up.

Best Practices

Structure prompts for caching

Place stable content first, dynamic content last:

System instructions (static) ← CACHED
Tool definitions (static)    ← CACHED
Few-shot examples (static)   ← CACHED
Conversation history          ← CACHED (grows incrementally)
Current user message          ← NOT CACHED (changes each request)

For coding agents

Coding agents send the same system prompt + tool definitions with every request, which is exactly the shape caching is for. The saving depends on the model you pick: see What the saving depends on below for the arithmetic.

What to avoid

Don’t put timestamps or request IDs in system prompts — breaks cache
Don’t reorder tool definitions between requests
Keep system prompt identical across requests

Cache Stats in Response

On /v1/chat/completions, Kyma adds three fields to usage and passes the model’s own cache counters through:

{
  "usage": {
    "prompt_tokens": 5050,
    "completion_tokens": 200,
    "prompt_tokens_details": { "cached_tokens": 5000 },
    "cached_tokens": 5000,
    "cost": 0.000382,
    "cache_discount": 0.002430
  }
}

Field	Description
`prompt_tokens_details.cached_tokens`	Tokens read from cache, when the model reports them. Absent on models that do not
`cached_tokens`	Tokens read from cache that this request was priced with, added by Kyma; `0` when nothing was cached
`cost`	Total cost charged for this request (USD), added by Kyma
`cache_discount`	Amount saved from caching (USD), added by Kyma; `0` when nothing was cached, and also `0` on a model whose cached rate is not below its input rate

In a stream, cost and cached_tokens are on the final usage chunk, and cache_discount appears there only when tokens were cached. Read prompt_tokens_details defensively: its presence depends on the model. On /v1/messages, usage follows Anthropic’s shape with input_tokens and output_tokens only; non-streamed responses carry the cost in the X-Kyma-Cost-USD header.

Tracking Your Savings

Per-request

Every /v1/chat/completions response includes usage.cost (what you paid) and usage.cache_discount (what you saved). Sum these over your session to track total savings. For /v1/messages and for any request after the fact, use the usage page or GET /v1/auth/usage/logs.

Community-wide

Visit the Cache Stats rankings to see:

Overall cache hit rate across all Kyma users
Per-model cache breakdown (cached vs uncached vs output tokens)
Total community savings in USD

Supported Models

Not every model supports prompt caching, and among those that do the cached rate is a price of its own — not a fixed fraction of the input rate. Every chat completion carries usage.cost and usage.cache_discount; what you do not get is a single discount percentage, because there isn’t one. Read the rate per model from /v1/models or the pricing page. pricing.cached is null when no cached rate is published for that model. Check which models are actively caching:

curl https://api.kymaapi.com/v1/models | jq '.data[] | {id, supports_caching}'

Pricing

Each model publishes its own cached rate, beside its input rate. There is no single Kyma-wide cache price.

Token type	Rate
Input (non-cached)	The model’s input rate
Input (cached)	The model’s cached rate — `pricing.cached` on `/v1/models`, `null` where the model publishes none
Output	The model’s output rate

There is no catalogue-wide discount, and any page that quotes one is rounding a range to a slogan. On most models Kyma publishes a cached rate for, it lies between about 5% and 40% of the input rate, and it differs from model to model, so always compare the two numbers for the model you use. A model can also publish a cached rate at or above its input rate, and then a cache hit saves nothing — usage.cache_discount reports 0 and the full price is in usage.cost. Some models publish no cached rate at all, and those send pricing.cached: null rather than a number nobody stated.

`usage.cost` and `usage.cache_discount` answer different questions

usage.cost is what the request was charged. When the route that served it reports what the request cost, that report is what the charge is built from, not any rate in the catalogue — see Routing and pricing. usage.cache_discount is a reported saving. It is computed from the model’s published cached rate, or from a fallback rate when the model publishes none. On a model that caches but publishes no cached rate, those two can disagree: cache_discount can show a saving the charge did not have. qwen-3.6-plus (alias best, supports_caching: true, pricing.cached: null on the live catalogue today) is the case most readers meet; llama-3.3-70b, qwen-3-32b, gpt-oss-120b, minimax-m2.5, sonar and sonar-pro are in the same position. Budget from usage.cost, and read cache_discount as an indicator that a prefix was reused, not as money off that request.

# What a cached token costs on each model, next to what a fresh one costs
curl -s https://api.kymaapi.com/v1/models \
  | jq -r '.data[] | select(.pricing.cached != null)
           | [.id, .pricing.input, .pricing.cached] | @tsv'

What the saving depends on Take a 50-request coding session with a 5,000-token system prompt that never changes and ~500 new tokens per turn. The first request pays the input rate on the whole prefix; the other 49 pay the cached rate on it.

input cost without caching = 50 × 5,000 × input_rate
input cost with caching    =  1 × 5,000 × input_rate
                           + 49 × 5,000 × cached_rate

So the saving is 49 × 5,000 × (input_rate − cached_rate), and it is entirely governed by the gap between those two numbers for the model you chose. On a model whose cached rate is a tenth of input, that is close to a 90% cut on the repeated portion. On a model whose cached rate equals its input rate, it is nothing. Look the two rates up before you design around the cheaper case. The usage.cost and usage.cache_discount fields on every chat completion let you track savings in real time.

Guides

Use Cases

Integrations

How It Works

Automatic Caching

Marking a prefix with `cache_control`

Best Practices

Structure prompts for caching

For coding agents

What to avoid

Cache Stats in Response

Tracking Your Savings

Per-request

Community-wide

Supported Models

Pricing

`usage.cost` and `usage.cache_discount` answer different questions

​How It Works

​Automatic Caching

​Marking a prefix with cache_control

​Best Practices

​Structure prompts for caching

​For coding agents

​What to avoid

​Cache Stats in Response

​Tracking Your Savings

​Per-request

​Community-wide

​Supported Models

​Pricing

​usage.cost and usage.cache_discount answer different questions

How It Works

Automatic Caching

Marking a prefix with `cache_control`

Best Practices

Structure prompts for caching

For coding agents

What to avoid

Cache Stats in Response

Tracking Your Savings

Per-request

Community-wide

Supported Models

Pricing

`usage.cost` and `usage.cache_discount` answer different questions