Skip to main content
Kyma’s job is to sit between your app and the best model for the task without becoming the slow part. For streaming workloads, where every 100ms is noticeable, here is how the connection works.

​
Edge-proxy: one hop, full control

Every request streams through Kyma’s edge: your client ↔ Kyma edge ↔ source. Opening a direct connection instead would mean handing a long-lived credential to your client — unsafe. The edge adds one network hop, and in exchange you keep:
  • Accurate billing — metered per character / token / minute as it streams, not estimated after the fact.
  • Automatic failover — if a route fails before it sends the first byte, Kyma tries the next route and your client sees one normal response. Once a response has started streaming, it is not switched to another route.
  • Usage visibility — the request lands in your logs and usage with real numbers.
A second, direct mode (Kyma mints a one-time token and your client streams straight to the source) was used only by realtime voice. Realtime audio is paused, so today every request takes the edge path.

​
Why the hop is the right trade

For almost every product, one hop is imperceptible next to the value of correct billing and reliability — which is why production voice platforms proxy too.

​
Progressive delivery

You never configure the connection. You can still opt into progressive delivery where it helps:
  • Chat streaming — pass "stream": true on /v1/chat/completions to receive tokens as they are generated. See Streaming.
  • TTS streaming — pass "stream": true on POST /v1/audio/speech to receive audio as it’s synthesized. The first audio arrives while the rest is still being synthesized, instead of after the whole clip. Same complete audio/mpeg result, just delivered progressively.

​
What this means for you

  • Billing and reliability are never traded away for speed.
  • You don’t manage connections, tokens, or failover — Kyma owns the connection model so your client stays simple.

​
See also