Skip to main content

Best Model for This

Costs assume ~300 tokens input + ~200 tokens output per message exchange.

Quick Start

Tips & Best Practices

  • Stream always — users perceive streamed responses as 3-5x faster even at the same token speed.
  • Cap history length — trim messages to the last 10-20 turns or ~4K tokens to keep latency low and costs predictable.
  • System prompt sets personality — define tone, scope, and any constraints in the first system message.
  • Use qwen-3.6-plus for quality, llama-3.3-70b for speed — swap the model string; the code is identical.

Cost Estimate

Assumes 300 tokens input + 200 tokens output per exchange. Longer conversations cost more.

Next Steps