Best Model for This
Costs assume ~300 tokens input + ~200 tokens output per message exchange.
Quick Start
Tips & Best Practices
- Stream always — users perceive streamed responses as 3-5x faster even at the same token speed.
- Cap history length — trim
messagesto the last 10-20 turns or ~4K tokens to keep latency low and costs predictable. - System prompt sets personality — define tone, scope, and any constraints in the first system message.
- Use
qwen-3.6-plusfor quality,llama-3.3-70bfor speed — swap the model string; the code is identical.
Cost Estimate
Assumes 300 tokens input + 200 tokens output per exchange. Longer conversations cost more.
Next Steps
- Streaming — deeper streaming patterns
- Model Aliases — use
best,fast,balancedshortcuts - Prompt Caching — cache system prompts to cut costs by up to 90%