Best Model for This
Costs assume ~2K tokens context + ~300 tokens output per query.
Quick Start
Tips & Best Practices
- Use
gemini-2.5-flashfor large documents — its 1M token context window can hold an entire codebase or book. Skip chunking for smaller corpora. - Enable prompt caching for repeated context — if the same document is queried multiple times, caching cuts input cost by 90%. See Prompt Caching.
- Be explicit about citation style — asking the model to cite
[1],[2]reduces hallucination and makes answers verifiable. - Instruct the model to say “I don’t know” — without this, models will confabulate answers from training data even when context is insufficient.
Cost Estimate
Large context = most of your cost. Use prompt caching if the same document appears in many queries.
Next Steps
- Prompt Caching — up to 90% discount on repeated context
- Streaming — stream RAG answers for faster perceived latency
- Models — compare context windows across all active models