Skip to main content

Best Model for This

Costs assume ~2K tokens context + ~300 tokens output per query.

Quick Start

Tips & Best Practices

  • Use gemini-2.5-flash for large documents — its 1M token context window can hold an entire codebase or book. Skip chunking for smaller corpora.
  • Enable prompt caching for repeated context — if the same document is queried multiple times, caching cuts input cost by 90%. See Prompt Caching.
  • Be explicit about citation style — asking the model to cite [1], [2] reduces hallucination and makes answers verifiable.
  • Instruct the model to say “I don’t know” — without this, models will confabulate answers from training data even when context is insufficient.

Cost Estimate

Large context = most of your cost. Use prompt caching if the same document appears in many queries.

Next Steps

  • Prompt Caching — up to 90% discount on repeated context
  • Streaming — stream RAG answers for faster perceived latency
  • Models — compare context windows across all active models