Skip to main content

Best Model for This

Costs assume ~2K tokens context + ~300 tokens output per query.

Quick Start

Tips & Best Practices

  • Use gemini-2.5-flash for large documents — its 1M token context window can hold an entire codebase or book. Skip chunking for smaller corpora.
  • Enable prompt caching for repeated context — if the same document is queried multiple times, the repeated part bills at the model’s cached-input rate. See Prompt Caching.
  • Be explicit about citation style — asking the model to cite [1], [2] reduces hallucination and makes answers verifiable.
  • Instruct the model to say “I don’t know” — without this, models will confabulate answers from training data even when context is insufficient.

Cost Estimate

Large context = most of your cost. Use prompt caching if the same document appears in many queries.

Next Steps

  • Prompt Caching — repeated context at the model’s cached rate
  • Streaming — stream RAG answers for faster perceived latency
  • Models — compare context windows across all active models