Part 6: Retrieval (RAG) — The AI Engineering Handbook

6.0intermediate
When you actually need RAG
RAG vs long context vs fine-tune as a cost decision: long context pays per query, RAG pays once to index. The default rule, the exceptions, and when RAG is the wrong answer.
10 min
6.1intermediate
Document processing and chunking
How parsing and chunk boundaries set your retrieval ceiling, why overlap and chunk size are eval questions, and the defaults that survive contact with real corpora.
15 min
6.2intermediate
Embeddings and vector search
How to pick an embedding model on your own data, why pgvector is the default store, what HNSW and IVFFlat actually do, and the honest threshold where a dedicated vector DB earns its keep.
15 min
6.3intermediate
Hybrid search and reranking
Why BM25 still beats vector search on exact terms, how to fuse them with RRF, and when a reranker earns its latency.
10 min
6.4intermediate
Query understanding
Rewriting, multi-query, decomposition, and routing: closing the gap between what users type and what your retriever can match.
15 min
6.5intermediate
Citations and grounding
Make RAG answers verifiable: span-level citations, refusal on empty retrieval, and the faithfulness vs fluency tension that decides both.
10 min
6.6advanced
Advanced retrieval
Three patterns that beat vanilla RAG on specific queries: agentic retrieval, GraphRAG, and vision-native PDF search. When each earns its complexity, and when it doesn't.
10 min
6.7intermediate
Evaluating retrieval
RAG eval is mostly retrieval eval. Recall@k and MRR in plain terms, building an eval set from real queries, and the 2x2 that separates retrieval failure from generation failure.
10 min
6.8advanced
Production RAG
The five operational concerns that turn a RAG prototype into a system you can run: incremental indexing, the reindex tax, multi-tenant isolation, metadata filtering, and per-stage latency budgets.
15 min