Three RAG patterns I keep coming back to
Not every retrieval problem needs a vector database. A look at three patterns that solve 80% of the cases I see in production banking work.
After two-plus years building RAG pipelines for enterprise banking, I notice myself reaching for the same three patterns over and over. None are exotic. All beat naïve "embed everything, top-k, stuff into context" by a wide margin.
1. Hybrid retrieval is a requirement, not an optimization
Pure dense retrieval misses identifiers. Pure BM25 misses paraphrases. Real questions are a mix: "What did Account 472911 do in Q3?" needs both an exact match (the account number) and semantic understanding (Q3 ≈ third quarter ≈ July–September).
The simplest hybrid that works: run both, normalize scores, and weighted-sum. We found 0.6 dense + 0.4 lexical to be a reasonable starting point for Turkish banking text. Tune the weights per corpus, not globally.
2. Query rewriting before retrieval, not after
The user's question is rarely the right query. It has pronouns, follow-ups, and assumptions inherited from earlier turns. A small LLM pass that rewrites the question into a self-contained query — given the conversation so far — improves recall by something like 15–20% in our internal evals.
Crucially: do this before you embed, not after. Re-ranking after retrieval can't recover documents the embedding never saw.
3. Don't grow context, grow precision
The temptation is always to retrieve more. Top-20, top-50, "just give the model everything." This works in toy demos and breaks in production: irrelevant chunks are not free — they introduce noise the model has to actively suppress, and they push the relevant content past the cliff where attention degrades.
The fix is unglamorous: a re-ranker. We use a cross-encoder over the top 30 → keep top 5 → done. The latency cost is 100–200ms; the quality gain is the difference between a tool people trust and one they stop using.
What we don't do
- No "agentic retrieval" for read-only Q&A. It's overkill. Save the agent loop for tasks that genuinely need state — multi-step bookings, cross-system reconciliation, anything where one retrieval can't answer the question.
- No fine-tuning the embedder unless you've measured a domain mismatch. OpenAI / Cohere / Voyage embeddings handle 95% of cases. Fine-tuning is a long detour with maintenance cost.
- No dropping the lexical index. Even if dense retrieval looks "better" on average, the failure modes of lexical-only and dense-only are different. Hybrid is insurance, not duplication.
The boring lesson: most RAG quality comes from the unsexy parts. Chunking strategy, query rewriting, re-ranker. The model itself is rarely the bottleneck.