Hybrid search: when BM25 beats embeddings
Embeddings miss exact codes, product names and section numbers that keyword search finds at once. Why my RAG systems combine BM25 and vector search.
Vector search finds passages that mean the same thing as the question. That is exactly what you want, until the question contains something that has no meaning, only a spelling: a procedure number, a chemical name, a form code.
Where embeddings fall short
Ask "what does procedure 4.2 say about confined spaces?" and an embedding model may return three passages about confined spaces from other procedures. The number that matters most to the user carries almost no weight in the vector.
Where keyword search falls short
BM25 finds "4.2" at once, but it misses a question that uses different words from the document: "tight spaces" instead of "confined spaces".
Combining them
I run both searches and merge the results with reciprocal rank fusion, which needs no tuning of score scales. On a set of real questions, I then check context recall for each method alone and for the combination before choosing the default.
Keep reading
All posts
How to evaluate a RAG system without fooling yourself
A high retrieval score can still hide wrong answers. The reference sets, metrics and release gate I use to know whether a RAG change made answers better.
Running open-weight LLMs on your own GPUs
When health data cannot leave the building, the model has to come to the data. What self-hosting open-weight LLMs costs, and what it buys you.
Human approval steps for LangGraph agents
An agent that sends reports or changes records should stop and ask first. How I build approval steps, retries and tracing into LangGraph workflows.