	In Browser
	StumbleUpon
	del.icio.us
	Google
	Google Buzz
	reddit
	LinkedIn

	Facebook
	Twitter
	Linkedin
	E-Mail

Generative AI > RAG Pipelines > Hybrid Search - Combining Keyword and Vector Search

Hybrid Search - Combining Keyword and Vector Search

Author: Venkata Sudhakar

Vector search excels at semantic matching - it finds documents with the same meaning even if they use different words. But it can miss exact keyword matches. If a user searches for "Debezium 2.4 release notes", pure vector search might return general CDC articles instead of the specific version document. BM25 keyword search (the algorithm behind traditional search engines) is the opposite: it reliably finds exact terms but misses paraphrases and synonyms. Hybrid search combines both to get the best of both worlds - semantic understanding plus exact term matching.

The standard approach is Reciprocal Rank Fusion (RRF). You run both a vector search and a BM25 keyword search in parallel, getting two ranked lists of documents. RRF combines these rankings using the formula score = 1/(k + rank) for each document in each list, where k is typically 60. A document that appears high in both lists gets a high combined score. Documents that only appear in one list still contribute, but less. This is simple to implement and requires no training data or thresholds.

The below example shows implementing hybrid search with RRF using ChromaDB for vector search and the rank_bm25 library for keyword search.

Running hybrid search with Reciprocal Rank Fusion,

def hybrid_search(query: str, top_k: int = 3, k_rrf: int = 60) -> list:
    # 1. Vector search
    q_emb = client.embeddings.create(model="text-embedding-3-small", input=query).data[0].embedding
    vec_results = coll.query(query_embeddings=[q_emb], n_results=len(docs))
    vec_ranked = {vec_results["ids"][0][i]: i+1 for i in range(len(docs))}

# 2. BM25 keyword search
    bm25_scores = bm25.get_scores(query.lower().split())
    bm25_ranked = {f"d{i}": rank+1
                   for rank, i in enumerate(sorted(range(len(docs)),
                       key=lambda x: bm25_scores[x], reverse=True))}

# 3. Reciprocal Rank Fusion - combine both rankings
    rrf_scores = {}
    for doc_id in [f"d{i}" for i in range(len(docs))]:
        vec_rank = vec_ranked.get(doc_id, len(docs)+1)
        bm25_rank = bm25_ranked.get(doc_id, len(docs)+1)
        rrf_scores[doc_id] = 1/(k_rrf + vec_rank) + 1/(k_rrf + bm25_rank)

# Sort by combined RRF score
    top = sorted(rrf_scores.items(), key=lambda x: x[1], reverse=True)[:top_k]

results = []
    for doc_id, score in top:
        idx = int(doc_id[1:])
        results.append({"doc": docs[idx], "score": round(score, 5),
                        "vec_rank": vec_ranked[doc_id],
                        "bm25_rank": bm25_ranked[doc_id]})
    return results

# Query with a specific version number - keyword search helps here
query = "Debezium 2.4 PostgreSQL 15"
results = hybrid_search(query)
for r in results:
    print(f"[RRF={r['score']}] vec_rank={r['vec_rank']} bm25_rank={r['bm25_rank']}")
    print(f"  {r['doc']}")

It gives the following output,

[RRF=0.03150] vec_rank=1 bm25_rank=1
  Debezium 2.4 adds native support for PostgreSQL 15 logical replication.

[RRF=0.01573] vec_rank=2 bm25_rank=3
  CDC with Debezium captures row-level changes by reading the MySQL binlog.

[RRF=0.01538] vec_rank=3 bm25_rank=5
  Apache Kafka stores messages in immutable, ordered, partitioned logs.

# "Debezium 2.4 adds native support for PostgreSQL 15" ranked 1st in BOTH searches
# -> highest RRF score by a wide margin
# Pure vector search alone might have ranked generic Debezium doc higher

When to use hybrid search: whenever your corpus contains proper nouns, product names, version numbers, acronyms, or technical identifiers that vector search might miss. If users search for "error code ORA-01000" or "model gpt-4o-mini", BM25 will find exact matches that vector search might not rank highly. Start with pure vector search for simplicity, then switch to hybrid search if you see retrieval misses on exact-term queries. The RRF weighting between keyword and vector can also be tuned by multiplying each score by a weight before combining.

Send your comments, suggestions or queries regarding this site to [email protected].