Hybrid document retrieval

Retrieval is the query-time counterpart of everything you did during indexing. Given a (possibly HyDE-rewritten) user question, RAG Me Up must find the chunk set most likely to ground a good answer. In this framework that always means hybrid retrieval over Postgres: dense similarity via pgvector and sparse keyword search via ParadeDB BM25 (pg_search).

There is currently no switch to run dense-only or sparse-only without changing code. Hybrid is the product decision.

From question to search inputs

Before any SQL runs, the question text (or HyDE draft) is embedded with the same SentenceTransformer used at index time:

prompt_embedding = self.embeddings.encode(prompt)
documents = self.handle_documents(prompt, prompt_embedding, datasets)

handle_documents calls PostgresHybridRetriever.get_relevant_documents, optionally reranks, and returns the chunk list later steps inject into the prompt.

Dataset filters ride along as metadata predicates. If the UI passes datasets=["hr", "kennisbank"], both legs only consider rows whose metadata->>'dataset' is in that list. An empty list means search everything.

Dense vector retrieval

Dense retrieval asks: which stored chunk embeddings are nearest to the query embedding under cosine distance?

In Postgres that is the <=> operator on an HNSW-indexed vector column:

SELECT id, content, metadata,
       embedding <=> %s::vector AS distance
FROM ragmeup_dense_embeddings
WHERE {dataset_filter}
ORDER BY distance
LIMIT %s;

What dense search is good at:

• Paraphrase and semantic match ("partner leave" ↔ "spousal parental leave entitlement").
• Queries that are full sentences rather than keywords.
• Crossing vocabulary gaps when the embedding model was trained for retrieval (see MTEB).

What dense search is bad at:

• Ultra-short keyword queries that barely form a semantic sentence.
• Exact identifiers, codes, article numbers, and rare proper nouns the embedder collapses.
• The question-vs-document distribution shift HyDE tries to mitigate.

The number of dense candidates pulled before fusion is governed by vector_store_k (min-max mode) or a wider window in RRF mode.

Sparse BM25 retrieval

Sparse retrieval asks: which chunks best match the query's terms under BM25? ParadeDB exposes this with the @@@ operator and paradedb.score(id):

SELECT id, content, metadata, paradedb.score(id) AS score_bm25
FROM ragmeup_sparse_embeddings
WHERE content @@@ %s AND {dataset_filter}
ORDER BY score_bm25 DESC
LIMIT %s;

What BM25 is good at:

• Keyword and Boolean-ish lookup ("WI265", "Artikel 7", product codes).
• Short queries where dense embeddings are unstable.
• Complementary signal when dense retrieval drifts to topical-but-wrong neighbors.

Queries are sanitized through escape_query (NLTK tokenize + strip non-letter/number characters) so user punctuation does not break ParadeDB's query parser. Dense search does not need this — it only consumes the embedding vector.

If Re2 ever left its bridge phrase in the query string, the retriever also strips that suffix before BM25 so keyword scores are not polluted.

Combining the two rankings

Dense scores (distances) and BM25 scores live on incompatible scales. RAG Me Up therefore fuses ranked lists, not raw numbers blindly. get_relevant_documents switches on use_rrf:

def get_relevant_documents(self, query, query_embedding, datasets):
    if os.getenv("use_rrf") == "True":
        return self._get_relevant_documents_rrf(...)
    return self._get_relevant_documents_minmax(...)

Min-max hybrid score (default)

Pull the top-vector_store_k BM25 hits and the top-vector_store_k vector hits, union them, deduplicate by chunk id, then blend:

hybrid = 0.5 * (bm25 / max_bm25)
       + 0.5 * (1 - (distance - min_distance) / (max_distance - min_distance))

Cosine distance is inverted and min-max normalized inside the current result batch; BM25 is normalized by the batch maximum. Equal 50/50 weights keep the blend simple and inspectable entirely in SQL (_get_relevant_documents_minmax in PostgresHybridRetriever.py).

Reciprocal Rank Fusion (RRF)

Score magnitudes are the weak point of min-max fusion. A very high BM25 score on a short keyword hit can dominate even when the semantic hit is better. Reciprocal Rank Fusion ignores magnitudes and only looks at ranks. For each document, every list it appears in contributes:

1 / (rrf_k + rank)

Those contributions are summed across BM25 and vector lists.

With use_rrf=True, RAG Me Up:

1. Fetches a wider window from each leg: max(vector_store_k * 4, 20).
2. Accumulates RRF scores in Python.
3. Returns the top vector_store_k.

The smoothing constant defaults to rrf_k=60.

Why prefer RRF? It is stable when one retriever's raw scores dwarf the other, and adding/removing a low-ranked result does not reshape everyone else's normalized scores. Why keep min-max? You get an explicit, SQL-visible hybrid score and fixed 50/50 semantics that are easy to explain in audits.

Configuration

Variable	Role
vector_store_k	Final chunk count after fusion (also per-leg limit in min-max mode)
use_rrf	True for RRF; False for min-max blend
rrf_k	RRF smoothing constant (default 60)
postgres_uri	Hybrid Postgres connection string

Practical guidance

• Start with vector_store_k larger than what you want in the prompt (e.g. 10–20) and let reranking cut down to 3–5.
• If keyword traffic dominates, verify BM25 is contributing — inspect sources / distances in DEBUG logs.
• If dense and sparse disagree wildly in score magnitude, try use_rrf=True before hand-tuning blend weights.
• Retrieval quality is bounded by chunking and embedding choices. Fusion cannot rescue systematically bad chunks.