Provenance attribution

Retrieval scores answer a different question than users (and auditors) often ask.

• A hybrid or rerank score says: how relevant was this chunk to the query?
• Provenance attribution asks: how much did this chunk actually contribute to the answer that was just generated?

Those can diverge sharply. A chunk may look highly relevant and still be ignored by the LLM. Another may contribute a single critical clause the answer hinges on. For trust UIs, compliance, and debugging hallucinations, you want the second notion.

Provenance is one of the areas where RAG Me Up is unusually intentional. Most frameworks stop at "here are the retrieved docs." RAG Me Up treats post-hoc source attribution as a first-class, pluggable stage with multiple algorithms — including a method that reads attention weights directly from open-weight models.

Where it sits in the pipeline

Provenance runs after the answer is produced, and only when new documents were fetched for that turn. It does not change the answer; it annotates the documents returned to the client.

Select the algorithm with provenance_method:

Value	Idea	Best when
attention	Read the LLM's own attention over query / docs / answer	Local open-weight models that expose attentions
rerank	Cross-encode answer (and optional query) vs each chunk	Closed APIs; you already use FlashRank
similarity	Cosine similarity of chunk embeddings vs answer (and optional query)	Cheap, model-agnostic baseline
llm	Ask the LLM to score each chunk in isolation	Flexible discrete scores; accepts higher cost
None / other	Skip attribution	Latency-critical paths

Orchestration lives in RAGHelper.compute_provenance_scores; algorithms live in provenance.py. Scores are merged onto each document as provenance before the HTTP response (and on the SSE done event).

---

Attention-based provenance (open models)

This is the most distinctive method. When you run a local causal LM through Hugging Face Transformers, you can request output_attentions=True on a forward pass over the full chat thread (template applied). The model then returns, for each layer, tensors describing how much every token attended to every other token.

RAG Me Up uses the last attention layer and asks a precise question:

Relative to everything the model attended to among query, documents, and answer, how much attention mass involved this document?

Why attention is a natural provenance signal

Transformer attention is literally a soft routing mechanism over context tokens while producing (or, in a teacher-forced forward pass, representing) the answer. If answer tokens barely attend to document A but strongly attend to document B, B is a better candidate for "what the model used." The reverse direction (document tokens attending to answer tokens) and query–document interactions add complementary evidence.

Closed APIs do not give you these tensors. That is why attention provenance is inherently an open-weight / in-process technique — and why RAG Me Up treats it as preferred when you control the model weights.

The computation, step by step

The core routine is compute_attention(model, tokenizer, thread, query, context, answer):

1. Encode the full thread (chat template already applied) as thread_tokens.
2. Forward pass with output_attentions=True (no generation required for scoring — teacher-forced over the known answer text in the thread).
3. Take attentions[-1] — the last layer.
4. Locate spans inside the thread by searching for the token sublists of:
   • the user query,
   • the answer,
   • each document string in context (documents are split the same way they were injected).
5. From those spans, collect mean attention masses for several blocks (only keeping positive values — some models effectively zero out long-range attention, and including zeros would unfairly dilute scores on long inputs):

Always included in the normalizing pool

• query → query
• query → answer
• answer → answer
• answer → query

Per document d

• answer → d
• d → answer

Additionally, if provenance_include_query / attribute_include_query is true

• query → d
• d → query

6. For each document, average its (positive) directed scores into a raw document score.
7. Divide each document score by the mean of all collected positive attention masses (documents + query/answer self/cross terms). That relative normalization is the provenance score.

In prose: each document's score is "how much attention involved this document" divided by "how much attention was flying around among the parts we care about." Documents that dominate answer↔doc traffic rise; documents the model ignored fall toward zero even if the retriever loved them.

flowchart TB
  T[Full chat thread with docs + answer] --> F[Forward pass output_attentions]
  F --> L[Last layer attention tensor]
  L --> S[Locate query / doc / answer spans]
  S --> A[Mean attention per directed block]
  A --> N[Normalize doc scores by mean total mass]
  N --> P[Per-document provenance]

Implementation notes that matter in practice

• Span alignment uses exact token-sublist search (find_sublist_positions). Semantic chunking and chat templates must leave document text recoverable as contiguous token spans inside the thread. Historical fixes in RAG Me Up hardened this for semantic chunkers and cases where attention to long documents was "diminished" by zeros — hence only aggregating positive attention.
• Last layer only is a deliberate trade-off: earlier layers often encode syntax/alignment; later layers correlate more with task-relevant routing. Using all layers would be noisier and more expensive.
• Mean over heads and token positions inside each block keeps the statistic stable; you are not interpreting a single head.
• Include query or not changes the story. With query included, provenance mixes "relevant to the ask" with "used in the answer." With query excluded, you focus more purely on answer↔document traffic — usually what auditors want.
• Autoregressive self-attention on the answer (answer→answer) and query (query→query) belongs in the denominator so a document is scored relative to the conversation's overall attention budget, not on an absolute scale that varies wildly across models.

When you can use it

Setup	Attention provenance?
In-process Hugging Face AutoModelForCausalLM with accessible weights	Yes — this is the intended path
Ollama over HTTP	Typically no — attentions not exposed
OpenAI / Anthropic / Gemini APIs	No

The current default server path talks to providers through LLMHelper and therefore ships rerank / similarity / llm in the hot path. The attention algorithm and design remain part of RAG Me Up's provenance story for deployments that load open models locally (as earlier RAGHelper local backends did with provenance_method=attention). If you need it on a fork, wire compute_attention after answer generation with the same model/tokenizer that produced the reply, and pass the fully templated thread plus the document strings exactly as injected.

How to interpret the scores

Attention provenance scores are relative, not probabilities. A document at 1.4 drew more than average mass among the tracked blocks; one at 0.2 was largely ignored. Compare documents within one answer, not across conversations or models. For UIs, rank or bar-chart the scores beside citations rather than treating them as calibrated percentages.

---

Method: rerank (preferred for closed LLMs)

Reuse FlashRank, but change what you score against:

def compute_rerank_provenance(reranker, query, documents, answer):
    if os.getenv("attribute_include_query") == "True":
        full_text = query + "\n" + answer
    else:
        full_text = answer
    return reranker.rerank_documents(documents, full_text)

Query-time reranking asks: relevant to the question?
Provenance reranking asks: relevant to the answer (and maybe the question)?

Same cross-encoder machinery, different target string. Fast if FlashRank is already loaded; no extra generative LLM calls. Requires rerank=True when you rely on the shared reranker instance.

Method: similarity

DocumentSimilarityAttribution embeds the answer (and optional query) plus each chunk with provenance_similarity_llm, then cosine-similarities chunk↔answer (averaged with chunk↔query if enabled). Cheap and provider-agnostic; weaker at detecting paraphrases a cross-encoder or attention pass would catch. Prefer it as a baseline or when you cannot rerank and cannot read attentions.

Method: llm

For each retrieved chunk, call the LLM with provenance_llm_prompt ({query}, {answer}, {context}). The template forces a discrete 0–5 score with no explanation — easy to parse into badges. Most flexible, most expensive (cost scales with number of chunks). Prompt quality dominates: small ordinal scales beat "give a percentage."

Wiring scores to the client

On /chat and on the streaming done event:

for i, doc in enumerate(documents):
    documents[i]["provenance"] = provenance_scores[i]["score"]

The React UI renders these on document cards so users see retrieval evidence and post-hoc attribution side by side.

Configuration

Variable	Role
provenance_method	attention, rerank, similarity, llm, or off
provenance_similarity_llm	HF model for the similarity method
provenance_include_query	Also attribute against the question
provenance_llm_prompt	Scoring prompt for the LLM method

Some code paths also read attribute_include_query. Keep that aligned with provenance_include_query when you customize the environment.

How to choose

flowchart TD
  A[Need provenance?] -->|No| Z[provenance off]
  A -->|Yes| B{Control open model weights?}
  B -->|Yes| C[attention]
  B -->|No| D{FlashRank already on?}
  D -->|Yes| E[rerank]
  D -->|No| F{Budget for N LLM calls?}
  F -->|Yes| G[llm]
  F -->|No| H[similarity]

Pedagogical bottom line

Provenance is post-hoc explanation, not better retrieval. If the right chunk never entered the prompt, no method can invent it. What RAG Me Up adds is honesty about that gap — and, for open models, a way to ask the network itself which context tokens the answer actually leaned on.