Description

Level 2 — Taxonomy & Root Causes of Hallucinations

• Closed-Domain (Intrinsic / Context) Hallucination: The model contradicts or fabricates details relative to the source text provided right inside the prompt (e.g., summarizing a medical note and inventing a dosage not mentioned in the note).

• Open-Domain (Extrinsic) Hallucination: The model states false world knowledge when no reference document is provided (e.g., inventing fake Python libraries, fake legal citations, or wrong historical dates).

Why Do LLMs Hallucinate? (5 Core Technical Causes)

1. Next-Token Likelihood Truthfulness (Pretraining Objective): Cross-entropy loss rewards predicting statistically plausible token sequences from noisy web text, not epistemic truth.

2. SFT Exposure Bias & "Forced Guessing" (Sycophancy): If SFT datasets always contain confident answers—even for obscure facts the base model doesn't actually know in its weights—SFT teaches the model to bluff confidently whenever it is uncertain!

3. The Snowballing Effect (Autoregressive Error Propagation): Because decoding is causal, once an LLM samples one slightly inaccurate token at step , that token enters the context for step , forcing the model to invent justifications to stay self-consistent with its earlier mistake.

4. Parametric Knowledge Cutoff & Long-Tail Compression: A model's weights are a lossy compression of the internet up to its cutoff date; rare entities (the "long tail") suffer interference in weight space.

5. Attention Dilution ("Lost in the Middle"): In long prompts ( tokens), U-shaped attention bias causes the model to attend strongly to the beginning and end of the prompt while missing facts buried in the middle.

Level 3 — Engineering Mitigations Across the Stack

1. Context & Architecture Layer (RAG + Citation Grounding):
• Retrieve verified chunks via hybrid search (BM25 + dense embeddings + cross-encoder reranker) and enforce inline span citations ([Source ID]) with an explicit abstention instruction: "If the answer is not explicitly stated in the provided context, respond ONLY with INSUFFICIENT_CONTEXT."

2. Decoding Layer:
• Lower temperature () for factual tasks.
• Context-Aware Decoding (CAD) / DoLa (Decoding by Contrasting Layers): Contrasts logits with vs. without context (or late vs. early Transformer layers) to amplify factual knowledge and suppress surface-level priors:

3. Post-Training Layer (Factuality-Aware RLHF/DPO):
• Calibrate SFT/RLHF so abstaining on unknown facts receives higher reward than guessing wrong ().

4. Verification & Guardrail Layer (NLI & Claim Decomposition):
• Break the generated response into atomic claims and run a fast Natural Language Inference (NLI) model or LLM-as-a-Judge (RAGAS Faithfulness) to check whether each claim is Entailed, Contradicted, or Unsupported by the retrieved context.

Learning notes

Defense-in-Depth Anti-Hallucination Pipeline

Root Cause Symptom Mitigation Matrix

Example or implementation

Context-Aware Decoding (CAD) + Claim Faithfulness Scorer in Python

import torch
import torch.nn.functional as F

def context_aware_decoding_step(
    logits_with_context: torch.Tensor,     # Logits from P(y_t | context, query)
    logits_without_context: torch.Tensor,  # Logits from P(y_t | query) alone
    alpha: float = 0.5,
) -> torch.Tensor:
    """
    Implements Context-Aware Decoding (Shi et al., 2023):
    Amplifies tokens whose probability increases when the retrieved context is present,
    suppressing the model's stubborn prior hallucinations!
    """
    log_p_ctx = F.log_softmax(logits_with_context, dim=-1)
    log_p_prior = F.log_softmax(logits_without_context, dim=-1)
    adjusted_logits = (1.0 + alpha) * log_p_ctx - alpha * log_p_prior
    return F.softmax(adjusted_logits, dim=-1)

def compute_ragas_faithfulness(claims: list[str], entailed_flags: list[bool]) -> float:
    """
    RAGAS Faithfulness Metric:
    Faithfulness = (# of atomic claims supported by retrieved context) / (Total atomic claims in answer)
    """
    if not claims:
        return 1.0
    supported = sum(1 for flag in entailed_flags if flag)
    return supported / len(claims)

# Example: Model prior wants to say "Paris" (index 0), but context says "Lyon" (index 1)
logits_prior = torch.tensor([[5.0, 1.0, 0.0]])   # Prior strongly favors token 0
logits_ctx   = torch.tensor([[3.5, 4.0, 0.0]])   # With context, token 1 is slightly ahead
cad_probs = context_aware_decoding_step(logits_ctx, logits_prior, alpha=1.0)
print("Standard Context Probs:", F.softmax(logits_ctx, dim=-1).tolist())
print("CAD Probs (Suppresses Prior):", cad_probs.tolist())

Interview-ready explanation

Questions to revisit

Q1: Why does Chain-of-Thought (writing reasoning steps BEFORE the final answer) reduce hallucination?

Q2: What happens when a user's RAG context contradicts the LLM's pretraining knowledge (e.g., an internal company policy vs. general web knowledge)?

Q3: What is the difference between Faithfulness (Groundedness) and Answer Relevance in RAG evaluation?