Why Pattern 1 Fails: Full Document Injection
Assumption:
- More context = better understanding
Reality:
- More context = attention dilution + cost explosion + quality degradation
The Three Failure Modes of Full Document Injection:
-
Lost in the Middle
- Attention Distribution: Models attend to the first 20K and last 20K tokens, with significantly lower attention on the middle 60–80%.
- Example: In a context window of 150K tokens, critical information at positions 70K-130K (middle) is effectively ignored by the model.
- Impact: Retrieval accuracy drops by 30–60% for information located in this "lost in the middle" zone.
- Attention Distribution: Models attend to the first 20K and last 20K tokens, with significantly lower attention on the middle 60–80%.
-
Distractor Interference
- Semantic Similarity Misleads: Irrelevant but semantically similar content actively misleads the model, causing it to cite incorrect or less relevant information.
- Example: Asking about “current revenue” when context includes 5 years of prior revenue figures.
- The model may cite Q3 2022 revenue instead of Q3 2
- Example: Asking about “current revenue” when context includes 5 years of prior revenue figures.
- Semantic Similarity Misleads: Irrelevant but semantically similar content actively misleads the model, causing it to cite incorrect or less relevant information.
Read the full article at Towards AI - Medium
Want to create content about this topic? Use Nemati AI tools to generate articles, social posts, and more.
4
Contents
Ali NematiWritten by Ali
![[AINews] The Unreasonable Effectiveness of Closing the Loop](/_next/image?url=https%3A%2F%2Fmedia.nemati.ai%2Fmedia%2Fblog%2Fimages%2Farticles%2F600e22851bc7453b.webp&w=3840&q=75)
