What is context rot?
Context rot is the drop in an LLM's accuracy and recall as its input grows longer, even when the task stays the same.
Context rot is the loss of accuracy and recall as a model’s input gets longer. The Claude docs put it this way: as token count grows, accuracy and recall degrade, so what you put in context matters as much as how much space the context window offers. The facts can all be present in the window and the model still gets worse at using them.
What the research shows
Three studies carry most of the evidence.
| Study | What it did | What it found |
|---|---|---|
| Lost in the Middle (Liu et al., July 2023) | Moved the relevant passage around inside the input | Performance is often highest when the relevant information is at the beginning or end of the input, and it can degrade significantly when the position changes |
| RULER (Hsieh et al., April 2024) | Tested 17 long-context models on 13 tasks, beyond single-fact retrieval | Almost all models show large drops as length grows; all claim 32K tokens or more, but only half keep satisfactory performance at 32K |
| Chroma’s context rot report (Hong, Troynikov and Huber, July 2025) | Tested 18 LLMs, including GPT-4.1, Claude 4, Gemini 2.5 and Qwen3 models, while growing input length | Performance degrades with length across the tasks, including a repeated-words task |
RULER’s authors argue the usual needle-in-a-haystack test measures only shallow understanding: models score nearly perfectly on it, then drop on harder tasks such as multi-hop tracing and aggregation.
What makes it worse
The Chroma report varied one thing at a time and found four patterns:
- Less similar questions. As needle-question similarity decreases, models degrade more with increasing input length.
- Distractors. Even a single distractor reduces performance relative to the baseline, and four reduce it further.
- Coherent text. The authors write that structural coherence consistently hurt performance: models did better on shuffled haystacks than on logically ordered ones. They say they have no definitive explanation.
- Full history instead of the relevant part. On LongMemEval, prompts holding only the relevant material (about 300 tokens) beat full prompts (about 113k tokens) across all models. The authors note that for Claude models much of the gap came from abstentions.
The authors also state limits. The tasks are deliberately simple, so real applications may degrade more. Two haystack topics are too few to conclude that similarity always matters in one direction. Some models refused parts of the tasks, and those cases were excluded, while abstentions drove much of the Claude gap on LongMemEval.
Why it happens
Nobody has shown a single cause. Anthropic describes an “attention budget”: each new token uses up part of it, and because of the transformer architecture, “n² pairwise relationships for n tokens” stretch the model’s ability to capture them as context grows. That is an explanation from the model provider, while the Chroma authors say their experiments do not explain the mechanisms. Treat both as open.
The studies also date from 2023 to 2025 and name specific models. They show failure patterns worth testing for, not a fixed score for any model you use now.
How to reduce it
The fix is to put less in the window and put it there on purpose. Anthropic lists four techniques for long-running agents:
| Technique | How it works |
|---|---|
| Compaction | Summarize a conversation nearing the limit and start a new window with the summary |
| Structured note-taking | The agent writes notes to memory outside the window and pulls them back in later |
| Sub-agents | Specialized sub-agents handle focused tasks with clean context windows and return condensed summaries to a lead agent |
| Just-in-time retrieval | Keep lightweight identifiers such as file paths, stored queries and web links, and load data into context at runtime with tools |
The Claude docs call server-side compaction the primary strategy for long conversations and agentic workflows. As of November 2026 the docs list it in beta for Claude 4.6 and later models. The same page lists context editing, with tool result clearing and thinking block clearing. For retrieval-based ways to select what goes in, see RAG pipelines. For the controls each provider ships, see context engineering; for how these choices fit together as a discipline, see context engineering vs prompt engineering.
Check for it in your own system
Chroma’s approach carries over: hold the task fixed and change only the amount of context. If your answers get worse when the same question comes with the full history instead of the relevant slice, you have measured context rot on your own data. That is also the LongMemEval comparison in Chroma’s report.
Run that comparison as an evaluation rather than by feel. In HoneyHive, an experiment defines a function, scores outputs with an evaluator and compares results in the Experiments dashboard, so the short-context and long-context variants can be compared side by side. In production, HoneyHive’s tracing concepts describe events that store input messages and example metadata such as prompt_tokens, and sessions that group a conversation’s events, so you can see whether quality problems line up with growing input. More on that in LLM observability.
FAQ
What is context rot?
Context rot is the loss of accuracy and recall as the number of tokens in a model's context grows. The Claude docs describe it as accuracy and recall degrading as token count grows.
What causes context rot?
It is not settled. Anthropic points to a finite attention budget and n² pairwise relationships between tokens. Chroma's researchers say they do not explain the mechanisms behind the degradation they measured.
Does a larger context window fix context rot?
Not by itself. The RULER benchmark found that all 17 models tested claimed 32K tokens or more, yet only half kept satisfactory performance at 32K.
How do you reduce context rot in an agent?
Keep less in the window. Anthropic lists compaction, structured note-taking, sub-agents with clean context windows and just-in-time retrieval.
Is context rot the same as lost in the middle?
No. Lost in the middle is one effect, where models use information at the start or end of the input better than information in the middle. Context rot also covers degradation with length when position is not the variable.