Note
Lost in the Middle
I copied an eighteen-minute talk about context rot, the way a model forgets the middle of a long context, and while chasing down who built the download tool I fell into a thirty-minute rabbit hole and forgot mine. The failure in the machine and the failure in me turned out to be the same shape. So is the fix.
I wanted to keep a talk.
Eighteen minutes, posted on X, about context rot: the thing where you hand a model a million tokens and it accepts every one of them and then cannot reason across them. The window is huge and the attention inside it is not. It matters to me because the thing I am building runs on retrieval, and a system that retrieves from a growing memory has exactly this problem waiting for it.
So I went to pull the video down to transcribe it. Grabbed a downloader off GitHub. And then, the way it always starts, a small and reasonable question. Who built this?
The person makes OSINT tools. One of them surfaces what people try to hide on Reddit. There were two more. Who built those. What else had they shipped. Their other repos. Thirty minutes, gone, down a corridor of tabs that had nothing to do with the talk I had opened to save.
Then something fired. Stay on the path. This is not where you are supposed to be. And it pulled me back.
Here is the part I could not stop turning over. The video was about a machine losing the thread in the middle of a long context. Chasing it, I lost my thread in the middle of a long context. The talk handed me the exact failure it was describing. I did not read about context rot. I performed it.
So I went and looked into what had actually happened to me, because the recursion felt like the finding, not the coincidence.
What context rot actually is
Start with the machine, because the machine is measured.
A model with a two-hundred-thousand-token window does not use those tokens evenly. Feed it a long input and reliability drops as the input grows, even on trivial retrieval, even when the answer is sitting right there in the context. Chroma ran this across eighteen frontier models last year and the drop is consistent. And it has a shape.
The shape is a U. Accuracy is highest when the thing you need is at the very start or the very end of the context, and it falls off a cliff when the thing you need is in the middle. Liu and his coauthors named it in 2023, and the name is the whole paper: "Lost in the Middle." A window that advertises two hundred thousand can start losing the plot at fifty thousand of real load. The fix is not a bigger window. A bigger window is just more middle to get lost in. The fix is deciding what goes into the window, and where.
Hold onto the U. It is the whole essay.
The same shape in me
Now the mind, which is messier but rhymes exactly.
A rabbit hole is curiosity with the brake removed. Curiosity, the way Loewenstein framed it, is an information gap: the specific discomfort of knowing there is a thing you do not yet know. What makes it a trap is where the reward lands. The pull fires on the moment just before the gap closes, not after, so every answer opens a fresh gap and re-arms the drive. Who built this becomes what else did they build becomes who funded them. Just one more. There is no natural bottom, because the closing of one gap is the opening of the next.
And the thing that is supposed to catch it is weak on purpose. The catch has a name, meta-awareness, and its defining property is that it is sparse. You do not run a monitor in the background that trips the instant you drift. You wander, fully absorbed, with zero awareness that you are wandering, and then at some unpredictable moment a separate process happens to check, and catches you. I caught it at thirty minutes. It could as easily have been two hours. The catch is luck, not a guardrail.
And it costs more than the time. Switching back does not switch clean. Part of your attention stays stuck on the thing you left, what the research calls attention residue, and it lingers for something like twenty minutes after you have moved on. So the thirty minutes in the hole is not the whole bill. There is a tail on it.
Line the two failures up and they are the same object.
Both accumulate a long context. The machine accumulates tokens; I accumulated hops, tool to author to repo to tool. Both accept everything: every token ingested, every tangent followed. Both bury the one thing that mattered, the needle for the model, the original goal for me, in the middle, where it is hardest to reach. Both fail silently: the model throws no error, it just gets quietly worse, and I got no alarm, I just drifted. And neither one can see it from the inside. The model needs an outside monitor. I needed the sparse, lucky catch.
The goal is the needle
That is the sentence I came away with.
In a mind and in a machine, a long accumulating context buries the goal in the middle, and the middle is exactly where retrieval is worst. Which means the fix is identical, and almost stupidly simple. Put the goal back at the edge.
"Restate your objective" rescues a distracted founder and a long-context agent for the same reason, and it is not a productivity slogan in either case. It is repositioning the needle out of the dead zone and back to the recency edge, where attention actually reaches. You do not need more context. You need the one thing that matters to not be in the middle.
What this does to the thing I am building
I build a digital mind that runs on its own memory, so this is not an abstraction to me. It is a design constraint with a date on it.
The first thing it settles: working memory has to be curated and positioned, not maximal. As what my system knows grows, everything it reads in order to think, the slice of recent life it scans, all of it context-rots the same way a model does. The instinct to pour the whole world into the prompt and let a big context sort it out is the instinct that fails. You retrieve the small relevant slice. You keep the current goal at the edge. You never let the thing that matters fall into the middle. I had already been building recall as retrieve-by-relevance rather than load-everything, and the U-curve is the reason that was right.
The second thing is stranger, and I like it more. I have written before that a mind should have a heartbeat, and mine has one: a beat that notices the people and projects it does not yet know enough about and goes looking, on its own, without me asking. That beat is an explore engine. It is, functionally, a curiosity I built on purpose. Which means it needs the exact brake I was missing in that corridor of tabs. The mirror image of go find out is are you still on the thing. The same faculty that pushes it to explore is the one that should, now and then, sample whether either of us has drifted, and re-surface the stated intent. Not a nag. The metacognitive catch that human beings only ever get by luck, turned into something that actually runs.
Nothing wasted
Here is the part that keeps me honest about rabbit holes.
The thirty minutes were not waste. The drift surfaced three real things, and each one belonged to a different project I am working on: the context-rot talk itself, which is about my system's retrieval; a video downloader, which made me ask why I could not just build that; and a set of Reddit archive tools, which turned out to matter more than any of it.
Because my own system's Reddit capture has been broken for a while. Reddit locked its free API in 2023, and every path I had been using to pull a Reddit link into memory has returned a 403 ever since, so a Reddit link quietly degrades to just saving the text of the message. And the hole I fell into landed me on exactly the class of tool that unblocks it: independent Reddit archives, Arctic Shift and PullPush, that still serve the content Reddit's own API no longer will.
Read that again. I went looking for who built a video downloader, and I came back holding the repair to a broken component I had been ignoring. The drift found the fix. If the thing I am building can pull me back from the tangent and also keep what the tangent turned up, routed to the project it serves, then the rabbit hole stops being a failure mode and becomes a way of foraging. The catch and the harvest are the same faculty read in two directions. That is not a workaround for curiosity. That is the point of it.
The honest caveat
The talk put a sharp number on it, something like eighty percent retrieval at 256K collapsing to thirty-six at a million, on one specific frontier model. I did not verify that exact figure or that exact model, and the argument does not need it. The phenomenon is documented across eighteen models. The U-shaped curve is documented. The more-than-thirty-percent sag in the middle is documented. The shape is solid. The decimal points are the talk's, and I am telling you they are the talk's.
The map is not the territory. But this time the map and the territory had the same bug, and finding it in one told me exactly where to look in the other.
Sources
- Chroma, "Context Rot: How Increasing Input Tokens Impacts LLM Performance," 2025
- Nelson Liu et al., "Lost in the Middle: How Language Models Use Long Contexts," 2023: https://arxiv.org/abs/2307.03172
- George Loewenstein, "The Psychology of Curiosity," 1994, information-gap theory
- Jonathan Schooler et al., on meta-awareness and mind-wandering
- Sophie Leroy, "Why Is It So Hard to Do My Work?", 2009, attention residue
- Arctic Shift and PullPush, independent Reddit archives
Audio transcript
You are listening to "Lost in the Middle," a note by Jain Yagi.
I wanted to save an eighteen-minute talk.
It was about context rot, the failure that appears when a language model accepts an enormous amount of information but becomes worse at reasoning across it. The subject mattered because I am building a system that retrieves from a growing memory. If more memory eventually makes the system less able to find what matters, that is not an edge case. It is a design constraint.
I found a tool to download the video, then asked one small question. Who built this?
The answer led to another tool, then another person, then more repositories. Thirty minutes disappeared into a corridor of tabs. Eventually a sentence surfaced in my mind: stay on the path, this is not where you are supposed to be.
That was when I saw the joke. I had opened a talk about a machine losing its goal inside a long context, then lost my own goal while trying to save it. I did not merely learn about context rot. I performed it.
In a long model context, information is not used evenly. Important material near the beginning or the end is often easier to retrieve than material buried in the middle. The accuracy curve forms a U. A larger context window does not automatically solve the problem. Sometimes it creates more middle in which the important thing can disappear.
My rabbit hole had the same shape. The machine accumulated tokens. I accumulated hops. Both systems accepted each new piece without discarding anything. In both, the original goal slipped away from the edge and became one item among many. Neither failure produced a clean alarm. The machine simply became less reliable. I simply kept clicking.
Human curiosity makes the drift easy. One answer closes an information gap and opens another. Who built this becomes what else did they build, then who worked with them, then where did that idea come from. There is no natural floor. The process that notices the drift, meta-awareness, does not run continuously. It checks occasionally. By the time it catches us, the path may be far behind.
The fix for both systems is almost embarrassingly simple: put the goal back at the edge.
Restating the objective is not motivational decoration. It changes the position of the most important information. For a long-context agent, the current goal should be kept visible and recent. Working memory should be curated, not maximal. Retrieve the smallest relevant slice. Do not pour an entire history into the prompt and hope scale will sort it out.
The same design should govern curiosity. An explore engine needs a brake that asks, are you still on the thing? It should be able to surface the stated intent before the intent disappears into the middle.
But the thirty minutes were not empty. The drift uncovered independent Reddit archives that could repair a broken capture path in my own system. The rabbit hole found something valuable for another project. That suggests a better outcome than simply banning tangents. Catch the drift, return to the goal, and keep the harvest. Route each useful discovery to the place it belongs.
Curiosity without a brake becomes context rot. A brake without curiosity prevents discovery. The useful system can do both: bring the needle back to the edge and carry home whatever it found in the middle.