What the summary keeps

About this video

Leila's agent has worked on her quarterly report for two hours. It started sharp; now it's slower, it has redone a chart she had already approved, and a message says the conversation was compacted. Two different things have gone wrong, and both come from the context window.

The first: the window fills up. Everything the agent reads and runs stays in the context, and when the window is nearly full the app has to make room or the next request won't fit. Dropping the oldest messages would drop the most important one, your original request. So most agents compact instead: they ask the model to write a summary of the conversation so far and replace the long history with it. A summary keeps the gist and loses details, and it can't know which details you will care about later; an approval may not make it in. In Claude Code compaction happens by itself near the limit, and you can run /compact yourself, with a note of what to keep.

The second is sneakier: a crowded window works worse even before it is full. A well-known 2023 study, "Lost in the Middle", found that models used facts at the start and end of a very long context better than facts buried in the middle. Newer models are much better at long contexts, but people still see answers get worse as a context grows long and cluttered with old files, long outputs and failed attempts. Some call it context rot.

So treat the context like your own attention. Start a new session for a new task (/clear in Claude Code). Compact before the window decides for you, and say what to keep. Put plans and decisions in a file, which survives compaction and new sessions. Read less and search more, and hand big side searches to a helper with its own window.

Transcript

1. 0:00Two hours in, the agent is slower, and it has redone a chart you'd already approved. A message says the conversation was compacted. Two different things went wrong, and both come from the context window.
2. 0:20Problem one. The window fills up, and the app has to make room. Dropping the oldest messages would drop your request. So instead, the model writes a summary of the conversation, and the summary replaces the history. That's compaction.
3. 0:45But what does a summary of two hours look like? A page. The main points. It keeps the gist, and loses the details. Use last year's blue. The chart is approved. And it can't know which details you'll need later.
4. 1:10So compact on purpose, before the window decides for you, and say what must survive. In Claude Code, slash compact, then a note. Keep the chart decisions and the colours. Now they're in the summary.
5. 1:35Problem two is sneakier. A crowded window works worse, even before it's full. A 2023 study, Lost in the Middle, found models used facts at the start and end of a long context better than facts buried in the middle. Newer models do much better, but clutter still tends to make answers worse.
6. 2:02So keep the window clean. One task per session. Put plans and decisions in a file, which survives compaction. Read less, search more. And hand big side searches to a helper with its own window.
7. 2:30Keep in the window what this task needs, and little else.