The shared start

About this video

Leila's agent session ran for 40 turns. Every turn sends the whole conversation to the model again, so she added it up: about 2.8 million input tokens, at 4 dollars per million about 11 dollars. Her usage page said about a dollar and a half.

Put turn 39 and turn 40 side by side. Turn 40 is turn 39 with a little added at the end: the model's last reply and its last tool result. Everything before that, the system prompt and tool descriptions, CLAUDE.md, the conversation so far, is identical. The model keeps nothing between requests, but reading a prompt means a lot of calculation on every token, and the provider can keep those results outside the model for a few minutes. That is prompt caching. If the next request starts with exactly the same tokens, the stored start, called the prefix, is read from the cache: a cache hit.

Cached input tokens cost about a tenth of the normal price, and on some models less: Claude's largest everyday model charges about 4 dollars per million input tokens and 0.20 for tokens read from the cache. A hit is also faster. On Leila's turn 40, about 106,000 tokens came from the cache and only the last 2,000 were paid in full, and the session fell from about 11 dollars to about 1.50.

Only an exact match counts. The provider compares the new request with the stored one from the first token, and at the first difference the match stops: everything after it is paid in full. So harnesses put what changes least first and add new turns at the end. In Claude Code, switching the model or effort mid-session, adding or removing an MCP server when all tool descriptions load up front, and compacting the conversation make the next turn start over; editing project files, using a skill and starting subagents don't. A stored prefix also expires when unused: after 5 minutes by default on the API, an hour for the main conversation on Claude's subscriptions. Every hit restarts the clock.

Transcript

1. 0:00Leila's agent session ran for forty turns. Every turn sends the whole conversation again, so she added it up, about eleven dollars. Her bill said a dollar and a half.
2. 0:22Put turn thirty-nine and turn forty side by side. Turn forty is turn thirty-nine with a little added at the end. Everything before it is identical.
3. 0:44The model remembers nothing. But the provider can keep the work it did on the start of a request for a few minutes. If the next request starts with exactly the same tokens, that part is read from the cache.
4. 1:10Cached tokens cost about a tenth of the normal price. On turn forty, about a hundred and six thousand tokens came from the cache. The whole session fell from eleven dollars to about one fifty.
5. 1:34But only an exact match counts. The provider compares from the first token. At the first difference the match stops, and everything after it is paid in full.
6. 2:02So switching the model mid-session, adding an MCP server, or compacting starts over. Editing files, using a skill, or starting subagents does not.
7. 2:26And a cached start expires, after five minutes on the API, an hour on subscriptions. Every hit restarts the clock.
8. 2:44Keep the start fixed. Add to the end.