Thinking on paper

About this video

What is 17 times 24? Say the first number that comes to mind. Most people blurt something like 400. With paper it's easy: 17 times 20 is 340, 17 times 4 is 68, and 340 plus 68 is 408. You didn't need more cleverness. You needed the steps.

A language model is in the same position. It does roughly the same amount of work for every token it writes, whether the token is "the" or the answer to a hard sum. Ask it to answer with just the number, and it gets one token's worth of work on the whole problem. Ask it to work it out, and each step is more tokens, so more work, and the steps stay in the text, where it can read them back. The text is its paper.

For years people added "think step by step" to their prompts. Today's models do it on their own: before the answer they write thinking tokens, also called reasoning, where they plan, try an approach, notice a mistake and try again. Apps usually hide the thinking or show a short summary, but thinking tokens are output tokens. You wait for them and pay for them, even when you don't see them. They help most on hard problems, like code, planning, maths and puzzles, and add time and cost for nothing on easy ones.

So many models let you choose how much thinking to do. In Claude this is called effort, from low to max. Low suits quick jobs like renaming a file; high is worth it for a tricky bug or a plan for a big change. One caution: the visible thinking isn't guaranteed to be an exact record of how the model reached its answer. Read it as a sketch, and check the answer itself.

Transcript

1. 0:00What's seventeen times twenty-four? Say it now, no paper. Most people blurt something like four hundred.
2. 0:16With paper it's easy. Seventeen times twenty is three forty, seventeen times four is sixty-eight. Together, four hundred and eight.
3. 0:38A model does about the same work for every token, whether it's "the" or the answer to a hard sum. Ask for just the number, and the whole problem gets one token's worth of work.
4. 1:04So today's models write their working first, in thinking tokens, and then the answer. They can read their own steps back. The text is their paper.
5. 1:30Apps often hide the thinking. But thinking tokens are output tokens. You wait for them and pay for them, even when you don't see them.
6. 1:54So you can choose how much it thinks. In Claude that's called effort. Low for renaming a file. High for a tricky bug, where a wrong answer is expensive.
7. 2:20Match the effort to the task. For hard tasks, ask for the plan first. And treat the thinking as a sketch, not proof. Check the answer.
8. 2:38Thinking tokens give the model more work, and a paper to do it on.