Thinking Before Answering

Some AI apps show a little note before the answer: "Thought for 12 seconds", sometimes with a summary of the thinking you can open. Agents do it too, often for much longer. What is the model doing in those seconds, and why does it help?

Leila What's 17 times 24? Don't work it out, just say the first number that comes to mind.

Omid Something like... four hundred? I'd be guessing.

Leila And if you can write it down?

Omid 17 times 20 is 340. 17 times 4 is 68. 340 plus 68 is 408.

Leila So you needed the paper.

Omid I needed the steps. Now think about the model. When it writes a token, how much work does it get to do for that one token?

Leila One step of work. One pass through the model, you said.

Omid Roughly the same amount of work for every token, whether the token is "the" or the answer to a hard sum. So if you ask "What's 17 times 24? Answer only with the number", how much work does it get to do on the actual problem?

Leila One token's worth. That's like you blurting out "four hundred".

Omid And if you ask it to work it out step by step?

Leila Then it writes the steps, and each step is more tokens, so more work. And it can read its own steps back, because they're part of the text now. It's using the text as its paper!

Omid That's exactly it. Writing out the reasoning gives the model more steps of work and a place to keep what it has figured out so far.

Thinking tokens

For years, people got better answers by adding "think step by step" to their prompts. Today's models do this on their own. Before writing the answer, they write thinking tokens: a stretch of working out, also called reasoning. They plan, try an approach, notice a mistake, try again, and only then write the answer you see.

Apps usually hide the thinking, or show a short summary of it. But it's real text the model wrote, and two facts about it matter:

• Thinking tokens are output tokens. You wait for them as they're written, and you pay for them like any other output, even when you don't see them.
• More thinking helps most on hard problems: code, planning, maths, puzzles, anything with several steps. For "what's the capital of France?" it adds time and cost for nothing.

Effort

Because thinking costs time and money, many models let you choose how much of it to do. In Claude this is called effort, with levels from low to max. An agent like Claude Code lets you change it as you work.

Low effort suits quick, simple jobs: renaming, formatting, a short question. High effort is worth it when the task is hard and a wrong answer is expensive: a tricky bug, a plan for a big change, a careful analysis.

Careful

It's tempting to read the visible thinking as an exact record of how the model reached its answer. It isn't guaranteed to be. Researchers have found cases where a model's written reasoning left out something that clearly influenced its answer. Read the thinking as a helpful sketch of the model's working, not as proof.

Leila So when my agent "thinks for two minutes" before touching anything, that's tokens I'm paying for.

Omid Yes, and often well spent: a plan made before acting saves a lot of wrong turns. But if it's thinking hard about renaming one file, turn the effort down.

Try it

Ask a chat assistant a small puzzle, like "A bat and a ball cost 1.10 in total. The bat costs 1.00 more than the ball. How much is the ball? Answer with just the number, no working." Then, in a new chat, ask the same thing but say "Work it out step by step." If your app has a thinking or reasoning option, try it on and off and compare how long each takes.

Using it well

1. Match the effort to the task. Turn it up for hard problems where a wrong answer costs you; turn it down for quick, routine work. You'll get faster answers and spend less.
2. For hard tasks, ask for the plan or the working first. Even with built-in thinking, asking to see the approach before the result lets you catch a wrong path early.
3. Don't treat the thinking as a guarantee. Check the answer itself, not just whether the reasoning sounds sensible.

Words to know

• thinking tokens - tokens a model writes to work a problem out before it writes its answer; also called reasoning
• reasoning - the model's working out, written as text before the answer, often hidden or summarised by the app
• effort - a setting for how much the model thinks: more effort means more thinking tokens, more time and cost, and usually better answers on hard problems

Big idea

A model does about the same amount of work for every token, so writing out its working gives it more work and a scratchpad. That's what thinking tokens are: real output you wait for and pay for, worth it on hard problems and wasted on easy ones.