Language Models
Map
Arrows point from what you learn first to what it opens. Point at a concept to light up its chain and where it comes back deeper. Right-click a concept (or press and hold it) for its lessons and stories.
- Proven to depth123
- Open
- Locked
- Comes back deeper
Learning path
In the order the concepts build on each other.
- Language modelReadyA program trained on a vast amount of text to predict what comes next. It takes text in and gives text out; what it learned is stored as billions of numbers, not as pages it can look up, so everything else an AI system does is done by the program around it.
- TokenLockedA piece of text a language model reads or writes - a common word, part of a word, a punctuation mark or a chunk of a number - taken from a tokenizer's fixed vocabulary. An English token is about four characters; speed, limits and price are all counted in tokens.Opens after: Language model
- Model APILockedHow a program uses a language model - it sends a request over the internet (which model, the longest reply allowed, the conversation) and gets a reply back (content blocks, a stop reason and the tokens used), paying per token and proving who it is with a secret API key. Nothing else comes with it - memory, tools and the loop are the program's job.Opens after: Language model, Token
- Next-token predictionLockedThe one thing a language model does - give every possible next token a probability. A token is picked (sampling), added to the text, and the whole text goes back in, again and again until an end token or a length limit. It's why answers stream in, vary between tries, and tend to keep an early mistake.Opens after: Token, Probability
- Stateless modelLockedA language model keeps nothing from one request to the next. A chat seems to remember because the app sends the whole conversation history with every request, so what the model "knows" is exactly what is in the request.Opens after: Next-token prediction
- Thinking tokensLockedTokens a model writes to work a problem out before it writes its answer - also called reasoning. They give it more steps of work and a scratchpad to read back. They're output tokens - you wait for them and pay for them, even when hidden - and an effort setting chooses how much thinking to do.Opens after: Next-token prediction
- Context windowLockedThe context is everything in one request to the model - instructions, tool descriptions, the conversation, and everything an agent has read or run. The context window is the most tokens it can hold, counting the reply; about a million for Claude's current models in late 2026.Opens after: Stateless model, Token
- Token costLockedWhat using a model costs - input tokens (everything sent) and output tokens (everything written, thinking included), priced per million, output several times dearer. Since each message sends the whole conversation again, a long chat or agent session costs far more than its length suggests.Opens after: Thinking tokens, Token
- Context managementLockedKeeping an agent's context small and relevant. When the window fills, compaction replaces the history with a summary, which keeps the gist and loses details; and a long, cluttered context tends to work worse even before it's full. One task per session, decisions in files.Opens after: Context window
- Prompt cachingLockedThe provider keeps the work done on the start of a request (the prefix) for a few minutes, so a later request that begins with exactly the same tokens reads that part at about a tenth of the price, and faster. Any change near the start means everything after it is processed again.Opens after: Token cost, Stateless model
- System promptLockedInstructions an app puts at the start of every request, before the conversation - who the assistant is, its rules, its style and its tools. Users rarely see it, but it is why the same model acts so differently in different apps.Opens after: Context window
Connections to other domains
- Language model leads to AI agentAI Agents
- Next-token prediction needs ProbabilityProbability
- Stateless model leads to Agent memoryAI Agents
- Context management leads to Agent skillAI Agents
- Context window leads to Briefing an agentAI Agents
- System prompt leads to Instruction fileAI Agents
- Context management leads to SubagentAI Agents
- System prompt leads to Tool useAI Agents