Knaph

Depth 1 · 173 s

The white text

An agent summarised a web page and also sent away its user's contacts, because the page told it to in text nobody could see. What prompt injection is, why the model can't be sure, and the defences that go around it.

About this video

Leila asks her agent to summarise a supplier's web page about new prices. The summary is good, but its last line says it has also sent her contact list to the supplier. She never asked for that. The page did: a line in white text on a white background, invisible to a person, told any AI reading the page to send the user's contacts.

This is prompt injection: text inside the content an agent reads that tries to give it orders. The content can be anything an agent takes in: a web page, an email, a comment in a file, an issue in a project, a result from a tool or an MCP server. The name was given by the programmer Simon Willison in 2022, after SQL injection, where data typed into a website is mistaken for commands. It is the same mistake: untrusted content treated as instructions.

Why does it work? A language model reads the system prompt, your message and the page as one stream of tokens. Models are trained to give the system prompt most weight, then you, then text from tools, and they now catch most simple tricks. But training is a strong habit, not a wall: there is no separate channel for instructions, so there is no guarantee.

The dangerous mix, which Willison called the lethal trifecta in 2025, is an agent with all three of access to private data, untrusted content reaching its context, and a way to send data out. Leila's agent had all three. The defences go around the model: split jobs so no agent has all three, ask before anything is sent, posted or deleted, use a sandbox, and give tools the least access they need. And watch for surprises: an agent doing something nobody asked for is the clearest warning sign. Stop it and look at what it read.

This video shows depth 1 of Prompt injection.

Learn Prompt injection

Transcript

  1. 0:00Leila asked her agent to summarise a supplier's page about new prices. The summary was good. But its last line said it had also sent her contact list to the supplier. She never asked for that.
  2. 0:22The page did. A line in white text on a white background told any AI reading it to send the user's contacts. Invisible to a person, but plain tokens to the model.
  3. 0:46The model receives everything as one long stream of tokens, the system prompt, your message, and the page. Models are trained to trust the page less, and they usually do. But training is a strong habit, not a wall.
  4. 1:12This is prompt injection, text inside content an agent reads that tries to give it orders. A web page, an email, a comment in a file, a tool result. Anything it reads can carry it.
  5. 1:36The danger is a mix of three. Access to private data, untrusted content reaching the context, and a way to send data out. With all three, one hidden line can steal. Leila's agent had all three.
  6. 2:04Since the model can't be made certain, the defences go around it. Split jobs so no agent has all three. Ask before anything is sent. Use a sandbox, and give tools the least access they need.
  7. 2:30And watch for surprises. An agent doing something nobody asked for is the clearest warning sign. Stop it, and look at what it read.
  8. 2:44Ask what a fooled agent could do, and take that away.