Lark or sparrow?

About this video

Nadia loves birds. In her park most small brown birds are sparrows, but a rare bird called a lark sometimes visits. She has a phone app that listens to a bird and names it, and it is a good app: it is right 9 times out of 10. One morning a small brown bird sings and the app says "Lark!" How likely is it that she has really found a lark?

Most people say about 9 in 10, since the app is right 9 times in 10. It sounds right, and it is badly wrong. The missing fact is how common larks are in the park in the first place. Say that out of every 100 small brown birds, 97 are sparrows and only 3 are larks, and imagine the app listening to all 100. It names almost all of the 3 larks correctly. But of the 97 sparrows it is wrong 1 time in 10, so it wrongly says "Lark!" for about 10 of them. The app says "Lark!" about 13 times, and only about 3 of those are really larks: less than 1 in 4. Nadia's bird is still most likely a sparrow.

How common something is before you look at the sign is called the base rate. The psychologists Daniel Kahneman and Amos Tversky showed in the 1970s that people very often forget it, even experts. The app isn't bad. But when the thing you are looking for is rare, most of the yes answers from a good-but-not-perfect test come from the many ordinary cases. So ask two questions: how often is this sign right, and how common is the thing at all? If it is rare, check again another way. This video goes with the How Common Is It? lesson of Knowledge Level 2 on Knaph.

Transcript

1. 0:00Nadia's bird app is right nine times in ten. It says, lark!
2. 0:05Most people say it's nine in ten a lark. But how common are larks?
3. 0:11Out of a hundred small brown birds, ninety-seven are sparrows. Three are larks.
4. 0:17The app names all three larks, but also ten sparrows. Thirteen larks, it says.
5. 0:23Only three in thirteen are real. Rare things bring more false alarms.
6. 0:29Before you trust a sign, ask how common the thing is.