The Ones You Don't See

Grandpa and Noor are walking through the old part of town. "Look at these old houses," he says. "Two hundred years old, and still standing strong. They really knew how to build in those days. Not like the houses today." Every old house on the street is solid and beautiful. Is Grandpa right?

What happened to the others?

Grandpa's evidence is real: the old houses he sees are well built. But think about which old houses are still there to be seen. Two hundred years ago, people built good houses and bad ones, just as they do now. The badly built ones cracked, leaked and fell down long ago. Some were knocked down to make room for something else. Only the strongest survived.

So the old houses Grandpa sees aren't a fair sample of old houses. They are the survivors. Comparing them with all new houses - good and bad together, since none have had time to fall down yet - makes the past look better than it was.

Looking only at the examples that survived or succeeded, and forgetting the ones that didn't, is called survivorship bias. It is a bias - a pull in one direction - like confirmation bias in the Seeing What You Expect lesson. But this time the missing evidence isn't forgotten by us. It is simply no longer there to see.

Famous winners

Survivorship bias is common when people talk about success. "That famous band started in a garage with no money. So if you start a band in your garage, you can make it too!" But for every band that became famous, there were many thousands of garage bands that worked just as hard and nobody ever heard of. You only hear about the winners.

If you have read Logic Level 1, this is like its examples all of one kind (asking only people at a football club what the best sport is). Here, too, the examples you see have been picked for you - not by a person, but by what survived.

The question to ask

Before drawing a conclusion from a set of examples, ask: what happened to the ones I can't see?

• Old tools, old furniture, old songs: the ones still around are the best of their time, not the usual ones.
• Stories of people who got rich quickly: you don't hear from the ones who lost.
• "Everyone in the advanced swimming class passed the test": the ones who found it too hard left the class before the test.

Try it at home

1. Find something old in your home or street that is still in good shape.
2. Ask: how many others like it were made, and what happened to them?
3. Listen for a success story this week. Who are the ones you don't hear about?

Words to know

• survivor - one of the examples still there, after the others have gone
• survivorship bias - drawing a conclusion only from the examples that survived or succeeded, forgetting the ones that didn't

Big idea

Sometimes the examples we can see are only the survivors - the strongest houses, the bands that became famous. Before drawing a conclusion, ask what happened to the ones you can't see, because they were part of the story too.