The Improbability Principle Summary and Review

by David J. Hand

Has The Improbability Principle by David J. Hand been sitting on your reading list? Pick up the key ideas in the book with this quick summary.

Nature is fundamentally driven by chance and probabilities. As such, chance events – even those that we consider to be total anomalies – have a deep impact on our everyday lives. But if chance plays such an important role in our day-to-day lives, why is it that we seem so ignorant about how probability works?

Indeed, many people turn to the supernatural when they witness a highly improbable event. They see them as omens, wonders, or miracles, rather than looking instead for a natural explanation.

In fact, these improbable events are really nothing to scratch our heads over. By turning our attention to probability theory, we can learn about the rules of chance that so deeply impact our lives.

By examining even the basic properties of probability, we can gain a better understanding of the many strange situations that we are confronted with. We no longer have to throw our hands in the air and say, “I have no idea how this could have possibly happened! It must have been a miracle!”

In this book summary, you’ll learn about the basic laws of chance as well as how to apply them in such a way as to better understand the world around you.

In this summary of The Improbability Principle by David J. Hand,You’ll also learn:

  • how extremely improbable events happen every single day – and why there’s nothing mysterious about it,
  • that being a park ranger can can get you zapped by lightning,
  • why some people buy insurance in case of alien abductions,
  • how to increase your winnings if you happen to pick a winning lotto ticket and
  • why optimists are more likely to find their keys.

The Improbability Principle Key Idea #1: Extremely improbable events are anything but miraculous. In fact, they're commonplace.

Have you ever witnessed an event which seemed so improbable and so unlikely that you had to ask yourself: What were the chances of that happening? One in a million? One in a billion?

And yet, there are many real life examples that suggest that events with vanishingly small probability happen every day.

Consider this strange coincidence from 1972, when the actor Anthony Hopkins was signed to play a role in a film based on G. Feifer's novel The Girl from Petrovka. He traveled to London to buy a copy of the book to study the part, but unfortunately, none of the popular bookstores had a copy.

On the way home, however, Hopkins saw a discarded book lying on the seat next to him in the subway station. That book’s title? The Girl from Petrovka. When Hopkins later told Feifer about this strange occurrence, he learned that he had found the copy that Feifer once lent to a friend who lost it!

The chances of this happening seem dizzyingly small, and yet there are many other such examples:

  • A woman from New Jersey won the lottery twice within a few months.
  • Liz Denial from Nottingham won a prize every single day from October 2012 until June 2013. These prizes weren’t trivial either, including things like a 37-inch LCD TV, a home cinema     system, two Xboxes, a five-star holiday trip to Kenya and ₤16,500 on a TV game show.
  • Roy Sullivan, a park ranger in Virginia, was struck by a lightning seven different times and survived.

It’s tempting to look at each example and think that each is nothing short of a miracle. After seeing events of vanishingly small probabilities, we might wonder if the scientific laws of nature and causality occasionally break down. In order to try to explain such rare events, some look to superstition, prophecies, gods or miracles.

Yet, as we’ll see later, we don’t actually need to appeal to the supernatural to explain these highly improbable events.

The Improbability Principle Key Idea #2: Nature is fundamentally driven by chance, and uncertainty lies at its very core.

Until the beginning of the twentieth century, most scientists held a view of nature which is sometimes called the clockwork universe, i.e., a completely deterministic universe that ticks along a well defined path.

Today’s scientists, however, recognize that some systems are intrinsically unstable, and as such, we can never measure anything with perfect accuracy. In fact, sometimes even a slight change in conditions can lead to completely different results!

For example, imagine cooling a glass of water from 1 to -1 degrees Celsius. As the water passes through 0 degrees, we'll observe a dramatic change: the water will freeze. This tiny change in the temperature suddenly changes the water from a liquid to a solid, thus proving its unstable nature.

Or what about this: imagine you’ve placed a marble atop a ball. No matter how miniscule, the slightest difference in where we place the marble will affect where the marble ends up once it’s rolled off the ball.

If we were to try to measure the point from where it started rolling, we might be able to get it down to maybe one or two decimal places. But because the marble would immediately start rolling off the ball, we could only know its approximate starting point.

Moreover, this uncertainty actually lies right at the heart of physical observations. We can see this intrinsic uncertainty in, for example, the famous Heisenberg uncertainty principle: when looking at particles, we can’t know both a particle’s position and momentum with complete accuracy. The more accurately we can determine its position, the less accurately we will know its momentum and vice versa. Fascinatingly, this limitation is not a result of inadequacies of our measuring instrument, but is in fact a fundamental property of nature.

Knowing this, it only makes sense to shift our view of nature from the clockwork universe to one based on probabilities.

These first two book summarys showed you that improbable events are far from “miraculous.” These next few will show why they are actually so commonplace.

The Improbability Principle Key Idea #3: Something must happen, so we shouldn’t be surprised if it’s outrageous.

So how exactly can highly improbable events happen and keep on happening? The answer lies in what the author calls the Improbability Principle: a set of five fundamental and interoperating laws of chance. In this book summary, we’ll examine the first two.

The first revolves around a simple and often overlooked observation: something must happen. For example, if you throw a die, you know that a number between 1 and 6 will appear face up. Similarly, if you toss a coin, you know it will come up heads or tails.

There are, however, many other possible outcomes: the coin might land balanced on its edge, be swallowed by a passing bird, or fall through the cracks in the floodboard. These might seem “random,” but if we were able to compile a list of all possible outcomes, we would know that one of the outcomes on the list, be that a heads-toss or a hungry bird, must occur.

The author calls this the Law of Inevitability: if you make a list of all possible outcomes, then one of them will inevitably occur.

Taking this one step further, we see that with a large enough number of opportunities, any outrageous thing is likely to happen.

For example, if you only examine a single clover stalk in your entire life, you'd be very lucky if it were a four-leaf clover – most have only three leaves, and about 1 in 10,000 has four. However, if you’re a clover aficionado who takes her similarly inspired friends out clover hunting, your chances of finding a four-leaf clover are much higher. Add this to the fact that you and your small group are not the only people who have ever looked for four-leaf clovers, it then begins to look almost inevitable that someone will find one!

The author calls this the Law of Truly Large Numbers: with enough opportunities, we should expect an event to happen no matter how unlikely it may seem at each opportunity.

The Improbability Principle Key Idea #4: We can make predictions come true by waiting until we know the outcome – or being clever beforehand.

Did you know that we can actually make probabilities as high as we like by making the right choices after something has happened?

For example, imagine that you’ve come across a barn and on the one side of the barn there are many painted targets, each with an arrow in the bull's-eye. "Wow, nice shooting!", you think.

Then, on the other side of the barn, you see that it, too, has many arrows stuck into it... and standing next to them a man busily painting bull's-eyes and targets around each arrow. Surely, this would change your perspective.

This illustrates the Law of Selection: we can alter probabilities with our choices after something has happened.

Consider this real life example: After a major disaster, people often ask, “Why didn’t we see this coming?”. It’s easy to put the pieces together after the fact, and show how they form a continuous, causal chain leading to the outcome.

Although we can’t truly foresee the outcomes of events, we can nonetheless make clever choices to influence those outcomes. Here, the Law of Selection works a bit differently: you can influence the outcomes of events before they happen.

For example, you might believe that, since the chances of choosing a winning lottery ticket are so small, the chances of two or more people choosing the same numbers must be incredibly small. However, people often don't choose their numbers at random; they instead choose numbers like birthdays or anniversaries, or base their choice on some kind of numerical pattern, thus making the chance of two or more people choosing the same numbers much higher than we’d initially think.

So, if you play the lottery, you shouldn’t choose numbers based on a pattern, but instead numbers that others are unlikely to choose. While the chances of you winning remain totally unchanged, you will change the amount you would win if you choose the winning number.

The Improbability Principle Key Idea #5: A slight change in circumstances can have a huge impact on probabilities.

You’ve probably heard the term butterfly effect before, where the tiny flap of a butterfly’s wing can cause a hurricane somewhere else in the world. Interestingly, this phenomenon can happen to probabilities.

In fact, under the right circumstances, tiny probabilities can transform into massive ones.

As an illustration, consider two people sitting on a seesaw. A lighter person sitting far from the balance point can counterbalance a heavier person sitting close to the balance point. But if the heavier person moves just slightly farther out, or if his weight increases, the beam tips, shooting the lighter person skyward.

This, in essence, is the fourth law of the Improbability Principle: the Law of the Probability Lever, which states that even the slightest change in the probability of a single event can have a substantial effect on the chance of other, rare events.

Consider for example October 19, 1987, or Black Monday, on which stock markets around the world crashed. According to standard risk calculation models, the probability of such a significant plunge was virtually non-existent. Even if the stock market remained open for twenty billion years, such an event couldn’t have been anticipated.

And yet, it happened. This is because mathematical models can only approximate real conditions, and so even very slight departures from the model can result in a drastic, unpredicted effects.

Moreover, individual circumstances can radically change the probability of an event occurring. For example, the chances of being zapped by a lightning strike is about 1 in 300,000 on average. However, that risk is much higher if you’re a park ranger, rather than an urban office worker. Indeed, Roy Sullivan, a park ranger in Virginia, was struck by lightning seven times and survived!

While the chance of being zapped seven times by lightning seems like a total anomaly on average, it’s far less miraculous with reference to someone who is more prone to lightning strikes, like a park ranger.

The Improbability Principle Key Idea #6: Similar events are often regarded as identical, which can mislead us.

If you ever spend enough time with a map, you might notice that the eastern coast of the Americas seems to fit the western coast of Europe and Africa, like a jigsaw puzzle. Could this mean anything, or is it mere coincidence? Sometimes it’s hard to tell.

We often regard similar events as identical by relaxing the criteria for a “match.” For example, if we throw a die and get a number close to what we had hoped for, we still might consider it a lucky coincidence.

This phenomenon is described by the fifth law of the Improbability Principle: the Law of Near Enough, which says that we regard events that are “sufficiently similar” as simply “identical.”

The author, for example, once received two successive emails, one titled “Meeting with Muir,” and the other “Miurs referees list.” Despite the different spelling of “Muir” and “Miur,” he nonetheless regarded this apparent match as an amazing coincidence, since the e-mails weren’t otherwise related to each other.

However, when we consider that the author receives a hundred emails a day, indeed many thousands per year, and when we remember the Law of Truly Large Numbers, we should expect these kinds of coincidences to occur.

And regarding similar events as identical can actually have drastic consequences. For example, one Astronomer Royal for Scotland from the 19th century, Charles Piazzi Smyth, found interesting similarities between Great Pyramid Giza and astronomical measurements. He claimed that the perimeter of the pyramid, in inches, equaled the number of days in a thousand years – a coincidence which led him to believe that the “pyramid inch” was a divine measurement passed down from the biblical Noah’s son, and that God had guided the Egyptians in building the pyramids.

Unfortunately, when later measurements were repeated with greater precision, Smyth's findings proved inaccurate.

That’s not to say that apparent coincidences never represent some underlying truth! For example, the matching shapes of the American, European and African coastlines is not mere coincidence – these continental edges were truly once together.

Now that we’ve seen how highly improbable events are both totally natural and to be expected, these final two book summarys will shed some light onto why we find it so hard to grapple with probability.

The Improbability Principle Key Idea #7: Our intuitive grasp of probability isn't good because of biases in the way we think.

Many of the aspects of the Improbability Principle arise because of common, natural idiosyncrasies in the way human beings think and perceive the world. In fact, even professional statisticians can be fooled until they sit down and actually do the math. So what are some of the things that make probability so difficult for us to grasp?

First, it’s extremely difficult for us to do anything randomly. For example, if you were to produce a stream of “random” digits, you’d likely produce a series that actually demonstrated a pattern, for instance, by trying to avoid repeating the same number in the spirit of “randomness.”

In addition, we have the tendency to overestimate low probabilities and underestimate high ones. For example, the chance of winning the 6/49 lottery is about 1 in 14 million. And yet, people still think they can produce the winning ticket! Likewise, people are often prepared to pay in order to reduce very small risks, for example, by buying insurance against alien abduction.

Furthermore, we sometimes perceive the combination of two independent events as more likely than either event by itself.

Consider, for example, this scenario: John is a gifted mathematician who found his dream job in the backroom of an algorithm trading company. So, which of the following do you think is more probable?

  • A: John is married with two lovely children.
  • B: John is married with two lovely children, and likes to spend his evenings tackling mathematical puzzles.

Many people – and maybe even you – would answer B. Yet, the probability that John is described by B cannot possibly be larger than the probability that he's described by A! For John to have the characteristics of B, he must have those of A and more. However, the fact that B nicely matches the stereotypical depiction of John leads to biases in our thinking.

The Improbability Principle Key Idea #8: We tend to ignore evidence that doesn't support our theories.

Although movements like the Enlightenment make great claims about the power of rational thinking and objectivity, it turns out that we aren’t particularly good at either.

In fact, we are heavily biased to try to confirm our own hypotheses, rather than test them.

Imagine, for example, that a friend of yours has a rule for generating a sequence of numbers starting with 2, 4, and 6, and that it’s your job to come up with the rule by guessing the next three numbers in the sequence. For instance, you might suggest 8, 10, and 12, and you’d be right!

Knowing this, you then suggest 14, 16, and 18. Right again! Feeling smug, you guess her rule: increase the number by 2 at each step.

But no! Her rule was actually “any increasing set of whole numbers.” You could have easily tried a different strategy to test your theory, but instead went with what already seemed to be working, and ended up being wrong.

These biases aren’t limited to hypothetical number games: optimists, for example, find “evidence” to support their luck, and pessimists do the same.

We already know the story of Liz Denial, who won a prize every single day for nine months straight. This is less surprising when you consider that someone who believes they are intrinsically lucky might create opportunities where their luck could manifest itself. And indeed, Liz Denial entered an astonishing number of competitions.

On the other hand, consider an anxious student who, convinced he would fail his statistics test, spends more time worrying than studying. What happens as a result? He fails his test!

Or consider this practical example: if you ever need to find your keys, it’s better to be an optimist. Optimists, believing they'll actually find whatever they are looking for, tend to keep looking for longer than pessimists, who feel their efforts to find their keys are futile. As a result, the extra time spent looking would mean that it would be more likely for the optimist to find his keys!

In Review: The Improbability Principle Book Summary

The key message in this book:

There is nothing mysterious about improbability. In fact, there are well known laws of chance which can explain why improbable events occur. However, these laws of chance run contrary to how we normally perceive the world, which keeps us believing in miracles.

Actionable advice:

Randomize your lottery numbers.

If you ever decide to try your luck at the lottery, remember not to choose patterns or common combinations of numbers, such as birthdays or anniversaries. If you do happen to win, this will increase your chances of not having to share the pot with other players.