Bayes's Theorem
You’re worried that your friend is mad at you. You threw a dinner party and didn’t invite them; it’s just the kind of thing they’d be annoyed about. But you’re not really sure. So you send them a text: “Want to hang out tonight?” Twenty minutes later you receive a reply: “Can’t, busy.” How are we to interpret this new information?
Part of the answer comes down to human psychology, of course. But part of it is a bedrock principle of statistical reasoning, known as Bayes’s Theorem.
We turn to Bayes’s Theorem whenever we’re uncertain about the truth of some proposition, and new information comes to light that affects the probability of that proposition being true. The proposition could be our friend’s feelings, or the outcome of the World Cup, or a presidential election, or a particular theory about what happened in the early universe. In other words: we use Bayes’s Theorem literally all the time. We may or may not use it correctly, but it’s everywhere.
The theorem itself isn’t so hard: the probability that a proposition is true, given some new data, is proportional to the probability it was true before that data came in, times the likelihood of the new data if the proposition were true.
So there are two ingredients. First, the prior probability (or simply “the prior”) is the probability we assign to an idea before we gather any new information. Then, the likelihood of some particular piece of data being collected if the idea is correct (or simply “the likelihood”). Bayes’s theorem says that the relative probabilities for different propositions after we collect some new data is just the prior probabilities times the likelihoods.
Scientists use Bayes’s Theorem in a precise, quantitative way all the time. But the theorem—or really, the idea of “Bayesian reasoning” that underlies it—is ubiquitous. Before you sent your friend a text, you had some idea of how likely it was they were mad at you or not. You had, in other words, a prior for the proposition “mad” and another one for “not mad.” When you received their response, you implicitly did a Bayesian updating on those probabilities. What was the likelihood they would send that response if they were mad, and if they weren’t? Multiply by the appropriate priors, and you can now figure out how likely it is that they’re annoyed with you, given your new information.
Behind this bit of dry statistical logic lurk two enormous, profound, worldview-shaping ideas.
One is the very notion of a prior probability. Whether you admit it or not, no matter what data you have, you implicitly have a prior probability for just about every proposition you can think of. If you say, “I have no idea whether that’s true or not,” you’re really just saying, “My prior is 50%.” And there is no objective, cut-and-dried procedure for setting your priors. Different people can dramatically disagree. To one, a photograph that looks like a ghost is incontrovertible evidence for life after death; to another, it’s much more likely to be fake. Given an unlimited amount of evidence and perfect rationality, we should all converge to similar beliefs no matter what priors we start with—but neither evidence nor rationality are perfect or unlimited.
The other big idea is that your degree of belief in an idea should never go all the way to either zero or one. It’s never absolutely impossible to gather a certain bit of data, no matter what the truth is—even the most rigorous scientific experiment is prone to errors, and most of our daily data-collecting is far from rigorous. That’s why science never “proves” anything; we just increase our credences in certain ideas until they are almost (but never exactly) 100%. Bayes’s Theorem reminds us that we should always be open to changing our minds in the face of new information, and tells us exactly what kind of new information we would need.
