The Controlled Experiment
The scientific concept that most people would do well to understand and exploit is the one that almost defines science itself: the controlled experiment.
When required to make a decision, the instinctive response of most non-scientists is to introspect, or perhaps call a meeting. The scientific method dictates that wherever possible we should instead conduct a suitable controlled experiment. The superiority of the latter approach is demonstrated not only by the fact that science has uncovered so much about the world in which we live, but also, and even more powerfully, by the fact that such a lot of it — from the Copernican principle and evolution by natural selection to general relativity and quantum mechanics — is so mind-bendingly counter-intuitive.
Our embrace of truth as defined by experiment (rather than by common sense, or consensus, or seniority, or revelation or any other means) has in effect released us from the constraints of our innate preconceptions, prejudices and lack of imagination. Instead it has freed us to appreciate the universe in terms that are well beyond our abilities to derive by intuition alone.
What a shame then that experiments are, by and large, used only by scientists. What if businesspeople and policy-makers were to spend less time relying on instinct or partially informed debate, and more time devising objectives ways to identify the best answers? I think that would often lead to better decisions.
In some domains this is already starting to happen. Online companies like Amazon and Google don't anguish over how to design their websites. Instead they conduct controlled experiments by showing different versions to different groups of users until they have iterated to an optimal solution. (And with the amount of traffic those sites receive, individual tests can be completed in seconds.) They are helped, of course, by the fact that the web is particularly conducive to rapid data acquisition and product iteration. But they are helped even more by the fact that their leaders often have backgrounds in engineering or science and therefore adopt a scientific — which is to say, experimental — mindset.
Government policies — from teaching methods in schools to prison sentencing to taxation — would also benefit from more use of controlled experiments. This is where many people start to get squeamish. To become the subject of an experiment in something as critical or controversial as our children's education or the incarceration of criminals feels like an affront to our sense of fairness, and our strongly held belief in the right to be treated exactly the same as everybody else.
After all, if there are separate experimental and control groups then surely one of them must be losing out. Well, no, because we do not know in advance which group will be better off, which is precisely why we are conducting the experiment. Only when a potentially informative experiment is not conducted do true losers arise: all those future generations who stood to benefit from the results. The real reason people are uncomfortable is simply that we're not used to seeing experiments conducted in these domains. After all, we willingly accept them in the much more serious context of clinical trials, which are literally matters of life and death.
Of course, experiments are not a panacea. They will not tell us, for example, whether an accused person is innocent or guilty. Moreover, experimental results are often inconclusive. In such circumstances a scientist can shrug his shoulders say that he is still unsure, but a businessperson or lawmaker will often have no such luxury and may be forced to make a decision anyway. Yet none of this takes away from the fact that the controlled experiment is the best method yet devised to reveal truths about the world, and we should use them wherever they can be sensibly applied.
