mahzarin_banaji's picture
Department Chair; Richard Clarke Cabot Professor of Social Ethics, Department of Psychology, Harvard University; Co-author (with Anthony Greenwald), Blind Spot

Bounded Rationality As An Explanation For Many of Our Ills

Explanations that are extraordinary, both analytically and aesthetically, share among others, these properties: (a) they are often simpler compared to what was received wisdom, (b) they point to the more true cause as being some place quite removed from the phenomenon, and (c) they make you wish so much that you had come upon the explanation yourself.

Those of us who attempt to understand the mind, have a unique limitation to confront: the object that is the knower is also the known. The mind is the thing doing the explaining; the mind is also the thing to be explained. Distance from one's own mind, distance from attachments to the specialness of one's species or tribe, getting away from introspection and intuition (not as hypothesis generators but as answers and explanations) are all especially hard to achieve when what we seek to do is explain our own minds and those of others of our kind.

For this reason, my candidate for the most deeply satisfying explanation of recent decades is the idea of bounded rationality. The idea that human beings are smart by comparison to other species, but not smart enough by their own standards including behaving in line with basic axioms of rationality is a now a well-honed observation with deep empirical foundation in the form of discoveries in support.

Herbert Simon put one stake in the ground through the study of information processing and AI, showing that both people and organizations follow principles of behavior such as "satisficing" that constrain them to decent but not the best decisions. The second stake was placed by Kahneman and Tversky, who showed the stunning ways in even experts are error-prone—with consequences for not only their own health and happiness but that of their societies broadly.

Together the view of human nature that evolved over the past four decades has systematically changed the explanation for who we are and why we do what we do. We are error-prone in the unique ways in which we are, the explanation goes, not because we have malign intent, but because of the evolutionary basis of our mental architecture, the manner in which we remember and learn information, the way in which we are affected by those around us and so on. The reason we are boundedly rational is because the information space in which we must do our work is large compared to the capacities we have, including severe limits on conscious awareness, the ability to be able to control behavior, and to act in line even with our own intentions.

From these bounds on rationality generally, we can look also at the compromise of ethical standards—again the story is the same; that it is not intention to harm that's the problem. Rather the explanation lies in such sources are the manner in which some information plays a disproportionate role in decision making, the ability to generalize or overgeneralize, and the commonness of wrong doing that typify daily life. These are the more potent causes of the ethical failures of individuals and institutions.

The idea that bad outcomes result from limited minds that cannot store, compute and adapt to the demands of the environment is a radically different explanation of our capacities and thereby our nature. It's elegance and beauty comes from it placing the emphasis on the ordinary and the invisible rather than on specialness and malign motives. This seems not so dissimilar from another shift in explanation from god to natural section and it is likely to be equally resisted.