HUMANS: DOING MORE WITH LESS
TOM GRIFFITHS: I’m going to talk about two problems that seem contradictory, but I’m going to argue how they are intimately related to one another. The first problem is that people are still smarter than machines. This is not necessarily a problem for people; it’s more of a problem for machines. Despite the recent advances in AI, you can point to lots of individual things that people can still do better than computers can, but, more generally, you only have one system that is capable of doing all of those different kinds of things, and that system is human beings.
The current trend in machine learning is one of solving problems by increasing the amount of data and the amount of computation that get thrown at them. If I were showing slides here, I would show you a nice picture that some of the people at OpenAI made, where they took a bunch of the recent milestones in AI, starting from image net classification through things like AlphaGo and AlphaZero, and they plotted out as a function of time how much compute went into each of those things. You'd see there’s a nice increasing line. I would argue that focusing on that trajectory is something that isn’t necessarily going to take us in the direction of getting systems that can do the kinds of things people can do, particularly, this generality that characterizes human intelligence.
As a historical example, the interaction between Deep Blue and Gary Kasparov has been taken as evidence for the success of AI, but you can instead look at it as revealing something important about the power of human cognition. While Deep Blue won the majority of those games, they were doing it under entirely different conditions. Kasparov was playing with the energy equivalent of a light bulb and was able to evaluate maybe three different moves a second, whereas Deep Blue was playing with a huge amount of energy resources going into it and the capacity to evaluate something like 100,000 moves per second.
The critical difference there is that one of the things that helps to make human beings intelligent in the way that human beings are intelligent is intrinsically the fact that we have limited cognitive resources. Our ability to efficiently manage, use, and deploy those cognitive resources in different ways to engage with the different kinds of computational problems that we encounter is part of what makes us intelligent in the way that we characteristically think is intelligent.
If we’re doing slogans like Alison Gopnik was doing, the slogan here would be, "Humans: Doing More With Less." That kind of perspective is not necessarily one that is encouraged in the current machine-learning based approach to AI, but it's going to be critical to being able to succeed in getting past some of the challenges that the field is currently facing.
The second problem, which as I said seems at odds with this, is that people are not so smart. On the one hand we have people smarter than machines, and on the other hand we have people with a reputation for being dumb. You have heard hints about this reputation. People have well known cognitive defects, and Danny Kahneman is one of the people who helped to reveal those defects. The way in which those defects are typically characterized is in terms of a comparison of human beings against a classical notion of rationality. This classical notion of rationality isn’t a good criterion for evaluating human behavior or, importantly, the behavior of machines.
This classical notion of rationality says that in any situation what you should be doing is taking the action that maximizes your expected utility without regard for how hard it is to compute that action. That characterization of rational behavior is something that is not achievable by any realistic organism, whether it be a human being or a computer, because all realistic organisms are limited in the amount of computation that they have available to them.
The reason why this is something that matters in the context of both AI and understanding human cognition is that it suggests there might be a different way that we could go about characterizing what constitutes rational behavior for realistic entities. It might be one that gives us different insight into understanding the ways in which human beings behave, and whether or not the things that we do constitute cognitive defects.
Part of the reason why that’s important for AI is not just because that’s a criterion that we’re going to hold AI systems to, but because if it gives us a model of human behavior that has the same generality as that classic notion of rationality, then it gives us an important tool that AI systems are going to need in order to be able to act in ways that are beneficial to humans. It gives us a component of a system that is going to be able to make inferences about what human beings want based on the ways in which human beings behave.
Imagine a superintelligent system with far more computational resources than us mere humans that’s trying to make inferences about what the humans who are surrounding it—which it thinks of as cute little pets—are trying to achieve so that it is then able to act in a way that is consistent with what those human beings might want. That system needs to be able to simulate what an agent with greater constraints on its cognitive resources should be doing, and it should be able to make inferences, like the fact that we’re not able to calculate the zeros of the Riemann zeta function or discover a cure for cancer. It doesn’t mean we’re not interested in those things; it’s just a consequence of the cognitive limitations that we have.
As a parent of two small children, a problem that I face all the time is trying to figure out what my kids want—kids who are operating in an entirely different mode of computation, and having to build a kind of internal model of how a toddler’s mind works such that it’s possible to unravel that and work out that there’s a particular motivation for the very strange pattern of actions that they’re taking.
Both from the perspective of understanding human cognition and from the perspective of being able to build AI systems that can understand human cognition, it’s desirable for us to have a better model of how rational agents should act if those rational agents have limited cognitive resources. That’s something I’ve been working on for the last few years. We have an approach to thinking about this that we call resource rationality. And this is closely related to similar ideas that are being proposed in the artificial intelligence literature. One of these ideas is the notion of bounded optimality, proposed by Stuart Russell.
Basically, what we want to do is come up with a criterion that describes how a rational agent, be it a human or a computer, with limited computational resources should use those computational resources and then act. You can think about this as characterizing a kind of optimization problem, similar to the classical optimization problem which says that what you want to do is maximize your expected utility. We’re going to think about how to go about choosing an algorithm that is going to lead to an action we take which maximizes expected utility while minimizing the associated computational costs.
If you have a model of the computations that are available to an agent, the costs that are associated with those computations—the amount of time that they take or amount of other kinds of resources—then you can define an optimization problem, which then gives us a way of saying what constitutes rational behavior. Rational behavior is no longer the agent who always takes the perfect action in the perfect circumstance, it’s the agent who follows the algorithm that leads them to take the action that best optimizes this joint criterion of maximizing expected utility while minimizing computational cost.
What I want to do is give you one concrete example of a way in which that is useful in understanding one of these classic cases where people behave irrationally. And that example is what’s known as the availability heuristic, particularly, the overrepresentation of extreme events. One way in which people often act irrationally with respect to a classical criterion is that if you ask them to estimate the probability of something like a terrorist attack, or a shark attack, or these other extreme negative circumstances, they significantly overestimate those probabilities.
So, when you’re getting on a plane, you’re spending more time than you should thinking about the possibility that the plane will crash. When you’re making a decision about going snorkeling, you’re thinking not enough about the tiny worms on the corner of the coral reef, and far too much about the sharks that are very unlikely to bite you. Those things seem irrational. They’re things that are going to affect your behavior in ways that aren’t necessarily consistent with the way in which a purely unbiased agent who’s appropriately evaluating expected utilities might act.
When we put that in the context of a resource rationality framework, we need to define the computational problem that we want to solve, talk about the resources that we have available, and then think about what the best kind of strategy is for deploying those resources. In this context, it’s clear that what we want to do is evaluate something like an expected utility. That requires summing over all of the possible outcomes of our action—the utility of that outcome, multiplied by its probability. That’s potentially a costly procedure. When you're taking an action in the real world, there are many possible outcomes and they’re going to have a variety of different utilities.
Let’s say you were going to try and approximate that calculation. And the way you’re going to try and approximate it is by drawing samples, doing a Monte Carlo approximation. You’re going to sample some possible outcomes, and then you're going to consider the utilities of those possible outcomes, add those up, and that’s going to be the way in which you’re going to evaluate whether you should take that action. The choice that you have to make is a choice about what distribution you’re going to sample from. Your goal is to choose a distribution such that you’re able to draw a relatively small number of samples because those samples are costly. That’s time you’re spending standing around rather than going snorkeling. You want to come up with a way of drawing those samples that will allow you to minimize those costs, make decisions quickly, make decisions with small numbers of samples.
When we think about trying to do an approximation via this Monte Carlo procedure, the intuitive, straightforward thing people think of is to sample directly from the distribution that you care about. You’re going to think about possible outcomes, but you’re going to think about them with the probabilities that are associated with those outcomes. So, you sample from that distribution that has the benefit that it gives you an unbiased estimate of the probabilities. But if you’re in a situation where there’s an extreme distribution, a skewed distribution of possible utilities, and where there are low probability events that have extreme negative utility, that strategy doesn’t work very well. The reason why is that the estimate that you end up getting is one that has a huge amount of variance. So, from one set of samples to another, there can be a very big difference in the value that you get because you may or may not have included those extreme events.
If we’re dealing with small samples, it’s not the bias in the estimate that’s going to be the problem, it’s literally the variance in the estimate that is going to end up killing us. So if you were going to make a decision about whether you should play a game of Russian roulette and there is a revolver in front of you with six places for bullets, one of which is actually a bullet, you can calculate how many samples you need to draw in order to be 99.9 percent sure that you should not play this game. It’s something like 51 samples. There’s a lot of variability there. You want to be able to make that decision much more quickly.
So, what’s the distribution from which we should sample in order to minimize that variance? There's a nice result which says that that distribution is proportional to the probability of an event occurring multiplied by the absolute value of the utility of that outcome. And this results in a biased estimate. It’s biased in the direction that it’s going to overestimate the probability of extreme events, but it’s the estimate that reduces variance. If you’re trying to do the least computation you can do while minimizing the chance that you end up accidentally killing yourself, this turns out to be the best strategy. It’s a resource rational strategy.
So, what you do is you wander around the world, you encounter those events and you remember those events as you encounter them. But the probability that you remember them or the probability that you retrieve them from memory is proportional to the absolute value of their utility. That’s a mechanism that instantiates that kind of stuff.
Herb Simon talked about this idea of bounded rationality, but he was very reluctant to define what bounded rationality meant. And in fact, there’s a letter that he wrote to Gerd Gigerenzer in which he makes it very clear that that ambiguity was a feature not a bug, that it was intended to make people think about alternatives. The way that I think about it is that the notion of boundedness picked out a subset of the space of possible strategies that you can follow. That’s the optimal strategy. That’s the thing that’s picked out by the classical notion rationality. Bounded rationality says you don’t have the resources to get there, so now there’s a space of alternatives. And then bounded optimality says, out of that space you’re going to be choosing the thing that is going to be the best thing. It gives you back that optimization criterion that gives you a way of then having a theory of behavior which has the generality of that classical notion of rational behavior.
The point of this example is that this is a case where it seems irrational under a classical notion of rationality, but it makes sense under a more realistic characterization of what rational behavior might be like. There are other cases that we work through where we can show that some of these classic heuristics fall out of this approach. And while heuristics result in biases, being biased doesn’t mean that you’re not doing the thing that is resource rational. Bias is a natural tradeoff to accept in order to allow you to operate with limited cognitive resources. Just because you see biased behavior, doesn’t mean that people aren’t doing something that makes sense. It might mean that people are doing something that makes sense, but they’re operating under resource constraints.
This ties back to some of these questions about machine learning that I started out, the immediate question that should come up for you is if people are following these resource rational strategies, how are they discovering them? How are we ending up finding good ways of using the limited cognitive resources that we have? This is a problem that’s called rational meta-reasoning. How do we rationally reason about the strategies that we should be following as agents in terms of the computational resources that we deploy for solving particular problems? We're not just reasoning, we’re reasoning about the way in which we should reason. Again, it's something where you can do a little bit of work and get a lot of leverage.
There are a couple problems with rational meta-reasoning. One is a problem that we call algorithm selection. This is a case where you know what the algorithms are and you’re trying to choose between them. The more interesting case is what we call algorithm discovery, where your goal is to come up with the right algorithm to use to solve a particular problem, putting together these pieces of computation that you’re going to use to solve that problem.
The way that you can approach this is by recognizing that in fact this deployment of an algorithm is itself a decision problem. It’s a sequential decision problem where you’re making a decision about the sequence of computations that you’re going to execute, one after another. Framing it in those terms allows us to tie it back to classical problems that are faced in decision-making, reinforcement learning. You can characterize the problem of selecting what computation to perform through this process as a Markov decision process, which is something that we can solve using these classical tools.
One of the ways in which most machine-learning methods work at the moment is, you take a single monolithic neural network and you throw it at your problem. That’s different from the way in which we’ve been characterizing human cognition as working, which is that you’ve got the pieces of these computations and you're deciding how to put these together to solve different problems. And then in the neural network world, that’s about constructing what they call a computation graph, the sequence of transformations that you’re applying in order to get to a solution. So, you can formulate the problem of constructing the right neural network to use to solve a particular problem as a decision problem in these same terms. And it’s a decision problem about the computations that you’d apply in order to solve a problem in the world.
Using these kinds of tools and engaging with a set of problems in ways that are much broader than the canonical approaches currently being used in machine learning, such as trying to train the same system to find the right cognitive modules to put together in order to solve all of these different problems, gives us a path beyond the current monolithic approach to building these machine-learning systems, and maybe a path towards building more human like AI systems.
* * * *
JOHN BROCKMAN: I have a question vis-à-vis the claims made by the deep-learning community. Where is this going? When you talk about the next revolution, is this going to be AI that we can use?
GRIFFITHS: I’m focused on what I see as the gap between what people do and what current approaches in machine learning do. Whether you want to close that gap is going to depend on what your motives are. If you just want to build the best image classification system, then you don’t have to care about this. But if you want to build a system that is doing a wide range of tasks in the same general way that humans do, then those are the kinds of architectural questions that you want to ask.
ROBERT AXELROD: How would you deal with the idea that humans eat too much sugar and fat? We know the evolutionary reasons why, but the human is told that it's not optimal for your health and they do it anyway. That is not a question of limited calculation, but it is sub-optimal behavior.
GRIFFITHS: I’m not claiming that all instances of sub-optimal behavior are explained in this way. Adopting this perspective changes the way that you should think about debiasing. Classical approaches to debiasing try to make people act in ways that are more rational. But from my perspective that’s not going to work, because the strategy that people are following might be a good strategy given the resources that are available to them, so instead of focusing on modifying people’s strategies, what you do is focus on modifying the structure of the environment around them.
AXELROD: You could also modify their resources. For example, if you're teaching calculus, they can think about rates of change in a way that somebody who is naïve cannot.
GRIFFITHS: You can change the set of computations that they’re able to perform, and as a consequence that can change the strategies.
PETER GALISON: When you were talking about bounded rationality, we often assume that if we don’t have the resources to sample very widely, we’re making poorer decisions than we would have if we could have sampled more widely, but we understand it, because in that circumstance we didn’t have access. So, being scared of a shark attack, we don’t have that information, so we would have realized only a handful of shark attacks in ten years.
Recently, some of my nearest and dearest went swimming at the beach near where we live, where there was a shark attack two days before, and there were seals swimming there. The conditional probability was elevated, partly because there was a bound on accessed information. So sometimes it seems to me that bounded rationality could be better than the universal knowledge because it might include conditional probabilities that were specific to the circumstance.
So, it may be that people are not only being rational when they make these local decisions because their knowledge of the broader universe of data is limited, it may also be because they see something. They add factors that are relevant, or that they suspect might be relevant to their decisions that would make it quite sensible.
GRIFFITHS: Yes. I can give you an interesting example like this, which is we looked at a case in which you're doing decision making over some horizon, but we assume your horizon is limited relative to the problem that you want to solve.
So, say you’re making decisions about a twenty-year trajectory in terms of your career, but you’re only able to see or plan out into the future five years or something like that. Under those circumstances, it turns out that it’s beneficial to be optimistic. So, it’s beneficial to conflate your utility and probability in exactly the way that I’m talking about, where you over-estimate the probability in this case of good events.
And it’s beneficial to be optimistic because if you were only able to plan over that limited horizon and you were being perfectly calibrated to what the probability of events are, you miss the chance to pursue a low probability outcome which has a big payoff in terms of fifteen years beyond the part which you’re able to see. Whereas if you’re optimistic, you erroneously pursue those attractive career opportunities that you’re going to fail at, but the fact that you pursued them put you in the position to be able to then benefit from those in the future.
SETH LLOYD: I’m a little confused about what you mean by sampling here. I never quite understood what people meant by sampling. If you mean look at my past history and count the number of times I was attacked by a shark, then I estimate the probability to be zero. And that’s clearly wrong, because you know that people are attacked by sharks. So, your information is just, "I read in the newspaper that somebody was attacked by sharks."
GRIFFITHS: When we talk about sampling as a cognitive mechanism, we’re pretty agnostic about what the distributions are that you have access to.
LLOYD: Well, what do you actually mean by sampling? Do you mean I am drawing from some set of events with this distribution? How do you draw from that if you’re talking about shark attacks?
GRIFFITHS: The best example of this is cases where you’re able to somehow generate samples from memory, as a consequence of your experience, plus as you’re saying the testimony of others and the things that you’ve read about. And you’re putting all of those together into a distribution that you’re then generating samples from when you’re considering outcomes.
FREEMAN DYSON: Is this just simply a formalized version of Kahneman's Thinking, Fast and Slow?
GRIFFITHS: What psychology has done very well, and what Kahneman's work is a good example of, is characterizing a wide range of circumstances where people behave in ways that deviate from this classical notion of rationality. What psychology has not done particularly well is develop a formal theory with the same generality as that classical theory of rationality that actually explains in particular circumstances how people are going to act.
F. DYSON: So, your answer is yes?
GRIFFITHS: My answer is, it would be very nice if we were able to formally express those things. And that’s the kind of goal that we have in mind, yes.
ALISON GOPNIK: So, it’s doing prospect theory right, right? The point is that prospect theory was supposed to be here’s the positive theory that’s the counterbalance to that.
GRIFFITHS: Prospect theory is still a descriptive theory that says that people are making decisions using this function to characterize probability, this function to characterize utilities, those are empirically derived functions. So, the kind of thing that you want to be able to do is to say we can derive from this why people are taking actions in this particular way. And I’ll say that the method that I talked about, the sampling-based method, that actually predicts people’s decisions in an empirical choice prediction setting better than prospect theory does. So that’s just starting from first principles and driving the solutions to those problems.
GOPNIK: It is true that this gives you a good explanation of why people are acting the way they do, but another respect in which people seem to be superior to AIs gets back to this point about getting at vertical representations of what’s going on in the world around us, notably in science. Is that bug of having limited computational bounds actually a feature when it comes to extracting the structure of the world around us?
GRIFFITHS: It’s a feature because it forces us to be good at this kind of metacognition. If you think about the kinds of things that AI systems currently struggle with, one of these examples is being able to find a reasonable sub-goal in a reinforcement-learning task. If you look at the set of computer games that AI systems can play better than people, and then you look at the ones where they fail, the ones where they fail are the ones where you have to formulate some kind of abstract goal, like "I’m going to get the key so I can open the door, which is going to appear on two screens."
But the human ability to do that is entirely a consequence of the fact that we have limited computation. If you were able to see an arbitrary distance into the future, you don’t need to formulate sub-goals. You just follow the optimal policy. It doesn’t need any decomposition to do that. Decomposing the problem in that way is what you do when you try and solve it with less computation. So, being able to form those abstractions allows you to represent the problem in a way where you’re able to solve it, even though you’re only able to consider three moves a second.
IAN MCEWAN: Are you saying what seems like cognitive defects are actually useful features because we’re all descended from people who weren’t eaten by sharks? We must be getting something right.
GRIFFITHS: I’d say what seemed like cognitive defects are defects in the sense that they’re a consequence of the limitations that we operate under. But the fact that we operate under limitations means that we’ve had to develop the kind of cognition which is not well represented in current AI systems, which is being able to reason about how to use the bit of smarts that we’ve got to solve a wider range of problems.
MCEWAN: There seems to be something interestingly parallel with Alison Gopnik’s talk of a push to humanize AIs. To get them to do something more like what we do, if they’re going to live among us.
GRIFFITHS: That's another problem. It’s important that we have a theory of how they work in order for us to interact with them, but it’s even more important that they have a theory of how we work in order for them to interact with us.
One of the surprising things that we discovered is that expected utility theory is a terrible model of how people act, but it’s a good model of how people think other people are going to act. We have a theory of mind, but our theory of mind is flawed in that we think that people are more rational than they are. If you wanted to make a machine that could reason well about how to interpret human actions, it would be nice if we were able to do so in a way that took into account cognitive limitations as well.
MCEWAN: In the model of my Adam, he understands humans by reading world literature. And since he’s got a very good memory, the totality of world literature, imaginative literature, is not a prescription but a description of humans are. It doesn’t necessarily provide solutions, it just takes us through all the moral corners that people have been able to imagine.
GRIFFITHS: Yes. And that’s pretty consistent with the way that current machine-learning methods work, which is that you drown them in a huge amount of data and they’re able to memorize the relevant aspects of the data and generalize, but they're not necessarily forming a systematic theory that they’re able to use to generalize to new circumstances.
GOPNIK: There are reasons why his reading world literature as opposed to just taking in all the things that are in the papers might be an advantage. If you think about trying to see what the boundary cases are, so you’re trying to figure out the structure of a particular theory, trying to figure out what the consequences of it are, you’re often better off thinking about non-existent boundary cases than you are thinking about the things that you see all the time.
If you think about Einstein trying to explain his theory, if he just said, "Well, look, here’s a prediction of the theory. If you drop this, it will fall at this particular speed according to my theory." That would not be very informative. Having these fictional boundary cases seems to be a better way of telling you, "Here’s the big important differences between my theory and other theories."
If you’re thinking about human beings, for instance, it might be that having these fictional extreme boundary cases, which is what you typically find in world literature, is a better way of knowing what people’s psychological structure is, or at least what people’s theory of their own psychological structure is than it would be if you just looked at all the things that were in the newspapers.
MCEWAN: The sum of all the things that didn’t happen is near infinite. The possibility for world literature as a mental space is infinite.
LLOYD: Speaking from my experience teaching the "Miracle Methods of Probabilities" to undergraduate mechanical engineers, which is what they learn, everything obeys a central limit theorem, the deviations are ne Gaussian. In a Six Sigma event, like in Six Sigma management, Six Sigma means it has a probability of one in ten to the 12th of occurring. But from experience, we know that distributions have fat tails. There are power laws. They are lognormal distributions. And Six Sigma events occur all the time. If you designed a bridge thinking everything is just a Gaussian distribution and Six Sigma’s never going to occur, then you're going to have bridges that are falling down right and left. I’m actually just agreeing with you. The fact that shark attacks occur, even though they’re extremely rare, means you probably want to give it some thought that it might happen.
W. DANIEL HILLIS: It seems to me like this notion of bounded computation is also relevant to the discussion, early in the morning of Alison Gopnik’s point about the complexity of goals and the insolvability of goals. I’m very bad at predicting what’s going to make me happy or not sad, so I have goals instead. And what goals are, is they’re basically admissions of failure of my ability to make bad computations. So instead what I try to do is act in a way toward doing something else that’s a surrogate for that, a stand-in for that. And in the same way with machines, when I give machines goals, I can’t really decide what’s going to make me happy for the machine to do, so what I’m doing is I’m using my bounded rationality and establishing a stand-in for that of what would make me happy, for the machine.
GRIFFITHS: We’ve been thinking about ways of helping bounded humans do a better job of achieving their goals if they’re able to specify what they are. It's based on reversing that loop. Instead of having humans define the reward functions for machines, you have machines define the reward functions for humans. We have an approach called optimal gamification. The idea is that you have a sequential problem you want to solve, and we can write it down, but you’re not able to see far enough into the future to work out what the optional policy is, so you can give it to a computer, get the computer to solve that problem, and then we can take the solution that’s computed by the computer and use that to construct and modify the reward function for humans, such that even perfectly myopic humans following the modified reward function will achieve their long term goals.
We have deployed this system which has shown that we can reduce people’s procrastination, procrastination being a classic example where you’ve got a big payoff that’s far into the future, and then the optimal modification of that reward function makes that long term payoff smaller but spreads it out through time so you can follow the bread crumbs and then eventually get there.
HILLIS: This also suggests a modification of the voting algorithm that you suggested earlier, which is the real great democracy would not be the one where you submitted an algorithm showing what your preferences were, but merely you submitted an algorithm that showed what would make you happy and then you, the voting commission, takes all those algorithms, runs them under different scenarios and picks the optimal one.
NEIL GERSHENFELD: I would caution about the three moves per second being misleading. If you view that as an update rate, it’s updating a very high dimensional feature vector. Unlike the move generator you're comparing in Deep Blue. And so, if you look at high dimensional optimization algorithms, the effect of operations per cycle is a huge number because you’re moving this giant feature vector and the three moves per second is the velocity of this high dimensional feature vector, which is way more than three moves a second.
