"All my thinking is for doing," William James said, and it is important to remember what kind of thinking people actually do, in what contexts we do it, and why we do it. And then to compare these with what machines might someday do.
Humans spend between 25% and 50% of our mental life prospecting the future. We imagine a host of possible outcomes, and we imbue most, perhaps each of these prospections with a valence. What comes next is crucial: we choose to enact one of the options. We need not get entangled in the problems of free will for present purposes. All we need to acknowledge is that our thinking in service of doing entails imagining a set of possible futures and assigning an expected value to each. The act of choosing, however it is managed, translates our thinking into doing.
Why is thinking structured this way? Because people have many competing goals (eating, sex, sleeping, tennis, writing articles, complimenting, revenge, childcare, tanning, etc.) and a scarcity of resources for doing them: scarcity of time, scarcity of money, scarcity of effort, and even the prospect of death. So evaluative simulation of possible futures is one of our solutions to this economy; this is a mechanism that prioritizes and selects what we will do.
It is not just external resources that are scarce. Thinking itself uses up costly and limited energy and so it relies heavily on shortcuts and barely justified leaps to the best explanation. Our actual thinking is woefully inefficient: the mind wanders, intrusions rise unbidden, and attention is continually only partial. Thinking rarely engages the exhausting processes of reasoning, deliberating, and deducing.
The context of much of our thinking is social. Yes, we can deploy thinking to solve physical problems and to crunch numbers, but the anlage, as Nick Humphreys reminds us, is other people. We use our thinking to do socially: to compete, to co-operate, to convene the courtroom of the mind, to spin and to persuade.
I don't know much about the workings of our current machines. I do not believe that our current machines do anything in James's sense of voluntary action. I doubt that they prospect possible futures, evaluate them, and choose among them; although perhaps this describes—for only a single, simple goal—what chess playing computers do. Our current machines are somewhat constrained by available space and electricity bills, but they are not primarily creations of scarcity with clamorously competing goals and extremely limited energy. Our current machines are not social: they do not compete or co-operate with each other or with humans, they do not spin, and they do not attempt to persuade.
I know even less about what machines might someday do. I imagine, however, that a machine could be built with the following properties:
• It prospects and evaluates possible futures
• It has competing goals and it selects among competing actions and competing goals using those evaluations
• It has scarce resources and so must forgo some goals and actions as well as options for processing and so it uses shortcuts
• It is social: it competes or co-operates with other machines or with humans, it spins and it attempts to persuade people.
That kind of machine would warrant discussion of whether it had civil rights, whether it had feelings, or whether it was dangerous or even a source of great hope.