I think we can divide the space of possible AI minds into two reasonably distinct categories. One category comprises the “passive AI minds” that seemed to be the main focus of the Chalmers-Dennett exchange. These are driven by large data sets and optimize their performance relative to some externally imposed choice of “objective function” that specifies what we want them to do—win at GO, or improve paperclip manufacture. And Dennett and Chalmers are right—we do indeed need to be very careful about what we ask them to do, and about how much power they have to implement their own solutions to these pre-set puzzles.
The other category comprises active AIs with broad brush-strokes imperatives. These include Karl Friston’s Active Inference machines. AI’s like these spawn their own goals and sub-goals by environmental immersion and selective action. Such artificial agents will pursue epistemic agendas and have an Umwelt of their own. These are the only kind of AIs that may, I believe, end up being conscious of themselves and their worlds—at least in any way remotely recognizable as such to us humans. They are the AIs who could be our friends, or who could (if that blunt general imperative was played out within certain kinds of environment) become genuine enemies. It is these radicalized embodied AIs I would worry about most. At the same time (and for the same reasons) I’d greatly like to see powerful AIs from that second category emerge. For they would be real explorations within the vast space of possible minds.
I was enthralled by Dennett and Chalmers‘ recent discussion of the threats and prospects regarding artificial superintelligences. Dennett thinks we should protect ourselves by doing all we can to keep powerful AIs operating at the level of suggestion-making tools, while Chalmers is impressed by the market forces that will probably push us into devolving more and more responsibility to these opaque and alien minds. But I felt as if their picture of the space of possible AI minds could be usefully refined, and with that in mind I’d like to push on two further dimensions.
The first is action. Agents that can act on their (real or simulated) worlds can choose “epistemic” actions that both test and improve their model of that world. A simple example might be a robot equipped with a camera and an arm that can push and prod objects in its field of vision. Such a robot can actively create sensorimotor flows that help reveal objects as integrated wholes distinct from their backgrounds and from other objects. These systems, simple versions of which have been explored by Giorgio Metta and others, possess a crucial but under-appreciated capacity, which is to use their own worldly actions to refine or disambiguate information both for learning and during practical action.
The second dimension is the shape of the objective function. For these kinds of “active AI minds” may best be driven by a very general kind of imperative or objective function (my own favourite being try to minimize the expected prediction error of future outcomes). Minds like that inhere in systems that, like us animals, begin with some baseline knowledge, form, and a few intrinsic drives or expectations (to eat, be warm, hang out with others). They then spawn their own future goals, as sculpted by the general-purpose imperative, through embodied exchanges with the world. Such artificial minds (given a well-chosen general imperative) are also curious minds, forever trying to improve what they know by acting in the world.