Brainstorm #3 – cheating doesn’t pay (sometimes)
April 22, 2010 46 Comments
I was going to write about self-organizing maps and associative links next but I need to make a little detour.
One of the quandaries when making a virtual artificial creature, as opposed to a robot, is how much to cheat. In software, cheating is easy, while simulating physical reality is hard. And I’m writing a video game here – I have severe computational constraints, user expectations (like the need to simulate many creatures simultaneously), and a very limited development schedule. Hmm… So let’s cheat like mad!
Oh, but… For one thing, cheating is cheating. I think Creatures was successful in large part because I was honest. I did my genuine best (given the constraints of technology and knowledge) to make something that was really alive, and this time I plan to be even more tough on myself and do my best to create something that really thinks and might even be conscious.
There’s also an intellectual reason not to cheat more than I can help, though. Cheating doesn’t pay. I’ll walk you through it.
Take vision, for instance. How am I going to handle the creatures’ visual systems? The honest, not-cheating-at-all way would be to attach a virtual camera (or two) to each creature’s head and use the 3D engine to render the scene from the creature’s perspective onto a bitmap. This would act as the creature’s retina, and the neural network would then have to identify objects, etc. from the features in the scene. Well that’s not going to happen. For one thing I can’t afford that much computer power inside a game. For another, it would involve me solving ALL the major challenges of visual science, and even at my most ambitious I can see that’s not going to be feasible between now and next summer.
At the other extreme, in Creatures I simply told the norns the category of the object they were currently looking towards. If they looked towards a ball, the “I can see a toy” neuron was stimulated. If it was a carrot, the “I can see food” neuron lit up. It was the best I could do twenty years ago but it won’t cut it now. So I need something in-between.
But it’s harder than it at first appears. We don’t just use vision for recognizing objects; we use it for locating things in space and for navigating through space. Everyday objects can be treated as simple points, with a given direction and depth from the creature. But a close or large object extends over a wide angle of view. A wall may occupy half the visual field and the objective may be to walk around it. You can’t treat it as a point.
How should my creatures navigate anyway? The obvious way to handle navigation is to use a path-planning algorithm to find a route from here to there, avoiding obstacles. All the information is there in the virtual world for this to happen. Trying to do it from the creature’s own limited sensory information and memory seems like a ridiculous amount of effort that nobody will ever recognize or appreciate.
But here’s the thing:
Relating objects in space involves associations. Forming a mental map of your world is an object lesson in associative memory. Navigating to a target location is remarkably similar to planning, which in turn is remarkably similar to simulating the future, which is the core of conscious experience and the very thing I want to understand and implement. Come to that, navigating is very akin to servoing – reducing the distance between where I am and where I want to be. And for humans at least, this is a nested servo process: To go from the backyard to the shed to get a tool, I need first to go into my kitchen closet and get the key. To get into my kitchen I need to go through the back door, which is in the opposite direction to the shed. Then I have to go to the closet, reach for the key and then backtrack towards the shed. It’s a chain of servo actions and it’s nonlinear (the ultimate goal is reached by first moving away from it). These are precisely the things that I set out in Brainstorm #1 as the features I’m looking for. If I cheated, I might not even have seen the connection between visually-guided navigation and thinking.
In the brain, we know that there are “place fields” in the hippocampus (an older, simpler, curly fringe of the cortex). As far as I know, there’s no evidence (and it doesn’t seem awfully likely) that these “points of best representation” (see Brainstorm #2) are arranged geographically. I’ll have to catch up on the latest information, but it seems like these memories of place form a different kind of relationship and I can’t assume the brain navigates using a map in the conventional, geometrical sense. But somehow geographical relationships are encoded in the brain such that it’s possible for us to figure out (far better than any current computer game) how to get from A to B. We’re capable of doing this with both certain knowledge and uncertainty – navigating around moving obstacles, say, or traveling through unfamiliar territory. This is SO similar to goal-directed planning in general. It’s SO similar to predicting possible futures. All that differs is the kind of associative link (“is to the north-east of” instead of “tends to be followed by” or “is like”). There has to be a coherence to all this.
For a brief moment then I imagined a declarative statement written in PROLOG! God no! Please don’t make the brain a forward-chaining planner or an expert system! It’s interesting that the early days of AI were pretty close to the mark in some ways. Thinking IS a bit like deducing that “Mark is Sheila’s husband” from a set of predicates like “Mark is a man”; “Mark has a son called Peter”, “people who share a child are married”, etc. It IS a bit like a probablistic tree planning algorithm. But these are too abstract, too symbolic, too digital. Navigating through space is an analog process. Reaching out to grab an object is not a discrete, symbolic operation. Being fed a sequence of carefully chosen facts and rules is not the same as learning by experience. And yet…
The reasons why symbolic AI has failed are many and varied, and I don’t have the space or energy. But you can see that the early pioneers were heading in a good direction, thwarted only by some fundamental mistakes and false assumptions about symbol processing and abstraction. It was a fault of the paradigm and tools of both science and mathematics, not intent.
But my point here is that my creatures need to see in a very much more sophisticated way than norns did, and yet a more abstract way than true vision. And I need to find both an external (sensory) and internal (memory) representation that is authentic enough to make visually guided behavior a NATURAL PART OF thinking. The two are so close in concept that they must surely share a mechanism, or at least a set of computational principles. On the one hand this adds new problems – I have to think about navigation, obstacle avoidance, visual binding, retinotopic-to-egocentric conversion, egocentric-to-geographical conversion and a bunch of other things on top of all my other problems. On the other hand, by not cheating (too much, whatever that means) I’m now blessed with a whole new set of symptoms and requirements that give me a better grasp of what must be going on in the brain. It will help me see the central design problem more clearly. This, incidentally, is the reason why we should all be trying to create complete organisms, instead of fooling ourselves that the problem can be divided up and solved piecemeal.
I don’t know the answers to any part of this yet and there will be many future posts on visual representation, etc. But I’m glad I thought about this before starting to think more closely about associative links.