Pale blue dot

Some time on Friday evening (GMT), the defunct UARS satellite is going to fall to earth. Six tons of blistering, white-hot metal are going to splatter the lower atmosphere and the larger chunks will strike the surface at horrendous speeds.

NASA tells us that there’s only a one in 3,200 chance of any of us being hit, and since there are seven billion of us, I don’t think it’s going to be me.

But out of idle curiosity I thought I’d look up how much of the Earth’s surface is actually occupied by humans. I had to do it by subtraction, from various sources within that amazing oracle of truthiness that is the interwebnet.

I could dismiss 70% of the planet immediately, since everyone agrees that this much of it is ocean. How much is jungle or forest? About one third of the total surface of the earth, apparently. And Wikipedia tells me that one third of the land surface is desert. One third of the 30% that is not ocean works out as 10% of the total surface. I figure deserts don’t contain forests, and oceans contain neither (the polar cap isn’t included in the figure for deserts) so these are mutually exclusive areas. Whatever is left is habitable space.

Okay, so 100% – 70% – 33% – 10% = negative 13%. Hmm. Maybe I need to double-check the figures.

Either way, allowing a decent margin for error, I have to conclude that, at the most optimistic estimate, humans occupy roughly zero percent of the planet, give or take a little. Given that there are an awful lot of us anyway, this means we must be spread out like a spider’s web clinging to a beachball.

Well, maybe not a beachball. If Jupiter is a beachball, Earth is a marble, lost in an emptiness so huge that we’re a barely discernable, sizeless dot in the Jovian night sky. And the star we orbit is just one of two hundred billion in our galaxy alone, and there are about a hundred billion galaxies in the observable universe.

I lost my bike helmet yesterday. It doesn’t seem so important any more.

[Edit: whoops. Corrected the math. D minus.]


Seeing the wood for the trees

A while back I wrote a piece about bonobos and chimpanzees – how different they are and how human political differences might be a reflection of these two ways of life.

One thing that struck me about bonobos is that they are separated from chimpanzees by nothing more than a river. The Congo River is apparently what separated two populations of their common ancestors a couple of million years ago and prevented them from interbreeding. One population went on to become modern chimpanzees and the other bonobos. Once their genes were no longer able to mingle, it was inevitable that they would diverge from each other in both physiognomy and behavior.

What was it about the south side of the Congo that favored collaboration and appeasement instead of dominance and aggression? I have no idea, but it needn’t have been very much at all. The tiniest difference in habitat could lead to a change in culture (such as a shift in the roles of males and females) and this in turn would have knock-on effects. Positive feedback would soon lock in these changes and drive an expanding wedge between the two populations.

In modern humans, chimpanzee-like right-wing behaviors and bonobo-like left-wing behaviors coexist, but very uneasily. Empathy, for instance, serves different purposes in each mode: “socialism” (with a small “s”) is fundamentally based upon empathy in the form of sympathy – the understanding that other people suffer like we do, and if we help and support each other we can minimise this suffering for all. “capitalism”, meanwhile, makes use of empathy to outwit other people. A CEO who can walk into a business meeting and immediately grasp what everyone around the table is thinking will come away with a better deal. The consequences of this difference are profound. To a libertarian conservative, for instance, government is an unwanted imposition – a Them who controls Us. It’s an Alpha Male to be feared, opposed and ideally got rid of. Meanwhile, from the perspective of a liberal, the government actually is us; it is the collective will; the way we look out for each other. It’s no wonder the two sides fail to understand each other. In America and the UK this tension is very strong at the moment and it sometimes makes me feel that humans must be descended from the interbreeding of two previously separated species, because the two points of view aren’t very compatible and evolution might have been expected to opt for either one or the other. Bonobos and chimpanzees certainly did.

All this came back into my mind this morning when I read this article in Machines Like Us. The gist of it is that Australopithecus afarensis appears to have walked upright on two feet, in roughly the front-of-foot way that we humans do, rather than the bowlegged way that other primates do. And they did this almost four million years ago at the latest – around the time the human bloodline separated from the chimp/bonobo bloodline.

It made me wonder what kind of “Congo river” might have separated the two lines, and it’s really not hard to imagine. Chimpanzee and orangutan feet are designed for living in trees – their mastery of the arboreal mode of transport is astounding from the perspective of a human being, whose feet are utterly useless for dangling from branches. Every time I watch a primate leap confidently from branch to branch I find myself in awe and not a little envious.

But suppose the trees thin out? There are clear limits to how far apart branches can be whilst still being able to support two hundred pounds of leaping flesh. When trees get too thin on the ground, primates have to climb down and walk. For a quick dash, followed by a rapid climb back into safety, chimpanzee feet are ideal, but there will come a point when efficient running becomes far more important than efficient climbing and leaping. There are no tigers in the trees (which is basically why primates live in them), so being a bit ungainly in the canopy is not nearly as serious as being unable to reach the safety of the next trunk. The evolutionary advantage of good running feet would very quickly be tested, once running became necessary.

And what then? Once you perform better on the ground than in the canopy, you can free your hands. You have to watch out more carefully for predators and find ingeneous ways to thwart them (even using sticks as weapons, maybe). Sex becomes different. Meetings tend to happen face-to-face instead of face-to-ass. Perhaps females carrying young need protection. You are presented with vistas that exceed a mere wall of leaves. A thousand things have suddenly changed, and each of those thousand things would go on to create a thousand other changes. And all because the trees got too far apart to leap between.

Perhaps this was all it took to make the human race? Perhaps we’re just the descendants of incompetent leapers who had to evolve bizarre and expensive tricks like literature and intelligence in order to survive on the ground when we could no longer stay hidden in the trees. As we dash (by elevator) from the safety of our office-trees to the safety of our house-trees and climb the wooden stairs to bed, on feet and hips that are very much designed for the ground, it’s sobering to think that most of what we see around us might have been caused by a bit of a lingering drought, four million years ago.

Maybe I should go for a run…


If anyone’s wondering why they can’t get to the Grandroids pages on Kickstarter, it’s because they’re having server upgrade problems. Should be back online soonish.

[Edit: Ok, they’re back up again now.]

Grand falls at Grand Falls

Sorry, I couldn’t resist the headline. But it’s okay – I didn’t hurt myself. I just slipped on some lava. Yesterday I took the day off and went to a place called Grand Falls, so I thought I’d share it with you.
Arizona is stuffed full of gems that they hardly ever bother to tell you about. I guess if you have the Grand Canyon in your backyard it’s kind of embarrassing to admit to all the other wonders of the world that you have tucked away in corners. Anyway, this little marvel is actually in the Navaho Nation, some miles down an unsignposted gravel track called Indian Road 70. It’s in the volcano field just north-east of Flagstaff, among huge, arid cinder cones and on the edge of the Painted Desert, precisely where you don’t expect to find a waterfall that’s taller than Niagara Falls. I had absolutely no idea it was there, a few miles from my apartment, until my geology book just happened to fall open at the right page yesterday morning.

The San Francisco Peaks, with a couple of small cinder cones in the foreground

Grand Falls is on the chocolate-colored Little Colorado River – the baby sister of the one that made the Grand Canyon. Around 20,000 years ago, one of the nearby volcanoes erupted and spewed a tongue of lava across the desert, blocking the river canyon completely. The river wasn’t having any of this, so it found another way round, and ended up spilling in from the side of its former canyon. It dries up in the Summer, but at this time of year the snowmelt creates a series of cascades, followed by two stages of falls, totalling 185 feet.

Visitors sun themselves on spray-nourished grass, oblivious to the huge wall of lava advancing from behind

While I was at the lip of the canyon I took a pleasant stroll along the beach. What? In Arizona? Well yes, except that the beach was 225 million years old. The mudstones here were formed on a muddy coastal plain, and their original bedding planes are exposed once more on the desert floor. I was hoping for some dinosaur footprints, but no such luck. Even so, it’s incredibly satisfying to sit on a mud surface that formed so long ago and still see little pockets of shells, animal burrows and dessication cracks from the scorching Triassic sun. What a joy to be able to step back in time and sunbathe on a beach that has barely changed since a quarter of a billion years before humans evolved! Alright, so the sea is now hundreds of miles away and there’s sagebrush and tumbleweed where once there were cycads, but I could practically smell the brine and hear the plopping of muddy reptilian feet.

These hard pebbles were rounded slowly by waves, 225,000,000 years ago, and are now high and dry in the desert.

A little hollow in the mud had filled with sea shells. I wonder what made it?

Dessication cracks, formed while the mud was still soft (notice the lips on the crack sides). It looks like things may have been slithering about over the surface, too

Something lived here that burrowed into the mud between tides


Anyone know what these nodules were formed by? There were lots of them.

[Update on the nodules above: Norm tells me they may be concretions cemented by hematite that are known around here as ‘moqui marbles’ and possibly similar to purer hematite concretions found on Mars!]

The falls itself is an amazing place. Anywhere else it would be a major tourist attraction, but it’s on Indian land, far from the highway, and the locals seem quite happy to keep it to themselves. There’s not a lot of water in Arizona, so I’m not surprised! In another 20,000 years the river will probably have reclaimed its course and worn away the falls, so catch it while you can, I say.

A sense of the scale - you can see this boulder on the right of the first photo

Cascades of chocolate

Of camels and committees

I did a Biota podcast last night and Tom understandably asked me a little about my views on open source and collaborative development. I didn’t give a very good answer, but the subject keeps coming up, lately, so I thought I’d write a post about it to try to explain my position. People want to know why I don’t plan to develop my game as open source. Why don’t I collaborate with others (often specifically the person asking the question) and hence do a far better job than I can possibly do on my own? Why am I so opposed to teamwork? Why am I so stuck up and antisocial? (Alright, nobody actually asks that, but sometimes I suspect that’s what they’re thinking.)

I’m really not opposed to collaboration. Not at all. Nor open source. It just doesn’t work well for me personally, and in particular for this application. Collaboration is the norm, so it’s not like I’m discriminating against a minority here. It’s practically compulsory in many areas. Just try getting a European Commission science grant without including at least three different countries in the team. If it weren’t for Kickstarter and you lovely generous people I’d have little hope of getting my work funded at all, and for over a decade I’ve had to fund most of it myself. But that doesn’t mean collaboration is necessarily always the best way to go about things.

In the case of my Grandroids project, writing a computer game isn’t the objective, it’s the intended outcome. These are actually very different things. For instance, the intended outcome for the Kon-Tiki expedition was to arrive at the Tuamoto Islands, but it wasn’t the objective. If it was the objective then Thor Heyerdahl could simply have got on a plane. Any decent pan-European research collaboration could have told him that. At least after a few committee meetings to thrash out the reporting requirements.

If the game I’m writing was merely the objective then a bunch of us could sit down and discuss how we were going to achieve it. But for me it’s very much the other way round. I already have a theory that I’m trying to develop, and the game is intended to be an entertaining and useful expression of that theory. But the theory is in my head; it isn’t fully developed yet, and so I can’t delegate parts of it or even explain it properly to people. It therefore has to be a conversation between me and a computer.

And it’s not like I can even farm out the peripheral stuff. Not yet, anyway. The graphics and physics engines could be farmed out if it weren’t for the fact that they’re already written and I’ve bought the licence (in any case, without them I couldn’t do my part, so they had to come first). Even the 3D creature design is a biological issue, not predominantly an artistic one, because I’m using the physics engine and virtual muscles to control it, rather than conventional animation, so the weight distribution and anatomy have to work hand-in-hand with the muscle control system, which in turn is very co-dependent on how the brain is developed. If someone designs a beautiful creature but when I plug it into my code it keeps falling over, it’s not going to be held up by Art alone. Whereas if I develop the 3D art as well as designing the low-level postural control in the brain, my left hand can learn from my right and vice versa. These iterations occur on a minute-by-minute basis and I get a direct, personal insight into both the art and neuroscience problems that I would never have been able to take advantage of if someone else had done the graphics. This is why I’ve been building robots by myself, too. It was developing the electronics and signal processing that gave me insights and ideas into how the human brain might work, and it was neuroscience and biology that gave me new ideas about how to design the electronics and mechanics. Those intimate connections between apparently disparate ideas are the fuel for creativity. The creative act is primarily an act of analogy.

And all that has to happen inside a single brain, because in the brain ideas can connect up in myriad ways that aren’t confined to language and drawings. I don’t have any translation problems in my head; I don’t send memos to myself and then misread them; I understand every single word I say, which is rarely the case when I’m discussing things with other people. If I was a painter, this would be far more self-evident. It’s not like  Michelangelo could have restricted himself to painting the faces on the Sistine Chapel ceiling while other team members chose the layout, focus-grouped the storyline, painted the arms, etc. It had to be a single creative act. Although now that I think about it, perhaps that explains the Venus de Milo…

In computing terms it’s somewhat similar to Linux. Zillions of people can maintain Linux and add to it now, but the core of it had to come out of Linus Torvalds’s head. Yet, even then, people already knew what an operating system was and roughly how to go about designing one. That’s far from the case in AI. We know hundreds of ways not to do it, but how to actually achieve it is still an open question. There are plenty of other, often well-funded attempts to sit round a table and figure out how to create AGI collaboratively, so if that’s the best way to go about it we’ll soon find out. But sometimes a better way to search an area is for everyone to spread out and follow their own nose. I have a specific route that I want to follow, I can’t explain it to anyone else in a way that would enable them to see exactly what I have in my mind, so it’s best for me if I just stay in my hermitage and write code. Sometimes code is the best way to explain an idea.

So, I really have nothing against collaboration or open source software per se, although if you’d asked me that yesterday morning, while I was up to my neck in CentOS, I might well have given a different answer.

Mappa Psyche

I’m kind of feeling my way, here, trying to work out how to explain a lifetime of treading my own path, and the comments to yesterday’s post have shown me just how far apart we all wander in our conceptual journey through life. It’s difficult even to come to shared definitions of terms, let alone shared concepts. But such metaphors as ‘paths’ and ‘journeys’ are actually quite apt, so I thought I’d talk a little about the most important travel metaphor by far that underlies the work I’m doing: the idea of a map.

This is trivial stuff. It’s obvious. BUT, the art of philosophy is to state the blindingly obvious (or at least, after someone has actually stated it, everyone thinks “well that’s just blindingly obvious; I could have thought of that”), so don’t just assume that because it’s obvious it’s not profound!

So, imagine a map – not a road atlas but a topographical map, with contours. A map is a model of the world. It isn’t a copy of the world, because the contours don’t actually go up and down and the map isn’t made from soil and rock. It’s a representation of the world, and it’s a representation with some crucial and useful correspondences to the world.

To highlight this, think of a metro map instead, for a moment. I think the London Underground map was the first to do this. A metro map is a model of the rail network, but unlike a topographic map it corresponds to that network only in one way – stations that are connected by lines on the map are connected by rails underground. In every other respect the map is a lie. I’m not the only person to have found this out the hard way, by wanting to go from station A to station B and spending an hour travelling the Tube and changing lines, only to discover when I got back to the surface that station B was right across the street from station A! A metro map is an abstract representation of connectivity and serves its purpose very well, but it wouldn’t be much use for navigating above ground.

A topographical map corresponds to space in a much more direct way. If you walk east from where you are, you’ll end up at a point on the map that is to the right of the point representing where you started. Both kinds of map are maps, obviously, but they differ in how the world is mapped onto them. Different kinds of mapping have different uses, but the important point here is that both retain some useful information about how the world works. A map is not just a description of a place, it’s also a description of the laws of geometry (or in the case of metro maps, topology). In the physical world we know that it’s not possible to move from A to B without passing through the points in-between, and this fact is represented in topographical maps, too. Similarly, if a map’s contours suddenly become very close together, we know that in the real world we’ll find a cliff at this point, because the contours are expressing a fact about gradients.

So a map is a model of how the world actually functions, albeit at such a basic level that it might not even occur to you that you once had to learn these truths for yourself, by observation and trial-and-error. It’s not just a static representation of the world as it is, it also encodes vital truths about how one can or can’t get from one place to another.

And of course someone has to make it. Actually moving around on the earth and making observations of what you can see allows you to build a map of your experiences. “I walked around this corner and I saw a hill over there, so I shall record it on my map.” A map is a memory.

Many of the earliest maps we know of have big gaps where knowledge didn’t exist, or vague statements like “here be dragons”. And many of them are badly distorted, partly because people weren’t able to do accurate surveys, and partly because the utility of n:1 mapping hadn’t completely crystallized in people’s minds yet (in much the same way that early medieval drawings tend to show important people as larger than unimportant ones). So maps can be incomplete, inaccurate and misguided, just like memories, but they still have utility and can be further honed over time.

Okay, so a map is a description of the nature of the world. Now imagine a point or a marker on this map, representing where you are currently standing. This point represents a fact about the current state of the world. The geography is relatively fixed, but the point can move across it. Without the map, the point means nothing; without the point, the map is irrelevant. The two are deeply interrelated.

A map enables a point to represent a state. But it also describes how that state may change over time. If the point is just west of a high cliff face, you know you can’t walk east in real life. If you’re currently at the bottom-left of the map you know you aren’t going to suddenly find yourself at the top-right without having passed through a connected series of points in-between. Maps describe possible state transitions, although I’m cagey about using that term, because these are not digital state transitions, so if you’re a computery person, don’t allow your mind to leap straight to abstractions like state tables and Hidden Markov Models!

And now, here’s the blindingly obvious but really, really important fact: If a point can represent the current state of the world, then another point can represent a future state of the world; perhaps a goal state – a destination. The map then contains the information we need in order to get us from where we are to where we want to go.

Alternatively, remembering that we were once at point A and then later found ourselves at point B, enables us to draw the intervening map. If we wander around at random we can draw the map from our experiences, until we no longer have to wander at random; we know how to get from where we are to where we want to go. The map has learned.

Not only do we know how to get from where we are to where we want to go, but we also know something about where we are likely to end up next – the map permits us to make predictions. Furthermore, we can contemplate a future point on the map and consider ways to get there, or look at the direction in which we are heading and decide whether we like the look of where we’re likely to end up. Or we can mark a hazard that we want to avoid – “Uh-oh, there be dragons!”. In each case, we are using points on the map to represent a) our current state, and b) states that could exist but aren’t currently true – in other words, imaginary states. These may be states to seek, to avoid or otherwise pay attention to, or they might just be speculative states, as in “thinking about where to go on vacation”, or “looking for interesting places”, or even simply “dropping a pin in the map, blindfold.” They can also represent temporarily useful past states, such as “where I left my car.” The map then tells us how the world works in relation to our current state, and therefore how this relates functionally to one of these imagined states.

By now I imagine you can see some important correspondences – some mappings – between my metaphor and the nature of intelligence. Before you start thinking “well that’s blindingly obvious, I want my money back”, there’s a lot more to my theories than this, and you shouldn’t take the metaphor too literally. To turn this idea into a functioning brain we have to think about multiple maps; patterns and surfaces rather than points; map-to-map transformations with direct biological significance; much more abstract coordinate spaces; functional and perceptual categorization; non-physical semantics for points, such as symbols; morphs and frame intersections; neural mechanisms by which routes can be found and maps can be assembled and optimized… Turning this metaphor into a real thinking being is harder than it looks – it certainly took me by surprise! But I just wanted to give you a basic analogy for what I’m building, so that you have something to place in your own imagination. By the way, I hesitate to mention this, but analogies are maps too!

I hope this helps. I’ll probably leave it to sink in for a while, at least as far as this blog is concerned, and start to fill in the details later, ready for my backers as promised. I really should be programming!

Introduction to an artificial mind

I don’t want to get technical right now, but I thought I’d write a little introduction to what I’m actually trying to do in my Grandroids project. Or perhaps what I’m not trying to do. For instance, a few people have asked me whether I’ll be using neural networks, and yes, I will be, but very probably not of the kind you’re expecting.

When I wrote Creatures I had to solve some fairly tricky problems that few people had thought much about before. Neural networks have been around for a long time, but they’re generally used in very stylized contexts, to recognize and classify patterns. Trying to create a creature that can interact with the world in real-time and in a natural way is a very different matter. For example, a number of researchers have used what are called randomly recurrent networks to evolve simple creatures that can live in specialized environments, but mine was a rather different problem. I wanted people to care about their norns and have some fun interacting with them. I didn’t expect people to sit around passively watching hundreds of successive generations of norns blundering around the landscape, in the hope that one would finally evolve the ability not to bump into things.

Norns had to learn during their own lifetimes, and they had to do so while they were actively living out their lives, not during a special training session. They also had to learn in a fairly realistic manner in a rich environment. They needed short- and long-term memories for this, and mechanisms to ensure that they didn’t waste neural real-estate on things that later would turn out not to be worth knowing. And they needed instincts to get them started, which was a bit of a problem because this instinct mechanism still had to work, even if the brains of later generations of norns had evolved beyond recognition. All of these were tricky challenges and it required a little ingenuity to make an artificial brain that was up to the task.

So at one level I was reasonably happy with what I’d developed, even though norns are not exactly the brightest sparks on the planet. At least it worked, and I hadn’t spent five years working for nothing. But at another level I was embarrassed and deeply frustrated. Norns learn, they generalize from their past to help them deal with novel situations, and they react intelligently to stimuli. BUT THEY DON’T THINK.

It may not be immediately obvious what the difference is between thinking and reacting, because we’re rarely aware of ourselves when we’re not thinking and yet at the same time we don’t necessarily pay much attention to our thoughts. In fact the idea that animals have thoughts at all (with the notable exception of us, of course, because we all know how special we are) became something of a taboo concept in psychology. Behaviorism started with the fairly defensible observation that we can’t directly study mental states, and so we should focus our attention solely on the inputs and outputs. We should think of the brain as a black box that somehow connects inputs (stimuli) with outputs (actions), and pay no attention to intention, because that was hidden from us. The problem was that this led to a kind of dogma that still exists to some extent today, especially in behavioral psychology. Just because we can’t see animals’ intentions and other mental states, this doesn’t mean they don’t have any, and yet many psychological and neurological models have been designed on this very assumption. Including the vast bulk of neural networks.

But that’s not what it’s like inside my head, and I’m sure you feel the same way about yours. I don’t sit here passively waiting for a stimulus to arrive, and then just react to it automatically, on the basis of a learned reflex. Sometimes I do, but not always by any means. Most of the time I have thoughts going through my mind. I’m watching what’s going on and trying to interpret it in the light of the present context. I’m worrying about things, wondering about things, making plans, exploring possibilities, hoping for things, fearing things, daydreaming, inventing artificial brains…

Thinking is not reacting. A thought is not a learned reflex. But nor is it some kind of algorithm or logical deduction. This is another common misapprehension, both within AI and among the general public. Sometimes, thinking equates to reasoning, but not most of the time. How often do you actually form and test logical propositions in your head? About as often as you perform formal mathematics, probably. And yet artificial intelligence was founded largely on the assumption that thinking is reasoning, and reasoning is the logical application of knowledge. Computers are logical machines, and they were invented by extrapolation from what people (or rather mathematicians, which explains a lot) thought the human mind was like. That’s why we talk about a computer’s memory, instructions, rules, etc. But in truth there is no algorithm for thought.

So a thought is not a simple learned reflex, and it’s not a logical algorithm. But what is it? How do the neurons in the brain actually implement an idea or a hope? What is the physical manifestation of an expectation or a worry? Where does it store dreams? Why do we have dreams? These are some of the questions I’ve been asking myself for the past 15 years or so. And that’s what I want to explore in this project. Not blindly, I should add – it’s not like I’m sitting here today thinking how cool it will be to start coming up with ideas. I already have ideas; quite specific ones. There are gaps yet, but I’m confident enough to stick my neck out and say that I have a fair idea what I’m doing.

Explaining how my theories work and what that means for the design of neural networks that can think, are things that will take some explaining. But for now I just wanted to let you know the key element of this project. My new creatures will certainly be capable of evolving, but evolution is not what makes them intelligent and it’s not the focus of the game. They’ll certainly have neural network brains, but nothing you may have learned about neural networks is likely to help you imagine what they’re going to be like; in fact it may put you at a disadvantage! The central idea I’m exploring is mental imagery in its broadest sense – the ability for a virtual creature to visualize a state of the world that doesn’t actually exist at that moment. I think there are several important reasons why such a mechanism evolved, and this gives us clues about how it might be implemented. Incidentally, consciousness is one of the consequences. I’m not saying my creatures will be conscious in any meaningful way, just that without imagery consciousness is not possible. In fact without imagery a lot of the things that AI has been searching for are not possible.

So, in short, this is a project to implement imagination using virtual neurons. It’s a rather different way of thinking about artificial intelligence, I think, and it’s going to be a struggle to describe it, but from a user perspective I think it makes for creatures that you can genuinely engage with. When they look at you, there will hopefully be someone behind their eyes in a way that wasn’t true for norns.