Brainstorm 7: How long is now?

I worry too much. I live too far into the future; always so acutely aware of the potential distant knock-on effects of my actions that I’m sometimes quite paralyzed. On the downside this can be a real handicap, but on the upside it means I’m intelligent, because seeing into the future is what intelligence is for. But how? And how do we differentiate between past, present and future? What do we really mean by “now”?

My main thesis for this project is that the brain is a prediction machine. In other words I think it takes so long for nerve signals to reach the brain and be analyzed by it (you may be surprised to know it takes about a tenth of a second merely for signals to reach the primary visual cortex from the retina, never mind be turned into understanding), that we’d be dead by now if it weren’t for our ability to create a simulation of the world and run it ahead of time, so that we are ready for what’s about to happen instead of always reacting to what has already happened. I’m suggesting that this simulation ability derives, at least in part, from a capacity to make small predictions based on experience, at ALL levels of the nervous system. These little fragments of “if this happens then I suspect that will happen next” are there to counter processing delays and reaction times, and give us the ability to anticipate. But they also (I suggest) provide the building blocks for other, more interesting things: a) our ability to create a contextual understanding of the world – a stable sense of what is happening; b) our ability to form plans, by assembling sequences of little predictions that will get us from a starting state to a goal state; and c) our capacity for imagination, by allowing us to link up sequences of cause and effect in an open-ended way. The capacity for imagination, in turn, is what allows us to be creative and provides the virtual world in which consciousness arises and free thought (unconstrained by external events) can occur.

I rather think some clever tricks are involved, most especially the ability to form analog models of reality, as opposed to simple chains of IF/THEN statements, and the ability to generalize from one set of experiences to similar ones that have never been experienced (even to the extent that we can use analogies and metaphors to help us reason about things we don’t directly understand). But I’d say that the root of the mechanism lies in simple statistical associations between what is happening now and what usually happens next.

So let’s look at a wiring diagram for a very simple predictive machine.

This is the simple touch-sensitive creature I talked about in Brainstorm 6. The blue neurons receive inputs, from touch-sensitive nerve endings, which occurred some milliseconds ago on its skin. The red neuron shows two touch inputs being compared (in this case the cell has become tuned to fire if the right input is present just before the left input). I think we can call the red neuron an abstraction: it takes two concrete “I am being touched” inputs and creates an abstract fact – “I am being stroked leftwards here”. This abstraction then becomes an input for higher-level abstractions and so on.

The green neuron is then a prediction cell. It is saying, “if I’m being stroked leftwards at this point, then I expect to be touched here next.” Other predictions may be more conditional, requiring two or more abstractions, but in this case one abstraction is enough. The strength of the cell’s response is a measure of how likely it is that this will happen. The more often the prediction cell is firing at the moment the leftmost touch sensor is triggered, the stronger the connection will become, and the more often that this fails to happen, the weaker it will become (neurologically I’d hypothesize that this occurs due to LTP and LTD (long-term potentiation and long-term depression) in glutamate receptors, giving it an interesting nonlinear relationship to time).

So what do we DO with this prediction? I’m guessing that one consequence is surprise. If the touch sensor fires when the prediction wasn’t present, or the prediction occurs and nothing touches that sensor, then the creature needs a little jolt of surprise (purple neuron). Surprise should draw the creature’s attention to that spot, and alert it that something unexpected is happening. It may not be terribly surprising that a particular touch sensor fails to fire, but the cumulative effect of many unfulfilled predictions tells the creature that something needs to be worried about, at some level. On the other hand, if everything’s going according to expectations then no action need be taken and the creature can even remain oblivious.

But for the rest of my hypothesis to make sense, the prediction also needs to chain with other predictions. We need this to be possible so that top-down influences (not shown on the diagram) can assemble plans and daydreams, and see far into the future. But I believe there has to be an evolutionary imperative that predates this advanced capacity, and I’d guess that this is the need to see if a trend leads ultimately to pain or pleasure (or other changes in drives). Are we being stroked in such a way that it’s going to hurt when the stimulus reaches a tender spot? Or is the moving stimulus a hint that some food is on its way towards our mouth, which we need to start opening?

Now here comes my problem (or so I thought): In the diagram I’m assuming that the prediction gets mixed with the sensory signal (the green axon leading into the blue cell) so that predictions act like sensations. This way, the organism will react as if the prediction came true, leading to another prediction, and another. Eventually one of these predictions will predict pleasure or pain.

[Technical note: Connectionists wouldn’t think this way. They’d assume that pleasure/pain are back-propagated during learning, such that this first prediction neuron already “knows” how much pleasure or pain is likely to result further down the line, since this fact is stored in its synaptic weight(s). I’m not happy with this. For one thing, thinking is never going to arise in such a system, because it’s entirely reactive. Secondly (and this is perhaps why brains DO think), the reward value for this prediction is likely to be highly conditional upon other active predictions. This isn’t obvious in such a simple model, but in a complete organism the amount of pleasure/pain that ultimately results may depend very heavily on what else is going on. It may depend on the nature of the touch, or have its meaning changed radically by the context the creature is in (is it being threatened or is something having sex with it?). It’s therefore not possible to apportion a fixed estimate of reward by back-propagating it through the network. That sort of thing works up to a point in an abstract pattern-recognition network like a three-layer perceptron, but not in a real creature. In my humble opinion, anyway!]

Oh yes, my problem: So, if a prediction acts as if it were a sensation (and this is the only way it can make use of the subsequent (red) abstraction cells in order to make further predictions) then how does the organism know the difference between what is happening and what it merely suspects will happen??? If all these predictions are chained together, the creature will feel as if everything that might happen next already is happening.

This has bugged me for the past few days. But this morning I came to a somewhat counter-intuitive conclusion, which is that it really doesn’t matter.

What does “now” actually mean? We think of it as the infinitesimal boundary between past and future; between things that are as yet unknown and our memories. But now is not infinitesimal. I realized this in the shower. I was looking at the droplets of water spraying from the shower-head and realized that I can see them. This perhaps won’t surprise you, but it did me, because I’ve become so conditioned now to the view that the world I’m aware of is actually a predictive simulation of reality, not reality itself. This HAS to be true (although now is not the time to discuss it). And yet here I was, looking at actual reality. I wasn’t inventing these water droplets and I couldn’t predict their individual occurrence. Nor was the information merely being used to synchronize my model and keep my predictions in line with how things have actually turned out – I was consciously aware of each individual water droplet.

But I was looking at water that actually came out of my shower-head over a tenth of a second ago; maybe far longer. By the time the signals had caused retinal ganglion cells to fire, zoomed down my optic nerve, chuntered through my optic chiasm and lateral geniculate nucleus, and made their tortuous and mysterious way through my cortex, right up to the level of conscious awareness, those droplets were long gone. So I was aware of the past and only believed I was aware of the present. (In fact, just to make it more complex, I think I was aware of several pasts – the moment at which I “saw” the droplets was different from the moment that I knew that I’d seen the droplets.)

Yet at the same time, I was demonstrably aware of an anticipated present, based upon equally retarded but easier to extrapolate facts. I wasn’t simply responding to things that happened a large fraction of a second ago. If a fish had jumped out of the shower-head I’d certainly have been surprised and it would have taken me a while to get to grips with events, but for the most part I was “on top of the situation” and able to react to things as they were actually happening, even though I wouldn’t find out about them until a moment later. I was even starting bodily actions in anticipation of future events. If the soap had started to slip I’d have begun moving so that I could catch it where it was about to be, not where it was when I saw it fall. But for the most part my anticipations exactly canceled out my processing delays, so that, as far as I knew, I was living in the moment.

So I was simultaneously aware of events that happened a fraction of a second ago, as if they were happening now; events that I believed were happening now, even though I wouldn’t get confirmation of them for another fraction of a second; and events that hadn’t even happened yet (positioning my hands to catch the soap in a place it hadn’t even reached). ALL of these were happening at once, according to my brain; they all seemed like “now”.

Perhaps, therefore, these little predictive circuits really do act as if they are sensations. Perhaps the initial sensation is weak, and the predictions (if they are confident) build up to create a wave of activity whose peak is over a touch neuron that won’t actually get touched until some time in the future. Beyond a certain distance, the innate uncertainty or conditionality of each prediction would prevent the wave from extending indefinitely. Perhaps this blurred “sensation” is what we’re actually aware of. Perhaps for touch there’s an optimum distance and spread. In general, the peak of the wave should lie over the piece of skin that will probably get touched X milliseconds into the future, where X is the time it takes for an actual sensation to reach awareness or trigger an appropriate response. But it means the creature’s sense of “now” is smeared. Some information exists before the event; some reaches awareness at the very moment it is (probably) actually happening; the news that it DID actually happen arrives some time later. All of this is “now.”

Or perhaps not. After all, if I imagine something happening in my mind, it happens more or less in real time, as a narrative. I don’t see the ghosts of past, present and future superimposed. This, though, may be due to the high-level selection process that is piecing together the narrative. Perhaps the building blocks can only see a certain distance into the future. Primitive building blocks, like primary sensations, only predict a few milliseconds. Highly abstract building blocks, like “we’re in a bar; someone is offering me a drink” predict much further into the future, but only in a vague way. To “act out” what actually happens, these abstractions need to assemble chains of more primitive predictions to fill in the details, and so the brain always has to wait and see what happens in its own story, before initiating the next step. I’m not at all sure about this, but I can’t see any other way to assemble a complex, arbitrarily detailed, visual and auditory narrative inside one’s head without utilizing memories of how one thing leads to another at a wide range of abstractions. These memories have to have uses beyond conscious, deliberate thought, and so must be wired into the very process of perception. And in order for them to be chained together, the predicted outcomes need to behave as if they were stimuli.

I’m going to muse on this some more yet. For instance I have a hunch that attention plays a part in how far a chain of predictions can proceed (while prediction in turn drives attention), and I haven’t even begun to think about precisely how these simulations can be taken offline for use as plans or speculations, or precisely how this set-up maps onto motor actions (in which I believe intentions are seen as a kind of prediction). But this general architecture of abstractions and predictions is beginning to look like it might form the basis for my artificial brain. Of course there’s an awful lot of twiddly bits to add, but this seems like it might be a rough starting point from which to start painting in some details, and I have to start somewhere. Preferably soon.

Advertisements

About stevegrand
I'm an independent AI and artificial life researcher, interested in oodles and oodles of things but especially the brain. And chocolate. I like chocolate too.

29 Responses to Brainstorm 7: How long is now?

  1. Ryan Olsen says:

    You have stated you disagree with “connectionist” models and that you don’t think a modular approach makes sense. Here are a few thoughts:

    ==Connectionist
    It seems like it would be hard to argue that our brain doesn’t follow a “connectionist” model in the sense that it is indeed a whole bunch of independent units (cells, proteins, etc.) working together to produce a powerful tool.

    It seems to me that a network of independent units is the computing medium that biology has available, but that doesn’t mean the computable functions that can be built with that medium are somehow restricted to something that resembles a purely reactive system.

    I think our brain takes advantages of the properties of a neural network where it works (pattern matching), and tries to build on top of that where it needs more discrete type functions (symbolic processing).

    ==Modular Approach
    In one of your blogs you objected to a modular approach with some examples of having to account for specific decision making conditions. I agree with you that any kind of enumerating the possibilities, adding exceptions, etc. is not the right path. But I think of a module as something performing a specific function (image processing, object identification) which is used in conjunction with other modules to create the whole working machine.

    ==Other Thoughts
    In an article about people being able to attend to 2 things simultaneously, it mentioned that region of the brain essentially dividing into 2 with each half taking control of a task. Some possible implications:
    1) 2 network structures exist but 1 only takes control of it’s portion of the brain when needed
    2) Structure is not how the function is divided – the brain waves and electrical fields are where the real computing is occurring and these can shift more dynamically than physical structure (although if this is the case why do we see such consistency in which areas of the brain are activated for specific activities)

    It seems like your general principles/brainstorming make sense, but I also think those same principles can be (and for some functions should be because it’s the right tool for the job) built on a neural network in a modular fashion. Not saying you should build it that way in your game, just talking theory.

    • stevegrand says:

      Hi Ryan, let me try to clarify my position on this stuff. I have no problem with the principles of connectionism or modularity – it’s just certain kinds of ideology regarding these subjects that I have objections to.

      I’ll start with modularity. OF COURSE the brain is modular. Neurons are modules; cortical maps are modules; thalamic nuclei are modules. But there’s a strong movement in AI that assumes modules are necessarily functionally decomposable in a certain way, and another doctrine that assumes each module is an ad hoc, separately-evolved circuit. Those are the kinds of modularity I disagree with.

      The first kind can be seen in the design of the digital computer itself: it embodies the functional decomposition of the mind, as seen by psychologists in the mid 20th Century: memory is separate from processing, for example. In the brain, memory and processing are not separate modules at all, and the modularity is broken down differently. It’s not even reasonable to say that we have a cortical module for handling colors, or edges, etc. There’s an assumption that the way we break down problems when we’re thinking about them is the way they’re broken down in the brain, and that’s not very supportable.

      The second kind is a result of the focus on invertebrate nervous systems. In a simple invertebrate, different aspects of behavior are often handled by separate circuits. Each evolved somewhat independently, and each has its own unique wiring diagram and method of solving the problem. There’s a dogma implicit in a lot of new AI that human beings are glorified invertebrates – the same basic principle but thousands of different modules. This is just plain wrong. During the amphibian-reptilian-mammalian evolutionary line (and for that matter the insect line), some quite different kinds of modularity evolved. In insects these massively parallel modules were easy for evolution to tune for different purposes – it didn’t need to reinvent the wheel to add new functions. In mammals some different massively parallel modules developed the ability for vastly more general intelligence – able to learn qualitatively new functions in one lifetime. Studying the modularity of invertebrates is important, but it’s not a prelude to understanding mammalian intelligence. I was with Rod Brooks a few years ago and he was showing a list of the modules for COG. One of them was a “theory of mind module”. Rod’s work is excellent and I’m not knocking it, but I just don’t believe that such modularism makes sense – ToM is not handled by a separate module but arises out of a different kind of organization. Societies don’t have a “democracy module”, for instance. Incidentally, nor do I believe evolutionary systems are going to solve the problems of AI for us, as many of my colleagues do. Not while they’re thinking in that kind of modularity.

      Equally, OF COURSE the brain is a connectionist machine, with a small “c”. I write connectionist simulations. BUT, thanks to its checkered history, Classical Connectionism (large “C”) has followed a bunch of assumptions that are just plain misleading and wrong. Neurons are NOT simply multiply-accumulate units. Biological neural networks are NOT feed-forward, static systems. The problems that beset the original Perceptron and resulted in the three-layer backprop net and its cousins are pretty much irrelevant to biology. The mammalian brain is a highly dynamic system, as you point out, with wave phenomena, temporal attractors and many other things that classical Connectionist networks lack. Neurons have multiple neurotransmitters, are frequency-modulated and have a wide range of nonlinear transfer functions. The tasks that real organisms solve are very different from abstract pattern-recognition. Mammals THINK – their intelligence derives in part from the dynamics of the system, and classical networks HAVE no dynamics.

      So it’s these doctrines and dogmas that I disagree with, not modularity or connectionism per se. I completely agree with you that structure and function don’t necessarily coincide, and that it’s the patterns of activity that do the computing (at least in some circumstances). I am designing a modular system and it is based on a network of neurons (although they may become more abstract later). It’s just not a back-propagation network and the modularity isn’t necessarily decomposable in the way intuition might assume (object identification may not be separate from image processing, for instance). Does that make sense? I wanted to clear that up because I don’t have space in a post to expound my whole philosophy of AI and so my choice of words can make it sound like I’m throwing out the whole of something when I only really have a problem with some aspects – usually dogma.

      • Ryan Olsen says:

        Yes, that makes sense. I was really getting a different impression from earlier blogs compared to what you posted in the reply, and it seemed like you were discarding valuable stuff, but I agree with your last paragraph.

  2. vanilla beer says:

    forgive this red herring but I was thinking about how intuition factors into this and realised that there is an extendable time element there – as in, this’ll-come-in-useful-sometime, meaning, this is going to be significant to me in ways that are not yet clear. Sometimes its simply untrue; sometimes you create the opportunity to make it true; sometimes events conspire to make it truly significant. Thats when you start to think about precognition, intuition, bendy time waves and other distractions.Your anticipation of the falling soap reminds me of Grey Walters identification of a place in a brain wave pattern he called precognitive negative variable (I think )Im sure you know it

    • stevegrand says:

      Not often I hear Grey Walter cited for his EEG work instead of Elmer and Elsie!

      Crumbs, yes, how do we discern the significance of things independently from a present need? Novelty is important, of course, and that’s something I haven’t been thinking about. It’s long struck me as strange that we can tell if something is novel rather independently of our actual memory of that thing. The hippocampus seems to be involved in this. If someone asks me a question I know whether I know the answer, long before I can actually remember the answer! So how do I know that I know it, without accessing the memory to find out? When I say this, most people look at me puzzled and insist that just because it takes me a long time to articulate the answer, doesn’t mean I didn’t have instant instant access to it. But I don’t think that’s true. Nor is it entirely the case that I just know if it’s the KIND of thing I might know. It took me a few seconds to remember the names of Grey Walter’s robots, but I knew instantly that it was worth trying to remember, because I knew that I knew the answer. How did I know I knew the answer without accessing the memory? I’m sure I don’t keep a separate memory of which questions I know answers to, and anyway, why would accessing that be faster than accessing the actual answer? It’s a mystery!

      • Vegard says:

        “It took me a few seconds to remember the names of Grey Walter’s robots, but I knew instantly that it was worth trying to remember, because I knew that I knew the answer. How did I know I knew the answer without accessing the memory? I’m sure I don’t keep a separate memory of which questions I know answers to, and anyway, why would accessing that be faster than accessing the actual answer?”

        Maybe this is an essential part of our mechanism of memory — I have a mental picture of one neuron piping up and saying “Hey, this sounds familiar!”, and sort of alerting its neighbours that they should take a second to think whether it sounds familiar to them too. I think this fits in well with the “hologram” model — the specific memory does not correspond to a specific part of the brain, but we rely on the cascades of “this sounds familiar — is it familiar to you too?” to eventually recall every part of the specific memory, be it the pronunciation of a name or the people involved in an episode.

        Then, when you know that you know the answer, but don’t remember the answer itself, perhaps enough neurons piped up to give you this certainty of knowing the answer, but not the specific ones that are relevant to the information you needed.

        Maybe?

      • Chani says:

        “Then, when you know that you know the answer, but don’t remember the answer itself, perhaps enough neurons piped up to give you this certainty of knowing the answer, but not the specific ones that are relevant to the information you needed.”

        and you remember all sorts of things *around* the answer, such as what you were doing when you learnt it, and where on the page it was, and so on… I think you’re onto something there. 🙂

        actually, sometimes I can use that to sort of sidle up to an elusive memory and peek at it while it thinks I’m not looking – the memory is too weak for me to see it directly, but if I think about enough things that are *related* to it, the general area seems to become stronger and sometimes the part I’m trying to get to eventually appears at the edge of my vision 🙂 if I can figure out what it is without “looking” directly at it, then I’ve got it again and it’ll be safe to look it up directly from then on (until I forget again, of course 😉

  3. Calos Acosta says:

    Hi Steve,

    I don’t know whether I have come around to your way of thinking or you have come around to mine, but either way I like the direction your train of thought is heading. Keep on brainstorming!

  4. Daniel Mewes says:

    Hi Steve,

    thank you for keeping us up-to-date and sharing some of your incredible ideas and thoughts with us.

    “Now here comes my problem (or so I thought): In the diagram I’m assuming that the prediction gets mixed with the sensory signal (the green axon leading into the blue cell) so that predictions act like sensations. This way, the organism will react as if the prediction came true, leading to another prediction, and another. Eventually one of these predictions will predict pleasure or pain.”

    Your approach about the “smeared now” sounds quite feasible to me, but as far as I see it is not absolutely necessary to stick to the “prediction = actual input” assumption. The idea is that it is quite possible to duplicate some (low level?) parts of the input-processing networks and have one network for the predictive and one for the actual input part. Since most of the time the predictions will meet the actual input, cells in both networks will wire up almost identically over time. Still there might be biochemical differences between prediction processing and input processing cells. This might be inefficient, but there could be a few advantages of this approach for some limited low-level processing areas – for example when it comes to learning dynamics and drive regulation. But I am not sure if it’s really necessary, probably your approach works completely fine (and maybe better).

    Two more things:

    You said that when consciously imaging a sequence of actions and events, this happens in real time, right? I don’t think this is true. IMO it is just that the *perception* of time gets predicted together with the other events. On an abstract level, thinking of something happening at a specific time is just the same as thinking of something having a specific color or making a specific sound.
    The reason why I think that this is how our brains do it is that in lucid and/or remembered dreams there often are time oddities and some story may span a long period of “dream-time” while still happening within one or even divided across a few short phases of REM sleep (for the latter I am not sure if that really happens).

    Another thing has to do with attention and surprise: In your circuit, you have special “comparator” cells to generate surprise. I have not put much thought into surprise itself, but for attention I mostly thought of it as mere strength of input reaching a higher cortex until now. The effect of prediction then would be to inhibit input of being passed on through the network. Therefore only stuff that is stronger than the prediction would come to attention. There however are two problems and one non-problem with this approach:

    1. problem: Predictions must not completely inhibit actual input. If we wait for something to happen in order to react on it (say waiting for a traffic light to turn green), we are able to “force” our attention to something which actually is highly predicted and not surprising at all.

    2. problem: this only works for separate prediction and actual input networks, since otherwise the inhibition of input signals would also hinder predictions to travel on.

    3. non-problem: we often feel surprise also if something predicted does not happen. Say we are pressing a light switch but the light does not turn on. An alternative way for thinking of “I am surprised because I expected the light to turn on, but it did not” would be “I am surprised because I did not expect it to still be dark, but it is”. So this is why I think of this as a non-problem.

    Again your network probably works better most of the time and there might be some more problems with my approaches that I have not discovered yet. I just wanted to point out some possible variations of the prediction network depicted by you.

    Best,
    Daniel

  5. stevegrand says:

    Hey Daniel,

    I wondered about having two systems too – one for perception and one for prediction. But I don’t think it’s justifiable. For one thing it’s not just a low-level effect. I see this mechanism applying throughout the brain, so that would imply we had two whole brains – one for dealing with the recent past and one with the future. We DO have multiple representations in the brain – object recognition appears to occur in two places (one with more of a spatial component than the other) – but I doubt this is the reason. But my thoughts are moving in a slightly different direction now so maybe it won’t matter. Hard to tell yet.

    > Predictions must not completely inhibit actual input.

    It’s interesting that you saw it as inhibiting. I saw it as an excitatory system. But a few years ago I started to think that maybe our internal model of the world is an inverse model. In other words its job is to counter changes in the environment and create homeostasis, so it’s the mathematical inverse of the laws of physics. I might think more about that – but that’s a very good point point about attending to the unsurprising, because that would be a potential issue.

    > I am surprised because I did not expect it to still be dark, but it is”

    Does that mean every sensation needs also to be represented by its converse (dark v light)? If each prediction also needs a NOT gate then basically we’re back to a comparator. It’s a good point, though. Yesterday I was thinking of sequencing walking and that needed to be able to detect the non-existence of a stimulus. Hmm…

    Thanks for the observations!

    • Daniel Mewes says:

      > Does that mean every sensation needs also to be represented by its converse (dark v light)?

      Hmm no, I don’t think it works like that. At least those negated neurons obviously cannot fire all the time a sensation is not present (just imagine the energy consumption). There might be neurons for non-sensations however which get activated if the corresponding sensation was anticipated, which leads to your idea of comparators, just that there would be two of them (one for prediction but no actual input, one for sensation but no prediction). This would even allow to distinguish between input-based perception and imaginary prediction wherever necessary, without going through the hassle of two largely separated networks.

      How funny that you were thinking about walking, because while writing my last comment, I was also thinking about that while wondering about the negation thing (non(?)-problem 3). I was thinking for a moment that the following cannot be explained with the “prediction inhibits input” idea: Say someone is walking stairs but then makes a wrong step and misses a stair. This “stepping into nowhere” usually causes an immediate and strong surprise and leads to an immediate reaction (like reaching for the handrail). At first I thought that this can only be explained if the absence of the sensation of hitting the ground with the foot is what causes the surprise. On the other hand our sense of balance probably is sensitive enough to kick in quickly here.

      There might however well be cases in which a purely input-inhibiting prediction process does not work satisfactory. And the more I think about it the more it seems reasonable to me to at least use some kind of comparators.

  6. Mike says:

    Tickling

    Just a thought on your touch-prediction model. If the speed of propagation of prediction was the same speed as stroke across skin receptors; you would have an especially strong sensor response. That is if it were an analogue reinforcing prediction rather than binary.

    • stevegrand says:

      Thanks Mike. That’s true, although if the two matched exactly then perhaps it wouldn’t be enough of a prediction, since the prediction wouldn’t reach consciousness until after the event had actually happened. Then again, perhaps that doesn’t matter – perhaps all we need to know is that things are progressing as we expect, and hence none of this information need become explicitly conscious at all. The eye doesn’t transmit any information about solid colors, just changes in color at edges, and yet we think we can see the fill color, so maybe we think we can feel ourselves being stroked in real-time, when really we’re just aware that a slightly earlier prediction was suitably fulfilled. Whatever that means… The more I think about “now”, the more smeared out and retrospectively filled-in it seems to be!

      • Mike says:

        I absolutely agree that the ‘now’ is smeared out in a temporal way. But I also think it is smeared out across possibilities.

        So in touch example, the stroke across sensors may be a straight line or a curve. Say a straight line is ‘dangerous’ and requires instinctual action. As you said, you have predictions streaking out ahead of the sensors; but for both straight and curved cases (in slightly different directions). It means that the instinct can be triggered off (when a threshold is reached) before a total straight-line happens. Because the reality-reinforced prediction is winning over the reality-unreinforced prediction.

        If you took this up a few levels, your linear consciousness is the most likely (reality-reinforced) of the prediction paths at any one instant. But crucially there is a cluster of possible paths that might be your next moment of linear consciousness, if reality reinforces them enough.

  7. Chani says:

    living in the future makes sense… the more stressed I am, the more likely I am to make mistakes from thinking that the future is already the present. I skip over words, or react to something someone hasn’t said yet (which can be really bad if you were predicting a joke and the person actually has bad news).

    stress makes it harder to interact with people in general, ’cause my mind is more and more wrapped up in trying to predict and control and avoid pain, so I’m not receptive to what other people are saying, and I’m more likely to react to what I *expect* them to say – which is further influenced by my mood – when I’m in a bad mood I assume the worst of everyone.

    prediction has its downsides.

    aanyways…
    where was I?

    well, one other thing – I agree with daniel on the temporal oddities. one thing I notice a lot is that when playing back a song in my head, it’s *really* hard to get it to go in realtime. mostly it’ll run at something like double-time; if I’m anticipating a part I like it’ll try to skip ahead to that and then jump back as my OCD demands it goes in order 😉 and then if I get distracted it’ll go back and repeat itself. or get stuck in a loop because I can’t remember how the darn thing ends or which order the verses are in…

  8. Mike says:

    Might have something useful to you on general context. Email me if you want it.

  9. stevegrand says:

    [Posted on behalf of John Bradley Gibbs, cos when he tried, it wouldn’t post for some reason]

    Yosteve – it has been a while since I had a look at your blog… you appeared to be kind of on a holiday away from AI for a while, so I didn’t follow much.

    I still like your thinking, and I still believe that you have the right approach. I find your sense of the timing of reality with what (and when) we perceive – and the predictive functioning of the models – rings true. Your aversion to dogma is a valuable ingredient.

    I understand that I may be deviating a little from the theme under discussion – but it is something that I believe has some significance on the maintenance and use of the predictive models.

    I have yet to hear you say much about any part played by the dream state.

    There was only brief mentioning of dreams in some of the comments. I believe that the process of dreaming is a very significant part of what makes up mammalian intelligence.

    It is in dreams that the mind is able to run these models – and connect them in “daring” combinations – far more so than the lucid mind would ordinarily entertain.

    A (long) while ago I mentioned “The Muse in the Machine” – David Gelernter. Not because I think he has “nailed” it – but rather because I think there is merit in appreciating that the mind does operate differently in accordance to a “focus” parameter that is linked to the individual’s level of lucidness – or fatigue.

    A periodic swinging of this focus is most important to the mental health of the individual. (We NEED sleep).

    Some may argue that this is a biological issue rather than a necessary, intelligence system function – I would be reluctant to accept that.

    I think that a variable focus approach to working with the models – and a dream-like state – are important aspects to consider when assembling any AI project that is to approach mammalian capability.

    What are your thoughts on the idea that models (built and tweaked and run when we are wide awake) undergo more and more “outrageous fooling around” as our focus broadens, permitting us to come up with novel ideas. (“Sleeping on it” can result in new ideas?) Maybe deadwood is cleared from the desktop – perhaps some “de-fragging” of sorts, making space for the next day’s doodlings? This could explain the sorry state of one’s mind when deprived of sleep? (Cluttered desktop, precious little white space left to doodle…)

    …There is also something in between the lines of Gelernter’s writing – after reading through his book I seemed to have been left with a hard to express empirical feel for aspects that he didn’t even mention. (Heck, I know that sounds stupidly vague – but I don’t know how else to say it.)

    Are you familiar with the book? – or similar ideas?

    Is there a thread discussing such matters already in existence on your blog?

    John

    • stevegrand says:

      Hi John,

      Yes, I agree with what you’re saying. I’m pretty convinced that dreaming evolved to perform a biological function (defragging, as you say) but one side-effect of it certainly seems to be the running of models (especially as they relate to recent experiences) in a fluid and ‘harmless’ way that allows us to make new connections and be creative. I come up with my best ideas when I’m only partly awake (perhaps I get even better ones during mid-sleep dreams, but sadly I don’t remember them). I have to try to turn these dreamy thoughts into words and get up gently, without shaking my head too much, or they just fade out. And some of them turn out to be nonsense, later, which does fit with the idea that creativity involves some very loose associations. If we thought like this when we’re awake then we’d be hopeless (or poets), but it does seem to serve a purpose when we’re asleep.

      I’d go so far as to suggest that we (or rather our ancestors) were perhaps conscious in our dreams before we developed waking consciousness. Waking consciousness is a kind of last resort for when our cortex is unable to respond to the world autonomously and unconsciously. Learning involves turning conscious deliberations into unconscious responses (like the ability to drive a car without thinking). Perhaps dreaming plays a vital role in this process and we needed it even before prefrontal cortex became developed enough to make us able to deliberate. Maybe the ability to dream is what gave us the ability to think? Thinking is like dreaming, rather than dreaming being like thinking.

      No thread on this here, as far as I can remember, but you seem to have started one!

      • Ben Turner says:

        Hi Steve – glad to see some activity! I wonder what your thoughts are on the role of sleep in things like implicit skill development? For instance, Dan Dennett gave an example in a talk he gave here a few years ago of how, after being introduced to Tetris, his waking mind began running non-stop simulations of Tetris for several days, although he wasn’t always necessarily aware of it. On the other hand, there was a lot of noise in the press a while back about how sleep helped consolidate memories, but it’s been the experience in my lab that whether sleep helps is highly contingent on what’s being learned–in particular, whether someone sleeps in the twelve hours between two training sessions had no effect on performance, but that might be due to the fact that the skill they were learning was presumably learned implicitly, and therefore might have been undergoing as much rehearsal during waking as during sleep. That is to say, some part of Dennett’s brain was running Tetris simulations even while he was awake, which had the effect of improving his skill, but I never heard him say anything about having Tetris-inspired dreams that had the same effect (even though this example was couched in a talk about memes, I won’t get into memetics… though it seems reasonable to assume that the same memes should be active in the sleeping as in the waking brain). I can’t really imagine the mechanism that would make sleep work to strengthen, e.g., declarative memories more than implicit knowledge, but that doesn’t mean there isn’t one or that it isn’t interesting.

        What you said above about dreaming consciousness preceding waking consciousness is intriguing; it seems then that you’re ascribing the cognitive function of sleep to REM sleep, in terms of running models, etc. I have the feeling that dreams are much more passive, in the sense that they are the attempts of the interpreter (in the Mike Gazzaniga sense) to sort out the fairly random storms of activity that happen during sleep; but regardless of which view one takes on what dreams are, do you think non-REM sleep also plays a cognitive role, or is it more strictly biological?

        Anyway, sleep is pretty far outside my area of study, so I obviously don’t really have anything intelligent to say, but I feel obligated to distract you from your wor… err, I mean, keep the conversation going…

      • stevegrand says:

        Hi Ben. Hah, yes, I’m back in work mode for a few days before another hiatus.

        Have you watched the lecture by Matt Walker that Vegard posted today?

        http://fora.tv/2009/0811/Matt_Walker_Secrets_of_the_Sleeping_Brain

        I’d be interested to hear how his results fit with what you were saying about sleep having no effect on performance.

        One of the assertions in the lecture is that EACH of the different phases of sleep consolidates or otherwise integrates a different class or aspect of memory.

        From an engineering perspective (which is the only one I can comment on), the model I’m trying to develop involves multiple kinds of learning – learning patterns, organizing them into categories, and forming temporal associations between them in both directions. They all exist in the same mechanism but it seems at first glance that they can’t develop simultaneously – some need to occur during experience and then become reorganized, while others can’t make any sense until this has happened. So I’m thinking that I’ll need several modes of offline memory development, perhaps analogous with the phases of sleep. I may need a “hippocampus”, too, to act as a short-term declarative store that caches unorganized experiences for later consolidation as semantic memories (but I’ve no idea how to make that work). Oddly, REM sleep is the one I don’t have any apparent use for at the moment, unless it explores potential pathways and thus helps either in strengthening associations that are likely to become linked during cognition, or supports the reorganization of the system during category formation. Either way, REM sleep would be (in my model) a cognitive phase – i.e. one involving dynamical behavior – as contrasted with a rewiring phase.

        But none of that is likely to make much sense until I describe my model, and right now I only wish I could describe it to myself! Hopefully I’ll have more to say on it soon (although I’m tempted not to describe the details, because discovering how it all works is likely to be a part of the gameplay).

        Matt Walker talks about post-training sleep having an effect on procedural memory (although nothing as complex as Tetris). His experiments didn’t involve a second training session – I don’t know whether that’s significant. I’d be interested in your (or anyone’s) feedback on the lecture, which I found interesting.

        POST SCRIPTEM: It just occurred to me that dreaming might make a good mechanism for the transfer from episodic/hippocampal to semantic/neocortical memory, by replaying the experiences of the day so that they can be relearned in a different form (one which can’t be achieved while awake and actively behaving). But the snag would be that dreams DON’T replay the day’s experiences; they’re influenced by them but they’re not just rehearsal. And although the HC has a role in episodic memory that doesn’t seem to be all it’s involved with. Oh well…

    • Vegard says:

      Hi. I just wanted to point out a great (I think) talk about sleeping and the brain, it’s called “Secrets of the sleeping brain” and given by Matt Walker, professor at Berkeley.

      Just to emphasize that there seems to be a harmony of ideas here, and this caught my eye in particular, Matt phrased your above sentence:

      “It is in dreams that the mind is able to run these models – and connect them in “daring” combinations – far more so than the lucid mind would ordinarily entertain.” (John Bradley Gibbs)

      as:

      “It’s almost as though sleep is preferentially biasing the brain towards building and seeking out more distant non-obvious connections.” (Matt Walker)

      I’m guessing the talk is slightly on the “high-level” end of the scale, but has many details too and offers a good mix of data and speculation.

      http://fora.tv/2009/08/11/Matt_Walker_Secrets_of_the_Sleeping_Brain

      End of advertisement 😉

      • stevegrand says:

        Ooh, that looks very interesting. Thanks!

        I promised myself I’d get some actual work done this afternoon, but…

      • stevegrand says:

        Wow! What a nice talk! I’m only half way through but already he’s confirming some conclusions I’ve been drawing about my brain design and the stages necessary to consolidate memory in it. My creatures are definitely going to have to sleep, in order to develop the relational memories that make previously acquired experience useful. It doesn’t seem possible to do both those steps at once. Very nice to hear confirmation that this fits with reality!

        Thanks for the link, Vegard!

  10. Parmeisan says:

    I think my post got swallowed when I made the reply to the actual post it’s a reply to, so I’ll try it here:

    (Re: Grey Walter’s robots)

    I think it’s because of the two-way memory thing you were talking about – turkey and Christmas. You remember having learned it at some point, as the emotions and thoughts associated with the learning is more easily recalled than the actual fact. I think there’s a storage mechanism of hard, boring facts that is separate from storing other types of memories.

    It possibly has something to do with the way many people memorize numbers. Different people told to memorize the number 4771 might do it in several ways, such as:
    – Hey, those numbers are all in a line on a telephone. Just remember which line and that it’s middle-bottom-bottom-top.
    – Hey, 47 and 71 are both prime numbers. I can just remember that it’s the last prime in the 40’s and the first in the 70’s.
    – Hm, I took a class last semester called CS 477, section 1. I’ll just remember that.
    – Dad’s birthday is April 7th, and my friend Jessica was born in 1971

    But it’s only four numbers, is it really easier to form the several associations required with each of these? How likely are you to be sitting there trying to remember which friend’s birthday you associated it with versus what that third number was? I suspect the answer is actually different for different people but most of us can remember which friend much more easily than which number, even if you have many more than 10 friends.

    (On another note, remembering that you’ve learned it at some point and it’s therefore worth trying to remember does not guarantee that the memory has not faded past recall, so I don’t think it’s actually a “hey, I know that” response but rather “hey, I knew that once”.) Also, that makes me notice how memories recently accessed are always faster to recall than something you haven’t thought about in a while. So memories are always sorted by last-accessed-date.

    Also, and you probably know this already, there is (or at least, common research believes there is) two different types of memory, the long-term and short-term. When you memorize a phone number just in order to make it to the phone and dial it, it goes in a different place than when you memorize it and actually try to remember it. Perhaps that has something to do with the different tricks you use, but I remember learning, when I did a project on this, something about if you remember it for more than 7 seconds, it goes into long-term. That seems silly, but maybe it’s not completely wrong. It would certainly save memories like “I just passed a green new-looking Honda with license plate 482ABR and a black-haired driver” from having to bubble all the way down through the other recent memories into oblivion, it just goes right there after a few seconds unless something interesting happens, like the driver flips you off and you spend some time fuming about it.

    • stevegrand says:

      > How likely are you to be sitting there trying to remember which friend’s birthday you associated it with

      Happens to me all the time. I sit there locked out of my bike or my phone, thinking “it has something to do with frogs…”

      Yep, there are more than two levels and kinds of memory, it seems. Working memory requires frequent refreshes and no distractions, short-term episodic memory is quite distinct from long-term semantic memory, etc. Interestingly, my artificial brain model now has to have such a distinction too. In my case they’re variants of the same thing, stored in the same way, but in the real brain they’re separate structures and mechanisms. I think of myself as having a really lousy memory, although I’ve never quite figured out which parts of it are lacking. Or if I ever did then I’ve forgotten.

  11. vanilla beer says:

    numbers come to me in colours. For instance, the number 4771 already mentioned, the 4 is a crisp clear green, the 7 purplish – the second 7 is a bluer purple and the 1 whitish bright . The colours change according to relationship.

  12. Vegard says:

    Interesting that you should bring this up again now. Only yesterday I was looking at license plates and thinking: “Can we always make up some rule for remembering them?” Because certainly 5555 or 7777 are easy to remember. Our old family car used to be 23578 — it’s kind of symmetric around 5 (2+8 = 3+7 = 5 + 5), that’s my rule at least. I also tend to remember powers of two easily. There are also those numbers that are “almost easy” to remember, like 5554 (“5555 minus one”) or 129 (“two to the power of seven plus one”). I don’t usually try to associate something external to numbers (like birthday), but I try to find properties of the numbers themselves — for example in 3162, we have that 62 = 31 + 31.

    (Sorry, gotta go. I want to write some more on this — maybe in relation to entropy. Will be glad to hear your further thoughts as well.)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: