WordPress brainstorm!

I seem to have hit some kind of word limit or bug in the comments thread to my last post, because I don’t seem to be able to reply to anyone there any more. I’ll just have to do it as a separate post instead. Thanks to all of you for such interesting comments! It’s both helpful and encouraging!

Jonet: Thanks for the link on blindsight. I’d read about this a number of times before, but never heard of anyone capable of obstacle avoidance after severe occipital damage before. It goes to show that the superior colliculus is a lot more powerful than was originally assumed. That makes sense, because it evolved from the optic tectum in amphibians, which is the top of their visual system and probably responsible for all their visually guided behavior. The inferior colliculus right next to it has auditory maps of space, so it was interesting to see mention of head-direction and place cells in that general area of the thalamus. I was talking with torea about those the other day.

John: Thanks, that’s a very interesting breakdown! Viewing attention as goal-directed like this makes it central to all action, perhaps. Since attending to a high-level goal, such as “avoid bodily harm”, requires the organism to carry out a sequence of actions to achieve the goal, some of which may themselves require attentional shifts, then it makes sense to hypothesize that the same mechanism is at work in selecting the actual actions. This would make attention the mechanism of volition, right down to the triggering of individual movements. Don’t you think?

For example, a bottom-up attentional shift in response to a sudden visual movement might orient the organism’s eyes to that stimulus. Something subcortical might then detect a snake-like pattern to the stimulus and raise the alarm, triggering the organism to select a high-level goal – to run or to fight. Fighting the snake would be a complex action schema. This might involve gathering more information from the senses – hence an attentional shift in the visual and auditory systems driven by the action schema itself. It would also involve hitting the snake with something, which in turn involves smaller goals, such as raising the arm, orienting the body and smiting the snake. Each sub-goal would be triggered in exactly the same way, but not top-down from the PFC, nor bottom-up from the senses, but from the middle-down, as part of the schema.

Does that make sense? A hierarchy of ideomotor actions, each of which specifies a goal state in some subsystem. This is very reminiscent of setting the target state in a servomotor, whose output then sets the targets in other servos lower down. It would make attention and intention exactly the same thing (as I surmised in my Lucy book, in fact!).

What is going on??? Now I can’t reply to an earlier post either! I’m being ostracized by my own blog!!! Maybe these comments will show up eventually, or maybe it’ll sort itself out in time for my next post. Oh well, any other replies to your comments will appear here until I know better…

Daniel: Thanks, I’d forgotten about PCA (mostly because I’m a klutz at math!). Fascinating that male-female might form an axis in face space. That would be biologically very valuable, as you say! I’ll see if I can get my head around PCA for inspiration.


About stevegrand
I'm an independent AI and artificial life researcher, interested in oodles and oodles of things but especially the brain. And chocolate. I like chocolate too.

10 Responses to WordPress brainstorm!

  1. John Harmon says:

    Hi Steve,

    Yes, I agree with your basic “hierarchy of ideomotor actions” model that you describe using the snake example. An intention to smite a snake involves triggering the individual self-movement sub-goals necessary to make this happen.

    I also agree that attention and intention are the same mechanism in the brain. To me an intention is a generalized goal which triggers the less-generalized, more specific sub-goals necessary to make the goal happen For example, the intention to kill a snake triggers the related sub-goals: self movement goals and perceptual experience goals (both of self and of environment) most strongly associated with killing the snake.

    These self action/experience goals, once activated, are what an agent then attends to in order to achieve the larger intention “kill the snake.” Actually, the agent attents to both the subgoals (action + perception), as well as the agent’s immediate perceptual experience. Perceptual experience is what tells the agent which sub-goals are being successful and need to be emphasized even more, and which sub-goals are not “working” (which ones are not energizing the higher goals) and need to be reduced in intensity, or replaced altogether.

    For example, an agent could energize their “aim at the target” subgoal when swinging their stick at the snake. But if the agent then notices that this strategy is not working because the snake is avoiding the strikes, they could then try a new sub-goal, such as “aim slightly ahead of the target” or “change the direction of the strike at the last moment.” The basic circuit at work here is agent intention → subgoals → self action → resulting experience (agent + environment) → match experience to subgoals → (1)increase the strength of matching (“successful”) goals, (2)decrease the strength (or eliminate) non-matching goals, (3) introduce new sub-goals that match the larger goals + intention, and the overall perceptual dynamic (schema?).

    So yes, I basically agree with your views on how an agent functions in real time — good stuff!


    • stevegrand says:

      Good! Although I guess it does raise the question of why some actions are unconscious. If intention == attention then we have to suppose that some aspects or levels of “attention” are below our awareness, which rather sounds like a contradiction. Any thoughts?

  2. John Harmon says:

    You raise a very interesting question, a question which I’ll paraphrase as “if motor action is caused by an intention to move, and by the specific attention to one’s own movement, then why is it that so many motor actions are performed unconsciously?”

    To answer this question, I need to back up a little and explain my views on how I think motor actions are generated in real time. My view is that a given motor action is caused primarily by the activation of a sensorimotor memory, composed of a number of sub-memories. This is basically the same idea as your ideomotor model of motor action. The activation of this sensorimotor memory — either top down by the prefrontal (self) or bottom up by one’s sensory stream/perceptual experience — in turn triggers an ASSOCIATED motor signal, and subsequent motor action. As long as this sensorimotor memory (or “premotor memory”) is activated with sufficient energy to trigger its associated motor response, then it doesn’t need to be energized with an energy level sufficient to become a conscious experience.

    For example, a motor intention to “take a sip of coffee” involves a large number of sub-movements, such as “lift one’s hand,” “move one’s hand toward the cup by unfolding and extending the arm,” “open one’s hand and then curl fingers in preparation to grasp the cup,” etc… This motor intention is actually a sequence of perceptual experience memories — the sequence of visual and somatosensory experiences associated with taking a sip of coffee. Associated with this motor intention is a particular set of motor signals sent from the motor cortex to the arm, hand, and fingers; signals which cause the actual movement. So this motor intention = a visuo-somatosensory memory + the associated motor signal memory.

    An initial activation of this movement goal/intention/memory “take a sip of coffee” activates the entire movement sequence, and is often a “conscious decision.” However, the lower-energy, individual parts of this movement sequence are, for the most part, performed unconsciously. Once the movement has commenced, the sensorimotor memory driving this movement can be re-energized, either as a whole, or selectively. For example, one can attend to what one is doing overall (taking a sip of coffee), which energizes this overall memory. One can also consciously energize and shape the individual memory parts, by energizing these parts top-down (now curl the fingers, now encircle the handle, etc…). In other words, top-down attention can either energize the sensorimotor memory as a whole, or the parts of the memory selectively. Bottom-up, the sensorimotor memory can be energized by matching perceptual experience. For example, seeing one’s hand moving toward the cup would work to trigger the memory most closely associated with this experience (the subsequent grasping of the cup).

    The motor action of speech production is another good example illustrating how the activation of a given sensorimotor memory creates an associated motor action. For example, the sound of the word “apple” is created by a given sequence of motor actions of the larynx, tongue, lips etc. This sequence is triggered in part by the memory of the sound of “apple.” The activation of this auditory memory works to trigger these associated motor actions. Because of this sensorimotor association, a person can say the word “apple” by thinking “say the word apple” which triggers the memory of what “apple” sounds like. In other words, a person doesn’t need to attent to, or even be aware of, the particular larynx, tongue, and lip actions that cause the creation of this sound, since these motor signals are triggered by the memory of this strongly-associated sensory experience.

    In my view, the basic circuit of sensorimotor control is (1) top down: prefrontal-generated movement goal → sensorimotor memory → associated motor signal, (2) bottom-up: sensory receptor signals (from retinas, body) → sensorimotor memory → associated motor signal. There is also a third circuit, which is simply: motor signal → next motor signal or series of motor signals, within a temporal motor memory sequence. In other words, the motor signal sequence itself works to prime for the activation of future motor signals in the temporal sequence. All three pathways in the brain are operating at once, to varying degrees, working to energize and shape a given spatial and temporal pattern of motor signals, and create resulting motor action.

    In any event, the larger point I’m trying to make is that in my view, it’s largely the activation of a sensorimotor memory (sensory memory + motor memory) which selects a given motor output signal to the body. A servo process then takes over, minimizing the difference between ongoing memory activation and perceptual experience. This servo process involves continually adjusting the contents, and strength of activation, of the sensorimotor memory to match (1) perceptual experience and (2) the overall movement goal.

    Steve, I hope this answers your question. I also hope that you found this to be a useful, if somewhat longwinded, explanation of sensorimotor control 🙂 … My own personal opinion is that this model can be applied not only to human beings, but to any animal with a brain!

    Does this basic model make sense to you?


    • stevegrand says:

      Mmm, yes, it makes sense. In fact I kind of wrote a book along those lines. But there seems to be an implicit assumption in what you say. I don’t disagree with it but I’d be interested to tease it out a bit.

      You say there are two paths (discounting the lateral, sequencing signals): top-down –> sensorimotor memory, and bottom-up –> sensorimotor memory. Since the pathways from memory onwards are identical, the only difference is that one “intentional force” starts in the PFC and the other starts lower in the perceptual system, yes?

      So the implication is that only those patterns of nerve activity that arise in prefrontal cortex are conscious. This isn’t a dumb assertion by any means. But WHY? What is it about prefrontal activity that makes it conscious?

      And isn’t the PFC architecturally just an extension of cortex? In which case we have two remarkably similar things:

      1/ A bottom-up signal rises to a medium level in cortex and then triggers a response that acts downwards (sequencing the movement) – this is unconscious.

      2/ A bottom-up signal rises to the very top of the cortical hierarchy and then triggers a response that acts downwards – this is conscious.

      Even deliberate, voluntary, considered actions of the self are a logical consequence of what that self perceives and associates with past experiences. Unless you’d like to suggest that we have a soul, which makes decisions independently of the brain. So that’s the SAME process as the bottom-up one, except that it travels somewhat further up before starting back down again. Isn’t it?

      I think there is some logic in seeing sensory information that rises to near the top of the hierarchy differently from that which triggers a response before getting that far: If the cerebral hierarchy is like a tree, with the PFC as the trunk and the primary sensory areas as the leaves, then something that doesn’t trigger a response until it reaches the top will necessarily involve the whole body and be exclusive in nature (we can unconsciously scratch an itch while still riding a bike, but we can’t choose to go out to dinner AND stay home at the same time. Consciousness seems to be associated with whole-body behaviors.

      But WHY am I, the Self, aware only of prefrontal activity? Again, there’s some evidence that this is indeed close to the truth – failure of the PFC (especially the dopamine circuit) can release impulsive behaviors that (in my estimation) aren’t nearly as conscious. Am I to conclude that there are two or more people in me, and I just happen to be the one at the top, who gets conscious awareness and control of the entire body and senses at once, while my fellow travellers are equally conscious to themselves but only aware of and able to influence a small part of the actions of the whole organism? I wonder how they feel about this?

      It’s a serious question! 🙂

  3. John Harmon says:

    Thanks for the discussion. You raise a number of interesting issues and it is a real challenge to try and come up with equally interesting responses. The puzzle of human consciousness — how the brain functions through space and time to create perception, cognition, and motor action — is absolutely fascinating!

    I agree with most of your analysis of top-down and bottom-up influences on motor behavior. One thing that I disagree with however is the idea that consciousness is ALWAYS caused by PFC activity. (If I implied this in earlier posts I didn’t mean to.) My view is that if someone decides to do something, then yes the conscious experience of that decision (ex: “I think I’ll have one of those cupcakes”) is PFC driven. In other words, the self (John Harmon) decides that the self (John Harmon) should do this action, in this situation. This involves using one’s memories to create an imaginary future scenerio (a goal or “prediction” if you like) of one’s future behavior. So this imaginary future scenerio creation is initiated by the PFC, and one’s attention to this imaginary future scenerio is what creates the energy that makes this future scenerio conscious.

    On the other hand, conscious experience can also be created independently of PFC activity. An example is when a person is reading, when they suddenly notice a knocking sound on their door, or they notice a sudden pain in their foot. These conscious experiences are not PFC driven, but instead the PFC reacts TO them (“how should John Harmon react to this experience?”)… Another example is your Brainstorm 7 post, where you describe the experience of water droplets streaming out of the shower head. To me this is a mixture of top-down attention (“I’ll focus on these droplets”) and bottom-up experience (“wow, look at those things!”). The latter experience is not a conscious experience of PFC activity, it is a conscious experience of low level visual qualia (colors, shapes, and motion in 3D). In other words, the water droplet experience is created both by a top-down (PFC driven) and bottom-up (retinal signal-driven) energizing of these visual memories. At times, the experience is entirely low level and bottom up, as when these visual memories collectively trigger an intense qualia experience. The experience here is in the qualia and qualia combinations themselves. At times, the energy is in the decision-making process, in which case conscious experience is PFC driven, and a more cognitive experience.

    I really liked your description of the temporal nature of conscious experience, with the water droplets. I think of it as the present + immediate past reverberation of memories, triggering a priming or predicting memory activation that is weakly experienced in the present, but will often become more active in the future.

    In regards to individual selves not liking what the overall self is deciding, yes I agree. The overall self can be a real drag sometimes! For example, my hedonist self hates it when my health-conscious self takes control and says, “John, walk away from those cupcakes.”… It seems like my selves are always in competition to see which one will take over. Different ones emerge at different times, an emergence which usually makes the other ones at least a little bit unhappy! 🙂


  4. Chani says:

    “But WHY am I, the Self, aware only of prefrontal activity?”

    because it makes it into memory.
    I’m beginning to think a lot of this is illusion 🙂
    now, let’s separate consciousness and intention.
    I can be conscious of an act but believe it was unintentional (an uncomfortable feeling). can I believe something was intentional despite not being conscious of it? perhaps – does automatically drinking coffee without thinking about it count as intentional? it was my intention to drink the coffee, but I wasn’t fussed as to how it got done (and might be disappointed to find it gone so soon, in which case the feeling of intention often evaporates).

    I already know that the feeling of *intending* to do something is something that can be faked or fooled; it’s an illusion, an after-the-fact rationalization that we’re almost entirely unaware of.

    hmmm. I feel like there’s something I’m not understanding here. darnit.

    oh, and on multiple selves: the older I get, the more I argue with myself. 🙂 perhaps these ‘selves’ (which I think I created somewhat intentionally, to help sort out my thoughts) are representations of possibly-conflicting goals? hmm. am I merely a collection of desires, like a more complex norn, or is there something else going on in my head? my wandering thoughts about thinking, are they a goal-satisfying routine? but then if all my goals were satisfied would I stop thinking? 😉 I guess Curiosity is *never* satisfied… but no, there’s Creativity too – the mind keeps thinking even when there’s no problem for it to solve. sometimes it goes looking for problems (I worry too much), sometimes it just makes semirandom connections, wanders around playing with ideas. 🙂

    do I only associate my “self” with the problem-solving part of my mind because that’s the part I remember best? who is “I” that is doing this remembering? the memories are called up for the purpose of this problemsolving, are they not? the part that has Thoughts is the only part capable of wondering who it is. 🙂 … but then what are thoughts? seems to me they’re a problem-solving process. 🙂 I’m going in circles. ha.
    I should probably stop rambling on your blog and wander back to my own now…

    • Vegard says:

      > “But WHY am I, the Self, aware only of prefrontal > activity?”
      > because it makes it into memory.

      I think this is an interesting notion, and I’ve been playing with this idea myself. I think there’s a very strong connection between them (memory and awareness), but I am fairly sure that they are not equivalent.

      For sure you can do something consciously and not remember it. Your actions don’t stop being conscious actions because you forgot about them, because whether an action is conscious or not depends on what happened BEFORE (and up to) the action, not on what happened afterwards (e.g. the forgetting it).

      It would be interesting to investigate whether we can remember something which was not done consciously, but I am not sure the question even makes sense. I think I can scratch my arm as an automatic response to the feeling of itching and afterwards remember this, but how can I prove that I didn’t remember it _because_ it reached my consciousness? (I.e. the action of scratching was itself an unconscious decision, but in the middle of scratching I became aware of it and therefore remembered it, much like you can remember a lot of other “passive” sensations.)

      Hm, now I realised that there is a difference between sensations and actions — can a sensation be conscious or unconscious? Certainly there are sensations that don’t reach my conscious mind (the kind that you get used to ignoring, like the temperature of the room or the pressure of your body against the seat). There are also sensations that do reach the conscious mind, like the words of the morning newspaper. But aren’t these exactly the kinds of things that I tend to not remember and remember, respectively?

      So maybe the memory of sensations is equivalent to consciousness and that memory of actions is in fact just a memory of (the sensation of) intention.

      I should probably stop rambling too and sort out my terms. Maybe there is in fact no such thing as “a memory” because the brain is always changing and the noun “memory” as we use it in our daily lives is a term that describes a higher-order phenomenon much like temperature is only a useful concept at the macroscopic scale.

    • stevegrand says:

      Interesting stuff!

      > perhaps these ‘selves’ … are representations of possibly-conflicting goals?

      That’s interesting. I guess conflict does have a lot to do with the unity (or otherwise) of self. Perhaps each of us is a whole bundle of independent “selves” but we can only walk or look in one direction, so they’d better come to a consensus, and that consensus is “us”. I sometimes wonder whether there are other people inside my head, but because they don’t often get their hands on the controls they just can’t express themselves. So *this* me thinks of itself as the sole owner of this body, but there may be other personalities with a much vaguer sense of self inhabiting my brain too. I even get the impression that these other selves get frustrated and grab the controls whenever they get the chance.

      I’ve known someone whose idle hand would get “taken over” and break things as if controlled by a frustrated alter ego, and someone who changed personality dramatically (to the point of having different memories) when the first self got over-anxious and collapsed. So perhaps it’s not that consciousness comes from the problem-solving side of our brain, so much as our problem-solving side is the part that usually has its hands on the controls and hence is the dominant and most expressive self in our body? Just a thought.

  5. Tim Hutton says:

    Oja’s rule implements PCA:

    PCA might be one of the big secrets of how the brain works…

    (This is going back to the pose here:


    • stevegrand says:

      Thanks Tim. The equations send my brain into spasms – I just don’t think that way at all. But I agree that something like this is important for self-organized mappings. I think I’m going to end up with something similar, but coming at it from a biological and engineering perspective.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: