There are a lot of unknowns when it comes to motor behavior. The logical assumption is that higher regions of the cortex are generating motor commands that parallel the representations in the higher regions, that is high level goal oriented behavior such as "get some food from the fridge" where lower levels of the cortex generate motor commands that are more specific and of shorter duration such as "open fridge door" or "grasp". So the different regions must work together. This basic idea has been around for decades.
The saccade is a good place to think about the issue because a lot of details are known about the superior colliculus (the structure that generates saccades) and its relationship with cortex. Here are a few interesting tidbits. - It has been shown in multiple studies that some cells in V1, and more in V2, anticipate what they are going to see before the eye has stopped moving during a saccade. They become active before the pattern that will make them active arrives on the retina. The brain knows what it is going to see after a saccade. - That doesn't answer your question of how the cortex decides where to saccade to but it does tell you that it isn't random and the cortex has an expectation of what it will see after it saccades. - The superior colliculus has layers. V1 cells project to one layer ( I think the outermost but I don't recall with certainty). V2 cells project to the next layer, etc. The cells in the superior colliculus that actually innervate the eye muscles are at the end of this stack of layers. I forget the details, but the point is eye movements are generated by cells in several levels of the cortex and this hierarchy is reflected in the superior colliculus itself. There are many papers on the superior colliculus. If I had a couple of free weeks I would scour them to see what can be learned about motor output of the cortex. Jeff -----Original Message----- From: nupic [mailto:[email protected]] On Behalf Of Patrick Higgins Sent: Thursday, August 22, 2013 4:39 PM To: NuPIC general mailing list. Subject: Re: [nupic-dev] Mentioned presentation on action with CLA? Interesting Jeff. I've been doing some thinking on this topic as well. You implied that motor related function is distributed, not centralized, and probably happens in layer 5 of each region in a hierarchy. I imagine that the lower regions (unconscious parts) are responsible for some involuntary triggering of motor commands, while the higher up regions (the ones we are conscious of) triggered decisive movements. I know there are other mechanisms for involuntary motor function in the brain and body, but I am looking at just how the cortex is involved. I don't know if this is correct, but thought I'd ask. I am still not seeing in your comments below how one decides (for example) where to saccade. We have at times many logical places to look next, and while it is generally involuntary, its still managed in some way. There seems to me to be more than just prediction at work here. When it is voluntary, it exposes this point further, being directed from a supervisor (conscious thought?). Patrick On Aug 22, 2013, at 12:34 PM, Jeff Hawkins wrote: > Addressing Mike's comment regarding "motor-related" output vs. "motor command" output. > > Here is what I think is going on. We have many built-in subcortical motor behaviors. Each of these built-in behaviors has some neural network that generates the built-in behavior. We can assume these hardwired neural networks use sparse activations (just like the cortex) and they will obviously be implemented by neurons. > > Now say some cortical layer 5 cells project to each of these built-in behavior generators. Initially they don't make connections or they make random connections. If the layer 5 cells are recalling a sequence of SDRs at the same time as the built-in pattern generator is creating a behavior then the layer 5 cells will form synapses with the cells in the built-in pattern generator. These synapses form exactly like synapse formation in the temporal pooler. In the temporal pooler a set of currently active cells forms synapses on the distal dendrites of recently active cells, thus learning a transition. The temporal pooler is a form of "auto-associative" memory. However, in the motor case a pattern of cell activity in the cortex associatively links to cells in the behavior generator. The end result is a sequence of patterns in the cortex can learn to invoke a sequence of patterns in the behavior generator. > > Let's use breathing as an example. Somewhere in the brain is a small > piece of neural tissue that makes you breath. This is hardwired from > birth. But your cortex can consciously control your breathing too. > You can will yourself to breath or not breath, up to a point. That > conscious control is coming from the cortex. If you try to stop > breathing eventually the old brain component takes over and makes you > breathe again. (Typically the cortical cells connect to the most > distal dendrites which can be overridden by inputs closer to the cell, > this is why cortical control of breathing only works up to a point.) > > I believe we have to learn how to control our breathing by associatively linking a cortical model of breathing with the built-in breathing generator. The same is true for swallowing, blinking, eye movements, walking, running, reflex withdrawal, etc. Note that we cannot learn to control our heartbeat. I assume this is because the cortex does not project to the heart beat generator. Therefore, even if we learn to "hear" our heartbeat we cannot learn to control it. I believe this associatively linking of sequences is how the cortex learns to make behavior. > > The same basic principle is occurring in the feedback connections between regions in the cortical hierarchy. Cells in layer 6 project to the apical dendrites of cells in layers 3 in the region below in the hierarchy. Thus a higher level region can invoke a sequence in the lower region. However, if the lower region receives contradictory feedforward input the feedforward input wins. > > In the CLA we have never modeled apical dendrites or this feedback process. The learning is identical to the learning in the temporal pooler, so I am highly confident it will work. > > So back to the initial question, are layer 5 cells "motor-related" or "motor commands"? Most layer 5 cells don't project directly to muscles (although some do) so you might say they are "motor related". I think the more nuanced answer is cortical cells learn to control behavior by associatively linking to parts of the brain lower down the hierarchy. Sometimes these cortical cells will directly control behavior and sometimes they will be overwritten by more powerful feedforward inputs. > Jeff > > > > From: nupic [mailto:[email protected]] On Behalf Of > Michael Ferrier > Sent: Tuesday, August 20, 2013 4:19 PM > To: NuPIC general mailing list. > Subject: Re: [nupic-dev] Mentioned presentation on action with CLA? > > I agree with all of that Jeff, just have two points to add: > > - While all regions of the cortex seem to have some motor-related output, I think it could be misleading to say that all regions have some "motor command" output. Guillery and Sherman (e.g. http://ironzog.com/nupic/Guillery_Sherman_2002a.pdf) talk about layer 5 cells having output that branches either to the spinal cord, tectum, or pons. The tectum (superior colliculus in primates) is involved in orienting the eyes and head toward attention-grabbing stimuli and the pons relays information to the cerebellum, which is involved in the smooth coordination and timing of movement (but is not necessary to produce movement). The only cortical areas that send out "motor commands" to the spinal cord (and from there to muscles) are the motor areas (with a small percentage also coming from the parietal cortex and cingulate cortex). > \ > > -Mike > > _____________ > Michael Ferrier > Department of Cognitive, Linguistic and Psychological Sciences, Brown > University [email protected] > > > On Tue, Aug 20, 2013 at 5:19 PM, Jeff Hawkins <[email protected]> wrote: > Two neuro-scientists, Ray Guillery and Murray Sherman have pointed out that in every region of the neocortex they have looked, they find cells in layer 5 that project to muscles, the spinal cord, or other behavior related parts of the brain. For example in primary visual areas V1 and V2 there are layer 5 cells that project to the Superior Colliculus which generates saccades and other eye movements. I don't believe they counted the basal ganglia as a "motor" destination. Sherman and Guillery have proposed that this is the normal state of affairs, that all areas of the cortex have a motor output. This is a beautiful idea and certainly mostly true. > > Sherman and Guillery have written extensively about these layer 5 cells. The axons from these cells split. One branch goes to the muscle or motor area and the other half goes to the next region up in the hierarchy. Thus all regions of the cortex have some motor output command, but that same command is passed up the hierarchy. The next region thus knows what behaviors are being generated. Layer 3 receives both sensory and motor input. > > Layer 3 is the primary feed forward layer. It is what I think of when thinking of the CLA. In the general case layer 3 is building a model of sensory data plus motor commands. Layer 5 is similar to layer 3 in many ways. I believe it is learning the same sequence of column activations and thus the same sequences. The unfolding patterns of layer 5 cells then associatively link to other motor areas and thus learn to control them. It is a bit hard to describe without images. > > Conventional wisdom says that the basal ganglia does not create behavior directly. It seems to be responsible for selecting between alternate motor plans stored in the cortex. > > I believe we can build a simple system consisting of one CLA representing layer 3 and another CLA representing layer 5. The Layer 5 CLA is driven by layer 3 and associatively links to some pre-existing motor generator. The system would learn to string together pre-existing behaviors in novel ways. I don't know if we would need a basal ganglia equivalent. There are several unknowns but the basic idea seems sound. I have a talk that goes into this idea. We hope to record it and make it available. > Jeff > > From: nupic [mailto:[email protected]] On Behalf Of > Michael Ferrier > Sent: Tuesday, August 20, 2013 11:54 AM > > To: NuPIC general mailing list. > Subject: Re: [nupic-dev] Mentioned presentation on action with CLA? > > The impression that I get from the neuroscience literature is that there are two basic types of learning in the brain. The first type could be called "model learning", it is what the cortex specializes in, and it's about learning hierarchical spatio-temporal models of input, from both external sensors and from other brain areas, representing the outside world, the body, and other internal states, and how they change over time. The second type is reinforcement learning, which uses built-in "reward" and "punishment" signals (such as pain or the taste of sugar) to learn what cortical patterns should be activated within a particular context of the activity of other cortical patterns, so as to maximize reward and minimize punishment. In the brain, reinforcement learning takes place in the basal ganglia, but uses input from many different areas of the cortex, and affects the activation of patterns within prefrontal and motor cortex to result in the control of attention, working memory and movement. > > For a more detailed discussion, see e.g. chapter 7 here: > http://grey.colorado.edu/mediawiki/sites/CompCogNeuro/images/8/89/ccnb > ook_01_09_2012.pdf > > It's this dichotomy that I think the BECCA system is getting at, with their distinction between a "feature creator" and a "reinforcement learner". All cortical regions contribute in some way to motor output, if only by providing contextual information to the basal ganglia or to other subcortical structures involved in shaping motor output, such as the cerebellum or superior colliculus. But the final output to the spinal cord that actually produces movement comes mostly from the motor areas. > > CLA strikes me as being potentially a major advance in simulating the cortex and its spatio-temporal "model learning", but I think the addition of reinforcement learning will be necessary in order to approach the problems of action selection, attention, working memory and cognition in a brain-like way. > > -Mike > > _____________ > Michael Ferrier > Department of Cognitive, Linguistic and Psychological Sciences, Brown > University [email protected] > > > On Tue, Aug 20, 2013 at 11:34 AM, Thompson, Jeff <[email protected]> wrote: > Hello SeH, > > While I appreciate your pointing out this open source project of which I was not aware, it seems to go against my question. I started paying attention to work on the CLA (again after many years) when I heard Jeff Hawkins speaking as he does below that "There are no pure "sensory" regions and no pure "motor" regions". It gave me hope that this work might avoid the pitfall of the classic "input-processing-output" loop of classic AI, which BECCA clearly seems to follow (see the attached diagram). > > We now know that there are just as many feedback connections going to back down to the "input" regions, and that action is not so different from perception (in that it uses similar machinery of prediction), and that "input" is intimately tied to the actions active during the input (instead of having "action" on the other side of world from "input", as in the BECCA diagram). > > I'm hopeful to see a diagram soon of many CLA modules for action and perception connected in a hierarchy which shows how action comes from similar prediction machinery as perception and how to avoid the pitfall of "input on one end, output on the other end." > > Thank you, > - Jeff T > From: nupic [[email protected]] on behalf of SeH > [[email protected]] > Sent: Monday, August 19, 2013 6:26 PM > To: NuPIC general mailing list.; [email protected] > > Subject: Re: [nupic-dev] Mentioned presentation on action with CLA? > > i imagined that something like OpenBECCA http://openbecca.org could be > integrated with NuPIC for a motor control system > > https://github.com/brohrer/becca > https://github.com/brohrer/becca/blob/master/doc/becca_0.4.5_users_gui > de.pdf > > from the opposite direction: part of BECCA's perceptual components may > be enhanced (or replaced) with NuPIC > > https://github.com/brohrer/becca/blob/master/core/agent.py > > > On Mon, Aug 19, 2013 at 8:25 PM, Thompson, Jeff <[email protected]> wrote: > Thank you for the quick reply. I'm in Los Angeles, so I hope someone does record your presentation at NASA. > > A similar question arose when I read "Thinking, predicting, and doing are all part of the same unfolding of sequences moving down the cortical hierarchy." (On Intelligence, p. 158.) I'm sure this is a FAQ, but do you have some writings or presentations about how a CLA would receive feedback signals coming down the hierarchy? > > Thank you, > - Jeff > > > From: Jeff Hawkins <[email protected]> > Reply-To: "NuPIC general mailing list." <[email protected]> > Date: Sunday, August 18, 2013 11:35 AM > To: "'NuPIC general mailing list.'" <[email protected]> > Subject: Re: [nupic-dev] Mentioned presentation on action with CLA? > > Jeff, > I wrote this presentation a couple years ago for a workshop on sensory motor integration. That workshop was held at the Santa Fe institute and I don't believe it was recorded. The genesis of the workshop was a paper written by Murray Sherman and Raymond Guillery where they point out that every region of the neocortex (as far as they have looked) has cells in layer 5 that have a motor function. The big idea is that every region of neocortex does sensory inference and generates behavior. There are no pure "sensory" regions and no pure "motor" regions. It is one of those beautiful results that make you slap your head and say "of course!" > > I have always envisioned the CLA as modeling a section layer 3 in a region of the neocortex. Layer 3 is the primary input layer and is therefore doing inference on the input to that cortical region. Layer 5 is driven by layer 3 and has the cells that innervate muscles, or more often project to some sub-cortical area that generates behavior. I see how two CLAs, one for layer 3 and the other for layer 5 can work together to learn a sensory motor model of the world where today's CLA is purely sensory. There is a lot I don't understand but there is enough that I think we can make progress. > > I gave this presentation again earlier this year at Numenta. It wasn't recorded. It looks like I might give it again this fall at NASA Ames here in Silicon Valley as there are a few roboticists there interested in it. > > I don't mind recording it if someone could take care of the logistics. > Jeff > > From: nupic [mailto:[email protected]] On Behalf Of > Thompson, Jeff > Sent: Saturday, August 17, 2013 4:57 PM > To: NuPIC general mailing list. > Subject: [nupic-dev] Mentioned presentation on action with CLA? > > Hello. > > > > In the introduction for the NuPIC Hackathon Kickoff, Jeff Hawkins talks briefly about the need for CLA integration with action. In response to a question, he says "We haven't done experiments with motor interaction. I have a presentation, I think about it." Is the presentation about motor interaction with CLA available? > > http://www.youtube.com/watch?feature=player_detailpage&v=yShNQvJEP6A&t > =2188 > > > > Thank you, > > - Jeff Thompson > > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org _______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org _______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
