Hello, I think it's very interesting to notice that the avian brain telencephalon (a brain region which in mammals includes the neocortex) is very different from ours, but birds are among the most intelligent animals. For example, crows can learn how to use tools, parrots have amazing language skills, blue jays have some kind of theory of mind, etc (I can point out the references if needed). They clearly can do a lot of tasks that used to be perceived as "mammal-only" with a different brain architecture that is subject to very rigid weight and size constraints.
The avian brain has some kind of homologue to our neocortex, but it is organised in nuclei instead of layers. (Some birds also have the Wulst, which has 3 layers and has been noted to have some neocortex-like purpose, but it's probably not the main force behind their intelligence). Unfortunately I couldn't find much information regarding the fine structure of those nuclei, probably because there hasn't been much research about it. This shows that using layers is actually just one way to achieve real animal intelligence. However, due to the lack of information regarding the avian brain, we're still very far from some CLA analogue for birds. If anyone is still interested, this is a good survey on the avian brain: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2507884/ (the "Avian Cognition and Brain Function" topic is particularly enlightening). Pedro. On Wed, Oct 23, 2013 at 6:41 PM, Jeff Hawkins <[email protected]> wrote: > Fergal,**** > > Sorry you weren’t there. A couple of additional comments.**** > > ** ** > > On evolution: There are laminar structures in many parts of the brain. > The neocortex certainly evolved from them. The neocortex itself is very > consistent across mammal species. In general if you look at a slice of > cortex in detail you can’t tell what animal it came from or what part of > the cortex. The same layers, and connectivity scheme seem to exist in all > mammals in all regions. This is why rats, cats, and monkeys are pretty > good substitutes for humans when doing research. It is the number of > regions, the areal size of the regions, and the hierarchy that vary across > species. There are some exceptions. For example some mammals have extra > have extra layers in V1, and some don’t. I recommend not focusing on the > exceptions. I view them as recently evolved enhancements on a general > theme.**** > > ** ** > > If you accept that cortex is cortex then each region has to do what the > entire cortex does. Therefore we know a region must learn and do inference > on spatial/temporal patterns. We know each region has a motor output > therefore each region is learning a sensorimotor model of the data. We > also know a lot about the detailed connectivity between layers that again > is consistent no matter what animal or what region. During the hackathon I > will be happy to share what I know about these connections. This > connectivity doesn’t tell you exactly what is going on but it does provide > hints and a set of constraints.**** > > ** ** > > The thickness of a cortical region can vary a bit intra- and > inter-species, but the number layers and the number of cells in a column > don’t vary that much. Typical numbers in the literature are 100 to 110 > cells in a mini-column, which spans all layers. Layer 2/3 has the most > cells so 32 is a reasonable guess for layer 2/3. BTW mini-columns are > still somewhat controversial. Some people argue that mini-columns don’t > exist everywhere, only in some regions, and therefore are not important. > Mini-columns are very clear in many parts of the cortex. The CLA only > needs mini-columns for learning high order sequences. Maybe some tasks > don’t require high order sequences and the corresponding regions lose the > mini-column feature.**** > > ** ** > > During the upcoming hackathon I will talk about layer 5 cells. These are > the motor output of a region. I am hopeful that a useful sensorimotor > system can be built with just two CLAs, one for Layer 2/3 and one for layer > 5.**** > > Jeff**** > > ** ** > > *From:* nupic [mailto:[email protected]] *On Behalf Of *Fergal > Byrne > *Sent:* Wednesday, October 23, 2013 12:33 PM > *To:* NuPIC general mailing list. > *Subject:* Re: [nupic-dev] Open Office Hour: Wednesday Oct 23rd 9AM PDT*** > * > > ** ** > > Hi Guys,**** > > ** ** > > Just watched the recording of the session (really sorry I missed it, got > called away at the last minute). I've a couple of comments.**** > > ** ** > > Firstly, thanks to all who took part. Despite a couple of glitches, I > think this is a really powerful way to expand the discussions we're having > on the list. As someone mentioned during the session, the list has > limitations in getting to the kernel of some questions, where it seems that > a lot of that was addressed in this medium.**** > > ** ** > > It was really useful to have several people from Numenta/Grok present at > the same time, because we were able to get many contributions from the > different points of view practically simultaneously. I'm looking forward to > even higher bandwidth discussions during the hackathon.**** > > ** ** > > I thought the discussions about hierarchy were very enlightening, and I > think this medium is great because you were talking about things we haven't > seen externally before. Until you have good answers, the information is all > in the questions, and especially the doubts that you expressed in the > session. We don't get to see that in Jeff's talks or in any publications, > for obvious reasons, but they're all-important for getting to the next > stages in the process.**** > > ** ** > > Specifically related to the discussion about hierarchy, I have some > thoughts which might be of some use.**** > > ** ** > > As you mentioned, you've temporarily stopped looking at hierarchy and are > concentrating on the usefulness of a single-region, single-layer CLA > implementation, and clearly that's working well. **** > > ** ** > > Correct me if I'm wrong, Jeff, but it seems that we have all these > expensive layers (and complex structure within several of them) for a > reason: there's a lot more going on than just spatial pattern recognition > and sequence learning. The layers are performing different parts of the > computation, perhaps including:**** > > ** ** > > 1) Pure feature detection.**** > > 2) Sequence memory.**** > > 3) Using sequence memory to constrain feature detection.**** > > 4) Attention**** > > 5) Incorporation of signals from layers above**** > > 6) Motor stuff**** > > 7) Sending signals downwards in the hierarchy**** > > 8) Interaction with subcortical structures**** > > 9) Using the predictive cells**** > > 10) (nearly forgot!) sending signals upwards**** > > ** ** > > [Numbers do not correspond to layer numbers! Just the order I thought of > them in].**** > > ** ** > > The laminar structure exists in lots of places in the brain, but the huge > area of almost identical laminar cortex does not. We must have evolved the > neocortex by having a few very small hierarchies with few layers, and added > more complexity as we evolved from the shrew-like original mammals. So we > should seek (as Jeff said in the session) to build a very simple hierarchy > using just a couple of CLA regions first, and see how it behaves.**** > > ** ** > > In the CLA, we might typically have 32 or 40 neurons per column. These are > all doing the same job. In the real cortex, we have many more than this. > Neurons are very expensive, so they must all be doing something useful. ** > ** > > ** ** > > I suggest that we should see how we can emulate some of this list of > functions in each CLA column, perhaps by "separating concerns" as the > pattern guys call it. If you look at the NuPIC code, you're already doing a > huge number of things which are not strictly included in the idealised > algorithm.**** > > ** ** > > I'm on Ian's side in the debate about reconstruction. We are clearly able > to do generative "perceptions" based on top-down activations of high-level > concepts. As Jeff has identified, it takes a long time between knowing that > we should be able to do something and figuring out even one good way to do > it, so I'd say that we need to find out how we can do it. It could turn out > to be the key to understanding the whole idea of how hierarchies work, in > the same way that cell choice is the key to understanding how sequence > memory may work.**** > > ** ** > > Delighted with the discussion, nuts that I missed it!**** > > ** ** > > Regards,**** > > ** ** > > Fergal Byrne**** > > ** ** > > ** ** > > On Wed, Oct 23, 2013 at 6:16 PM, Matthew Taylor <[email protected]> wrote:* > *** > > I had some reports of people getting kicked out of the Google Hangout, > unable to reconnect. Please report any issues you had to me so I can > address them. We may want to switch to another conferencing technology if > Hangout has issues. I don't want to exclude anyone.**** > > ** ** > > Thanks,**** > > > **** > > ---------**** > > Matt Taylor**** > > OS Community Flag-Bearer**** > > Numenta**** > > ** ** > > On Wed, Oct 23, 2013 at 8:49 AM, Matthew Taylor <[email protected]> wrote:* > *** > > Hangout URL: > https://plus.google.com/hangouts/_/d7fbbdde067c5ddb249c1c526473b2420ebc3519 > > YouTube URL: http://youtu.be/MWBFw4WoZxA**** > > > **** > > ---------**** > > Matt Taylor**** > > OS Community Flag-Bearer**** > > Numenta**** > > ** ** > > On Wed, Oct 23, 2013 at 8:06 AM, Matthew Taylor <[email protected]> wrote:* > *** > > Reminder: this online event starts in 54 minutes. **** > > > **** > > ---------**** > > Matt Taylor**** > > OS Community Flag-Bearer**** > > Numenta**** > > ** ** > > On Fri, Oct 18, 2013 at 6:12 AM, Marek Otahal <[email protected]> > wrote:**** > > Thanks Matt, this is a great step! **** > > I hope it'll get more knowledge sunken to the community. **** > > ** ** > > On Thu, Oct 17, 2013 at 10:57 PM, Matthew Taylor <[email protected]> wrote: > **** > > Have questions about NuPIC or the CLA? Jeff and Subutai will be in > attendance to help answer them in our first Open Office Hour at Grok. > This event is focused on NuPIC and our open source community, but > anyone is welcome to join and ask questions, or just discuss related > technologies. > > > https://plus.google.com/b/100642636108337517466/events/c2pt64fid2niuso3r4mp5n7u9os > > Please join in if you would like to chat. I'll be recording and > streaming on YouTube. I will post the URL for the Hangout and the > YouTube link here just before the event. > > Regards, > --------- > Matt Taylor > OS Community Flag-Bearer > Numenta > > PS: I know this time will not work for everyone interested in > attending. For the next office hour, I will try to schedule a > completely different time so others may attend.**** > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org**** > > > > > -- > Marek Otahal :o) **** > > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org**** > > ** ** > > ** ** > > ** ** > > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org**** > > > > **** > > ** ** > > -- **** > > > Fergal Byrne**** > > ** ** > > Brenter IT**** > > [email protected] +353 83 4214179**** > > Formerly of Adnet [email protected] http://www.adnet.ie**** > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > -- Pedro Tabacof, Unicamp - Eng. de Computação 08.
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
