Fergal, Sorry you weren't there. A couple of additional comments.
On evolution: There are laminar structures in many parts of the brain. The neocortex certainly evolved from them. The neocortex itself is very consistent across mammal species. In general if you look at a slice of cortex in detail you can't tell what animal it came from or what part of the cortex. The same layers, and connectivity scheme seem to exist in all mammals in all regions. This is why rats, cats, and monkeys are pretty good substitutes for humans when doing research. It is the number of regions, the areal size of the regions, and the hierarchy that vary across species. There are some exceptions. For example some mammals have extra have extra layers in V1, and some don't. I recommend not focusing on the exceptions. I view them as recently evolved enhancements on a general theme. If you accept that cortex is cortex then each region has to do what the entire cortex does. Therefore we know a region must learn and do inference on spatial/temporal patterns. We know each region has a motor output therefore each region is learning a sensorimotor model of the data. We also know a lot about the detailed connectivity between layers that again is consistent no matter what animal or what region. During the hackathon I will be happy to share what I know about these connections. This connectivity doesn't tell you exactly what is going on but it does provide hints and a set of constraints. The thickness of a cortical region can vary a bit intra- and inter-species, but the number layers and the number of cells in a column don't vary that much. Typical numbers in the literature are 100 to 110 cells in a mini-column, which spans all layers. Layer 2/3 has the most cells so 32 is a reasonable guess for layer 2/3. BTW mini-columns are still somewhat controversial. Some people argue that mini-columns don't exist everywhere, only in some regions, and therefore are not important. Mini-columns are very clear in many parts of the cortex. The CLA only needs mini-columns for learning high order sequences. Maybe some tasks don't require high order sequences and the corresponding regions lose the mini-column feature. During the upcoming hackathon I will talk about layer 5 cells. These are the motor output of a region. I am hopeful that a useful sensorimotor system can be built with just two CLAs, one for Layer 2/3 and one for layer 5. Jeff From: nupic [mailto:[email protected]] On Behalf Of Fergal Byrne Sent: Wednesday, October 23, 2013 12:33 PM To: NuPIC general mailing list. Subject: Re: [nupic-dev] Open Office Hour: Wednesday Oct 23rd 9AM PDT Hi Guys, Just watched the recording of the session (really sorry I missed it, got called away at the last minute). I've a couple of comments. Firstly, thanks to all who took part. Despite a couple of glitches, I think this is a really powerful way to expand the discussions we're having on the list. As someone mentioned during the session, the list has limitations in getting to the kernel of some questions, where it seems that a lot of that was addressed in this medium. It was really useful to have several people from Numenta/Grok present at the same time, because we were able to get many contributions from the different points of view practically simultaneously. I'm looking forward to even higher bandwidth discussions during the hackathon. I thought the discussions about hierarchy were very enlightening, and I think this medium is great because you were talking about things we haven't seen externally before. Until you have good answers, the information is all in the questions, and especially the doubts that you expressed in the session. We don't get to see that in Jeff's talks or in any publications, for obvious reasons, but they're all-important for getting to the next stages in the process. Specifically related to the discussion about hierarchy, I have some thoughts which might be of some use. As you mentioned, you've temporarily stopped looking at hierarchy and are concentrating on the usefulness of a single-region, single-layer CLA implementation, and clearly that's working well. Correct me if I'm wrong, Jeff, but it seems that we have all these expensive layers (and complex structure within several of them) for a reason: there's a lot more going on than just spatial pattern recognition and sequence learning. The layers are performing different parts of the computation, perhaps including: 1) Pure feature detection. 2) Sequence memory. 3) Using sequence memory to constrain feature detection. 4) Attention 5) Incorporation of signals from layers above 6) Motor stuff 7) Sending signals downwards in the hierarchy 8) Interaction with subcortical structures 9) Using the predictive cells 10) (nearly forgot!) sending signals upwards [Numbers do not correspond to layer numbers! Just the order I thought of them in]. The laminar structure exists in lots of places in the brain, but the huge area of almost identical laminar cortex does not. We must have evolved the neocortex by having a few very small hierarchies with few layers, and added more complexity as we evolved from the shrew-like original mammals. So we should seek (as Jeff said in the session) to build a very simple hierarchy using just a couple of CLA regions first, and see how it behaves. In the CLA, we might typically have 32 or 40 neurons per column. These are all doing the same job. In the real cortex, we have many more than this. Neurons are very expensive, so they must all be doing something useful. I suggest that we should see how we can emulate some of this list of functions in each CLA column, perhaps by "separating concerns" as the pattern guys call it. If you look at the NuPIC code, you're already doing a huge number of things which are not strictly included in the idealised algorithm. I'm on Ian's side in the debate about reconstruction. We are clearly able to do generative "perceptions" based on top-down activations of high-level concepts. As Jeff has identified, it takes a long time between knowing that we should be able to do something and figuring out even one good way to do it, so I'd say that we need to find out how we can do it. It could turn out to be the key to understanding the whole idea of how hierarchies work, in the same way that cell choice is the key to understanding how sequence memory may work. Delighted with the discussion, nuts that I missed it! Regards, Fergal Byrne On Wed, Oct 23, 2013 at 6:16 PM, Matthew Taylor <[email protected]> wrote: I had some reports of people getting kicked out of the Google Hangout, unable to reconnect. Please report any issues you had to me so I can address them. We may want to switch to another conferencing technology if Hangout has issues. I don't want to exclude anyone. Thanks, --------- Matt Taylor OS Community Flag-Bearer Numenta On Wed, Oct 23, 2013 at 8:49 AM, Matthew Taylor <[email protected]> wrote: Hangout URL: https://plus.google.com/hangouts/_/d7fbbdde067c5ddb249c1c526473b2420ebc3519 YouTube URL: http://youtu.be/MWBFw4WoZxA --------- Matt Taylor OS Community Flag-Bearer Numenta On Wed, Oct 23, 2013 at 8:06 AM, Matthew Taylor <[email protected]> wrote: Reminder: this online event starts in 54 minutes. --------- Matt Taylor OS Community Flag-Bearer Numenta On Fri, Oct 18, 2013 at 6:12 AM, Marek Otahal <[email protected]> wrote: Thanks Matt, this is a great step! I hope it'll get more knowledge sunken to the community. On Thu, Oct 17, 2013 at 10:57 PM, Matthew Taylor <[email protected]> wrote: Have questions about NuPIC or the CLA? Jeff and Subutai will be in attendance to help answer them in our first Open Office Hour at Grok. This event is focused on NuPIC and our open source community, but anyone is welcome to join and ask questions, or just discuss related technologies. https://plus.google.com/b/100642636108337517466/events/c2pt64fid2niuso3r4mp5 n7u9os Please join in if you would like to chat. I'll be recording and streaming on YouTube. I will post the URL for the Hangout and the YouTube link here just before the event. Regards, --------- Matt Taylor OS Community Flag-Bearer Numenta PS: I know this time will not work for everyone interested in attending. For the next office hour, I will try to schedule a completely different time so others may attend. _______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org -- Marek Otahal :o) _______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org _______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org -- Fergal Byrne Brenter IT [email protected] +353 83 4214179 Formerly of Adnet [email protected] http://www.adnet.ie <http://www.adnet.ie/>
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
