Fergal, I see several things wrapped into your spin off from motor implementation.
The first is I agree that feedback (in the form of prediction) needs to be implemented at some point to complete the model. I am troubled by the feedforward only schema of the current CLA. I do agree with Jeff's prioritization of tasks, as sometimes it is best to shelf something when progress is illusive. I tend to use the example of words in the context of a sentence, where a higher region is handling the sentence and the one below it is dealing with words. The one below that perhaps is working with phonemes and the one below that is working with letters. I'm sure you can imagine the levels below that, but at some point the data set is pixels. So, imagine this hierarchy of regions is reading some hand written text and encounters a letter that is flawed let's say, missing some pixels that complete the shape. Imagine that as the data works its way up the distorted letter its interpreted from feedforward information to be the letter "C", the region above sees the word "SCD". The best match may be SOD instead, so with a little feedback from above, the C is nudged into an O. I'm not sure if this relates in anyway to the concept of reconstruction, but I do see it as a mechanism for using prediction to improve recognition thru feedback. I would like to explore ways of implementing this. The second is that I agree that the encoder could be much improved, which I think is why it has been a hot topic of discussion. Looking at biology I agree with Jeff. I don't see any examples in the human body where the range of a domain is increased. If anything, the range of our sensors in our bodies degrade with age, yellowing lens in the eye, hairs that die in the cochlea, etc. I see biological "encoders" as basically fixed to a range with neurons that connect to differing slices of the number line and report their magnitude to processing beyond. So I don't see much value in pursuing an adaptive encoder with a flexible, slow moving range adjustment. I am curious tho about what happens to cells that were previously handing a range that is no longer active due to loss of sensitivity or injury. I would like to investigate online learning encoders but I'm not sure how much value that has right now, seems like there are other things more pressing. The third sounds like you want to see the classifier changed into a Real Temporal Pooler? I can't wait to hear more about your ideas on this! Or did I read that wrong? Given that this thread is about motor implementation I will attempt to get it back on topic by saying I agree with Jeff's approach to solving some of the current problems. I can't help but think that the other layers in the region work much like the ones currently modeled, but in a different domain, such as motor and attention (a form of inhibition or reverse boosting?). I heard a very interesting brainscience podcast about thoughts. I'll try to find it again, but what caught my interest was a comment about how we decide what to think about, given the notion that thoughts are generated in the subconscious and move into the conscious areas based on emotional attachment to the concept. I have never been at all interested in modeling emotions in any form, but some of the comments lead me to the idea that all thoughts (high level concepts that are not unlike the lower level ones produced by the CLA) have an emotional context that weights its importance in many ways, not the least of which is how certain we are of its truthfulness and also how it guides our attention with the goal of feeling good (perhaps about the correctness of the prediction or the motor behavior is generates that gives us things we want). This goal oriented behavior seems tied not only to motor control but is driven by some emotional state attached to each concept. It lead me to believe that some emotions, the ones attached to concepts are as distributed and decentralized like most other aspects of the cortex. I think some of these ideas are beyond our current state of affairs, but I am starting to see how these things may be connected and how important it is to keep moving on with new implementations as it will help guide improvement of the current ones. I really enjoyed your post, it has had me thinking about this quite a lot. Thank you!! Patrick _______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
