On that note, I actually wanted to ask about the CLA classifier. When I was learning about NuPIC, I was hoping that everything would be self-contained, emerging from the beautiful and simple model of the biological structure of the brain. But though it seems like multi-step prediction is something the brain must be doing on its own, NuPIC is still using a (pretty much brute-force) external classifier. Why is this? Is it just that we haven't yet figured out how to fit it elegantly into the existing framework? Or am I missing something?
Thanks, Chetan On Tue, Sep 3, 2013 at 5:48 PM, Chetan Surpur <[email protected]> wrote: > Continuing from my previous email, I'll try to outline how the probability > tables are updated for each cell during learning. > > If we're trying to predict 'n' steps into the future, we'll store the last > 'n' inputs and their cell activations. Now, let's say we're seeing the > input "E", with a particular activation. We visit all the cells that were > active in the last step, and increment the probability for "E" in the > 1-step prediction table. We then visit all the cells that were active 2 > steps ago, and increment the probability for "E" in the 2-step prediction > table. We do this up to 'n' steps in the past. > > In this way, the probability tables we'll use in the current step to > predict the future will be updated when we see the true outputs in the > future. > > That concludes my description of the classifier. Please correct me if I'm > wrong on any points! > > - Chetan > > > On Mon, Sep 2, 2013 at 4:15 PM, Chetan Surpur <[email protected]> wrote: > >> Pedro, >> >> I'll try to answer your question about how the external classifier for >> multiple step prediction works, because the best way to learn deeply is to >> try to teach :) If I'm incorrect on any point, anyone, please feel free to >> correct me! >> >> At each cell are probability distribution tables that contribute to >> predictions about future timesteps. These tables tell you the probability >> of a particular output occurring in the future, given that this cell is >> currently firing. The way we do multiple step prediction is by having >> multiple tables per cell, each one predicting for a particular step in the >> future. >> >> For example, if you're looking at an activation and trying to predict the >> chances we'll see the letter "C" in 3 steps, you look at all the 3-step >> prediction tables for all the currently firing cells, and add up the >> normalized probabilities for the "C" column. If you're trying to predict a >> scalar value instead of a categories (like letters), you break up the >> continuous output range into discrete buckets, and the probability tables >> will refer to those buckets instead. >> >> I'm on a plane, so I'll write up in a separate email the way we update >> these probability tables during learning. Hopefully it's making sense so >> far, and I'm not totally misunderstanding everything! >> >> - Chetan >> >> >> On Mon, Sep 2, 2013 at 6:38 PM, Pedro Tabacof <[email protected]> wrote: >> >>> Ian, >>> >>> Your 10 points are all spot on, great job on understanding my mess! >>> >>> I'm not sure if feeding the CLA open-loop (no kind of control on) data >>> would be useful on practice, because in this case you're probably better >>> off with standard MPC, but this is probably the right way to start to >>> tackle this problem. It's actually not that hard to simulate a complicated >>> dynamic system with noise and disturbances and gather "experimental" data. >>> If I have time I will look into this and share my findings here. >>> >>> It'd be cool to see how the CLA would respond to things such as large >>> time constants (slow dynamic response) and/or considerable deadtime >>> (time-delay) before trying to actually control a system with it. The main >>> difference from this to typical CLA applications is that the system inputs >>> are independent and thus their prediction is meaningless. Would this make >>> the prediction of the system outputs (which depend on their past values and >>> on current and past inputs) harder or just the same? >>> >>> From your explanation it seems the optimization time is not an issue, >>> especially considering you could turn it off after a while because the CLA >>> would probably have already learned the correct control patterns. It could >>> be turned on only when needed to improve the control and perhaps be done >>> offline. >>> >>> If I recall correctly, Jeff wrote on one email that multiple step >>> prediction is actually made by an external classifier, so it is not >>> actually inherent to the CLA. Can someone clarify this point? Multiple step >>> prediction is essential to MPC so I'd like to understand it better. >>> >>> Anyways, I've been pondering about my MPC idea and more and more I tend >>> to believe that it is just too convoluted to work - I always favor simple >>> solutions over complex ones. If we had motor control CLA I think this could >>> be a great target for application, but it seems this is nowhere near our >>> present. >>> >>> Perhaps training NuPIC on data from a classical controller such as PID >>> or even manual control and then using a simple reinforcement learning >>> procedure to train NuPIC's predictions in order to improve the control >>> scheme (squared error and smoothness as you put it) would be a better >>> solution, but I'm not clear on how this could be done. >>> >>> []'s >>> Pedro. >>> >>> >>> On Mon, Sep 2, 2013 at 3:32 PM, Matthew Taylor <[email protected]> wrote: >>> >>>> On Sep 2, 2013, at 11:05 AM, Ian Danforth <[email protected]> >>>> wrote: >>>> >>>> > I'm going to be stupid in public... >>>> >>>> If only everyone were so fearless. :) >>>> >>>> Matt >>>> >>>> _______________________________________________ >>>> nupic mailing list >>>> [email protected] >>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>> >>> >>> >>> >>> -- >>> Pedro Tabacof, >>> Unicamp - Eng. de Computação 08. >>> >>> _______________________________________________ >>> nupic mailing list >>> [email protected] >>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>> >>> >> >
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
