Subatai, I think the problem is, we are feeding temperature and tilt and NuPIC is saying "temperature and tilt are strongly correlated, so I'll forget the tilt and just try to predict temperature". The trouble is temperature is a weather effect and fundamentally unpredictable. It's hotter in the day and during the summer but the precise details are unpredictable. That is, the temperature data is not nearly as cyclic as the Hotgym example etc.
We need some way of telling Nupic "you don't need to predict temperature, you need to predict tilt GIVEN the temperature". You might argue this is the sort of task a traditional stateless perceptron network would do better, but the best result so far has been achieved with an echo state network (ESN). The ESN had 10 temperature inputs, 8 tilt outputs and was trained to give the right outputs (supervised learning). Between the input and outputs is a "reservoir" of randomly connected recurrent neurons which maintains a short term memory. Thus the ESN has a sense of time and sequence like the HTM. But it also has the concept of "cause and effect" which I think HTM lacks (or please correct me). Analogously, Pavlov's dogs were able to predict that food would come after a bell rang. The ringing of the bell was unpredictable but the food was predictable given the ringing of the bell. This is analogous to the bridge with bell = temperature, food = tilt. Can HTM "reason" like the dogs did? On Fri, Sep 26, 2014 at 9:35 PM, Subutai Ahmad <[email protected]> wrote: > > >if you have a problem with big stationary correlational structure in the > inputs-- > >you should transform it out so NuPIC can work on the non obvious features > > Agreed. In some sense this is what pooling does. The spatial pooler > transforms spatially correlated (but very different) patterns into a > consistent SDR pattern. The temporal pooler transforms temporally > predictable (but potentially totally different) SDR patterns into a single > coherent SDR representation. The encoders also can help greatly with this. > > --Subutai > > On Fri, Sep 26, 2014 at 12:01 AM, Archie, Kevin <[email protected]> > wrote: > >> Subutai, >> >> Thanks for the link, interesting reading. I'm amused by the problem of >> it being (1) a problem NuPIC isn't really well suited to and (2) the first >> thing lots of people are going to try anyway. >> >> Thinking about statistics, it seems to me that if you have a problem >> with big stationary correlational structure in the inputs--either temporal, >> as the sine wave example, or spatial, as John's bridge data--you should >> transform it out so NuPIC can work on the nonobvious features. Thinking >> about biology, by the time you get to cortex a lot of the stationary >> structure has been filtered out -- think about the processing in the >> retina. It's possible that NuPIC is good enough to solve lots of >> interesting problems even without preprocessing, but I suspect that some >> care to the input representation could greatly help with performance (in >> cycles, at least by reducing the input dimensionality) and performance (in >> error rate, by getting rid of lots of chaff). >> >> Or I could be all wet. It happens. >> >> - k >> >> On Sep 25, 2014, at 1:53 PM, Subutai Ahmad wrote: >> >> Hi Kevin, >> >> I did some simple experiments with swarming and correlated inputs [1]. >> One thing to note is that temporal correlation / sequence structure is also >> very important. That is independent from spatial correlation. >> >> --Subutai >> >> [1] https://github.com/subutai/nupic.subutai/tree/master/swarm_examples >> >> On Thu, Sep 25, 2014 at 11:33 AM, Archie, Kevin <[email protected]> >> wrote: >> >>> John, >>> >>> I think this is an example of an important general case. On both >>> statistical and biological grounds I suspect you should decorrelate your >>> inputs before producing SDRs from them. I haven't tried this myself (have >>> hardly done anything with NuPIC) but I'm wondering if anyone has done >>> substantial preprocessing to compensate for the statistics of the inputs. >>> Even better would be if anyone has done a comparison of feeding NuPIC >>> decorrelated vs. direct sensor inputs--or a theoretical argument that >>> decorrelating the inputs is unnecessary or unwise. >>> >>> - Kevin >>> >>> ------------------------------ >>> *From:* nupic [[email protected]] on behalf of John >>> Blackburn [[email protected]] >>> *Sent:* Tuesday, September 23, 2014 7:27 AM >>> *To:* Archie, Kevin >>> *Subject:* Re: Which NuPIC tutorial do you want to see next? >>> >>> Hi Matthew, >>> >>> Rather self serving, but I would love to see a tutorial related to the >>> "bridge" simulation I've been trying to do or similar. I have not got NuPIC >>> to work yet despite some effort. Basically the difference with Hotgym is we >>> have 18 sensors, 10 temperature and 8 tilt (ie strain) and we want to make >>> predictions on all taking account of cross-correlations. So a tutorial with >>> multiple correlated time series would be great! >>> >>> At NPL we monitored a bridge every 5 minutes for 3 years recording all >>> 18 sensors so I think this data would be a great showcase for a real-world >>> NuPIC example. We also perturbed the bridge at known times (cutting >>> supports, adding weights etc) so we know when the anomalies should appear. >>> >>> John. >>> >>> On Mon, Sep 22, 2014 at 4:57 PM, Matthew Taylor <[email protected]> >>> wrote: >>> >>>> I have more tutorials planned, but I'd like some help deciding which >>>> to do first. Please answer this 1-question poll: >>>> >>>> >>>> https://docs.google.com/forms/d/1GBYWg_-LIaYmOz9EJ5LbFo6N2ot1xv9AA22gaNdENs0/viewform?usp=send_form >>>> >>>> Thanks, >>>> --------- >>>> Matt Taylor >>>> OS Community Flag-Bearer >>>> Numenta >>>> >>>> >>> >>> ------------------------------ >>> >>> The material in this message is private and may contain Protected >>> Healthcare Information (PHI). If you are not the intended recipient, be >>> advised that any unauthorized use, disclosure, copying or the taking of any >>> action in reliance on the contents of this information is strictly >>> prohibited. If you have received this email in error, please immediately >>> notify the sender via telephone or return mail. >>> >> >> >> >
