As always Ritchie's results are very interesting. I have to point out that if Nupic cannot make a better prediction than a simple time-series baseline (just forwarding the last output), then it is of no use for time-series forecasting.
I think we should probe further to understand what is causing this, because I'm sure Nupic is better than this. From my own experience with time-series forecasting using Nupic, it should not be worst than a baseline (and should be close to state of the art algorithns). Pedro. Em 26/04/2014 20:44, "Marek Otahal" <[email protected]> escreveu: > > Hi Ritchie, > > > On Sat, Apr 26, 2014 at 8:25 PM, Ritchie Lee <[email protected]> wrote: >> >> Hi friends of NuPIC, >> >> ......However, looking closely why does the anomaly score only spike at the end of the first and second anomalies? Why not spike at the beginning (at least) and throughout the anomaly? > > > Just to check, your anomalies are introduced at {6000, 6500, 7000, 7500) for about 100 steps, right? > Looks like (un)lucky coincidence to me, but it seems at times 6000, 6500 you introduce the anomaly "at the middle of the wave", where sin(x)=0, so the first first anomaly step is actually not an anomaly, the for 99 steps same value is not an anomaly either, > only the jump at the end (from 0 to -1) triggers an anomaly. (#1) > > At 7000, 7500 thhe introduced anomaly might have cought the predicted sine at a different phase, triggering anomaly right off (and multiple times?) > ...if this is true, Nupic already outsmarted us! :) > >> >> Also, why is the prediction so good, and residual so small at the beginning? > > > Answer to this can be found in Nick Mitri's email: [nupic-discuss] Confusion about shifted predictions > (#2) At the beginning, almost all predictions fail, and CLA returns last-seen value. In your example you have 1000 steps per 5 "full sines", thus the resolution is pretty high, change is small -> residual is actually better at the beginning > than after learning :) > > > It would be interesting to see a rerun of your experiment with much smaller resolution (100, 10 steps per period?). > > My question: > > is the "perfect predictor for unlearned" on high resolutions actually a feature, or (misleading) bug? (#3) > > Also, anomaly is "actual(T)-predicted(T-1)/..."; now, what is fed as input(i) when learning=OFF? actual(i), or predicted(i-1)? I think it's actual(i) but no weight changes are stored. But for our example, it should be predicted(i-1). > Example: sine; introduce anomaly as a line at level 0, at the phase where sine and 0 meet. Then as in #1 the 1st point is not anomaly, on step 2, the context is unknown (new), so prediction would give 0 according to #2 (actual(i-1)) which leads > to 0 anomaly score! (as "bug" in #3). > > > Cheers, Mark > -- > Marek Otahal :o) > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
