Hi Pedro, That's fantastic! Congratulations on the nice result, and thanks also for keeping everyone informed throughout.
We should document all these details on our wiki. I like your idea of a separate thread on scalar prediction tips and will reply a bit later to that email. --Subutai On Sun, Oct 13, 2013 at 10:24 PM, Pedro Tabacof <[email protected]> wrote: > Hello, > > As promised elsewhere, here is the presentation of the results of NuPIC > application to the electricity load forecast competition. This dataset > comes from a 2001 competition on the prediction of max energy loads for the > whole month of January 1999 of the electrical grid of Slovakia. The > training data consists of two years (1997 and 1998) of half-hourly energy > loads and daily temperatures, with holiday information. This seems to have > been a serious competition as the winners got real prizes and they made > sure it wouldn't be possible to access the test data before the submission. > There were 56 registered competitor but only 26 of them submitted their > predictions. > > The winners of the competition used SVM regression (SVR) for one-step > predictions, but there were many different models used. Just as a > curiosity, the winners were part of the group that wrote the popular > "libsvm" library, so they are actually machine learning experts. > > Since this was my first time using NuPIC, I tried many different > strategies. My first one was to just select the daily max energy load and > do 31-days predictions, and this got me a good result (3.3% MAPE - among > the top 10). This good start motivated me to try to improve the results, so > I tried to use the whole half-hourly data, and while this gave me a great > improvement for the first half of the test month, the last half was not > good enough for an overall improvement. > > In the end a variation of the first idea was the simplest and quickest > method and achieved the best results: 2.5% MAPE, which would place me among > the top 3 best results. To get it down from 3.3% I had to remove summer > data and also use a different way of doing prediction (I will start another > thread about it). I also tried to use a multi-model approach, doing > 1,2,...,30,31 days predictions with 31 different models but it didn't > improve anything and was slower. I didn't use swarming and tried to tune > everything by hand, but in the end the parameters stayed very close to the > hotgym example, which is very motivating. > > I've attached the code, data and some graphs with comparisons of NuPIC > against the best results of the competition. It's very important to stress > that since I was in possession of the test data, the results are not really > "scientific". But I think that the fact I had never used the new NuPIC > before cancels that out! > > Pedro. > > -- > Pedro Tabacof, > Unicamp - Eng. de Computação 08. > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
