Sure! The black line is the sine wave it is supposed to predict. The red is the SDRRBFNetwork's prediction. The blue is a standard multilayer perceptron's prediction.
The SDRRBFNetwork doesn't have regularization yet, so it is a bit noisy. But that isn't the purpose of this experiment. The data was presented to both algorithms in sequential order: X starts at 0, and is incremented by some small value (e.g. 0.01) until 2PI, after which it loops back to 0. The networks are then supposed to output sin(X). So the issue with the MLP approach is that it will overtrain on the more recent values, destroying old information. This is called "catastrophic forgetting", and is a well-known problem in the ML community. Some existing approaches to fixing this problem are sharp activations, unsupervised pre-training, and experience replay. The most successful of these is probably experience replay, which is slow, uses lots of memory, inelegant, and works with a fixed time window of replay samples. The SDRRBFNetwork is pretty much just the spatial pooler of my continuous version of HTM, with a single-layer linear perceptron plopped on top. It barely suffers from catastrophic forgetting, due to the way it "bins" training samples to sub-network pathways as defined by the sparse distributed representation. This way, only a small part of the network receives attention at a time, living old information intact. My original source code was quite buggy, but I have since then fixed it. There may still be bugs, but the approximation is pretty darn good I think, at least in comparison with the MLP. Source is here, under the "rbf" directory in /Source: https://github.com/222464/AILib ZLib license. On Mon, Oct 27, 2014 at 7:07 PM, Matthew Taylor <[email protected]> wrote: > Eric shared this with me on Gitter, thought I would post it: > http://s1218.photobucket.com/user/222464/media/plot-1.png.html > > Eric, feel free to explain if you like. Pretty good prediction of a sine > wave. > --------- > Matt Taylor > OS Community Flag-Bearer > Numenta > > > On Sun, Oct 26, 2014 at 6:29 PM, Eric Laukien <[email protected]> > wrote: > > Hello! > > > > In my quest to make a HTM based reinforcement learner, I need a value > > function approximator. > > I could just use a multilayer perceptron (MLP), but there is a problem > with > > this: MLPs forget old information readily in order to assimilate new > > information (this is called "catastrophic forgetting"). A way around > this is > > by storing input/output pairs in a "experience" buffer, and then doing > > stochastic sampling on that. But, this approach is inelegant, slow, and > > requires a lot of memory. > > > > So, I have devised a new algorithm based on HTM's spatial pooler. I call > it > > SDRRBFNetwork (sparse distributed representation radial basis function > > network). > > > > Essentially, it performs unsupervised learning using the continuous > spatial > > pooling algorithm I developed, and then uses a standard linear > combination > > of the SDR to get output. > > > > The main advantage of this is that there is almost no catastrophic > > forgetting. Since only few cells receive attention at a time, most > weights > > are barely touched, keeping old information intact. > > > > I compared it to a standard MLP on a sine curve learning task. It needs > to > > produce the output of the sine curve for every input, but the inputs it > is > > given to train on are in order (high temporal coherence). SDRRBFNetwork > got > > 270 times less error than the MLP, in less training time. > > > > If that doesn't make a case for SDRs, I don't know what can! > >
