I had promised this a long time ago but here is a version of an adaptive encoder that isn't quite as brittle as the current adaptive encoder:
https://github.com/numenta/nupic/pull/155 It is loosely based on the idea of arithmetic coding and adapts to arbitrary value ranges, while typically not changing representations by more than a bit when a new value comes in. The main downside is that it requires a moving window of values seen and the size of this is arbitrary. When I tested it I did not find that it worked very well. Obviously if you can fix the encodings like the regular scalar encoder then you don't have to worry about losing learning when the representations adapt. But here it is in case you are curious. On Fri, Aug 2, 2013 at 10:33 AM, Fergal Byrne <[email protected]>wrote: > > Hi Patrick and Steve, > > We're trying to build NuPIC "based on the principles of the neocortex", > and one of the early choices is to have a fixed-width bit array for the > inputs, since this is how almost all sensory input works in the brain (it's > very hard to see how you can grow new neurons in the optic nerve!). > > So, the issue in this thread is about altering the encoding scheme of > scalar value to a fixed, n-bit array, not whether the value represents an > absolute or log-value. The choice of linear or logarithmic evaluation is > external to this - we should choose to preprocess the raw data as > logarithmic if that makes sense (eg for seismic data or sound amplitude). > The resulting input to the encoder is again just a scalar. > > The issue Patrick mentions about fewer bits as the value rises does not > then occur, as we still use the same fixed window of on-bits for the > log-value regardless of where it is. The only thing that is "sacrificed" is > the absolute precision at high values, but this is always a tradeoff when > converting measurements to log-values, and it's informed by the use of the > data (ie small changes in small values are more important or noticeable > than the same absolute change at high values). > > Regards, > > Fergal Byrne > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
