Hi Fergal, I look forward to your post!
Thanks for the links - I'm going to insert these and compare the test results. The link provided for the Mersenne Twister java doc doesn't provide the download link anymore from that java doc page's link. Use these instead if you are trying to download... David Ray [1] http://cs.gmu.edu/~sean/research/mersenne/MersenneTwister.java [2] http://cs.gmu.edu/~sean/research/mersenne/MersenneTwisterFast.java [3] http://cs.gmu.edu/~sean/research/ (this is the developer's general website) On Mon, Aug 11, 2014 at 4:38 AM, Fergal Byrne <[email protected]> wrote: > Hi David, > > Good points, I'll be answering those in my post (writing it now!). On the > last point, Chetan's version uses numpy's random number generator, which is > by default the Mersenne Twister MT19937 [1]. Most modern languages also use > this exact generator, apart from Java (which you'd be using in CoPIC-J and > I'm using indirectly in Clortex)! > > Happily, there is a very good MT algorithm here [2] which we should both > use for this (and for anything NuPIC-related) as it gives us identical > values to those produced in numpy, Ruby, C++, R, and many other languages. > > Regards, > > Fergal Byrne > > [1] http://en.wikipedia.org/wiki/Mersenne_twister > [2] http://cs.gmu.edu/~sean/research/mersenne/ > > > On Mon, Aug 11, 2014 at 9:58 AM, cogmission1 . <[email protected] > > wrote: > >> This also parallels the adjustment of our attention as the applicable >> bounds of perceptions change with increasing acceleration. For instance, >> when we are playing a race driving game and our speed accelerates, our >> attention is adjusted to always be further out or further down the road >> because paying attention to finer grained distances in front of us becomes >> less and less fruitful (and difficult) - so our attention becomes spaced >> down the road further. We are able to "adjust" the context of our attention >> to suit time spans and intervals which are appropriate to a given >> circumstance (or rate of input). >> >> @Fergal or @Anybody - I wonder if the comparison between the type of >> data traversal typical with the RDSE and that of the GOSE (<-- phonetic >> acronyms, I love em) is identical? Mind you, I just encountered the RDSE >> video a couple of days ago so I haven't spent much time contemplating it. >> But as far as I recall, the RDSE is an attempt to encode values regardless >> of order and scale, so buckets are formed (given a particular resolution or >> "range" really) relative to each other. Whereas the GOSE has an advantage >> in that the "world" is divided equally into identically sized squares >> implying that the range is "fixed" in comparison to the types of data that >> the RDSE has to account for? In other words there are no outliers or range >> or scale changes and each square has an assigned unique index also? I'm >> just wondering if the encoding scheme can be applied equilaterally to >> scalars that are unknown? >> >> Also, I'm doing a port of NuPIC to Java (CoPIC-J temporary name until >> Matt gets back from vacation) https://github.com/cogmission/copic-j - >> and I find the differences in pseudorandom number generation to be a slight >> annoyance (as my main requirement is identical unit/integration test >> results). Fergal, I think your idea of investigating a cross-platform >> random number solution is ideal especially as implementation of NuPIC in >> different languages becomes more and more prevalent. Probably better to >> saddle that horse early on? >> >> David >> >> >> >> >> >> On Sun, Aug 10, 2014 at 8:26 PM, Chetan Surpur <[email protected]> >> wrote: >> >>> Fergal, >>> >>> Thanks for the comments, you seem to have a good intuition about how the >>> encoder is working. >>> >>> Just one point about changing radius and its effect on the encoding. I'm >>> almost certain (but will have some tests to prove it one way or the other) >>> that the SDR is very robust to changes in radius, due to the sparsity of >>> the SDRs. >>> >>> >>> It depends on how much you change the radius by. If you change it by a >>> large amount, it can result in a totally different encoding. But it's >>> correct that if you change it by some small amount, the encoding doesn't >>> change very much. That's one of the nice properties of this system. >>> >>> - Chetan >>> >>> On August 8, 2014 at 10:08:03 AM, Fergal Byrne ( >>> [email protected]) wrote: >>> >>> Hi Matt, >>> >>> Thanks for getting those up in such good order. Well done Chetan, that >>> looks even more inventive than the RDSE! >>> >>> The discussions with Jeff and other Numenta staffers is really >>> intriguing, and I think that there are some really important implications >>> for both NuPIC and HTM in general from this step forward. I'm writing up my >>> ideas on this and will post a link once that's done. A quick teaser: >>> >>> 1. This is a very HTM-compatible idea. You could view each square's >>> "order" as a normalised "activation potential" and the process of choosing >>> the top w squares as analogous to local inhibition among neurons (with all >>> virtual neurons outside the search square forced to zero). So, the square >>> forms a "virtual SDR" which is expanded into a full SDR by the fact that it >>> projects randomly into the larger bit field of the encoder. >>> >>> 2. The general "coordinate" paradigm operates on the same principle, in >>> that it populates an n-dimensional space with a grid of hash-based >>> (pseudorandom) "signature neurons" which can be sampled using this local >>> inhibition scheme to derive a location fingerprint algorithm. This seems to >>> me to be a genuine invention in the field of high-performance semantic >>> encoding of high-dimensional spatial position (which is either treated as a >>> total mystery or abstracted away using fragile tricks). >>> >>> 3. It is clear how you could combine this encoding scheme with data >>> which varies by location, to create a richer idea of "order" in feeding the >>> SDR generation algorithm. For example, you could combine random "order" >>> with altitude or temperature data to choose the top w squares. >>> >>> 4. This is a wonderful model for how we "zoom in" or "zoom out" and >>> perceive a continuously but smoothly varying model of the world. It also >>> models how we can perceive gracefully degrading levels of detail depending >>> on how much time or attention we pay for a perception. In this case, the >>> "encoder" detailed here would be a subcortical structure or a >>> thalamus-gated (attention controlled) input or relay between regions. >>> >>> 5. This method solves the presentation-order dependence issue of the >>> original RDSE, where the encoding of a value depends on which values were >>> presented before it, unless you precalculated values from an initial >>> "centre" (which could have significant memory or compute costs). The >>> coordinate encoder has a deterministic, O(1), order-independent algorithm >>> for computing both "order" and bit choice. The only issue I see is that the >>> pseudorandom number is Python-specific, and so a C++ encoder (or a >>> Java/Clojure encoder) will likely produce completely different answers (but >>> see next point). >>> >>> 6. I believe it would be worth exploring using Perlin noise (or some >>> similar cross-platform pseudorandom generator) to generate the order and >>> bit choice values. This would give you a) identical encodings across >>> platforms, b) pseudorandom, uncorrelated values when the noise samples are >>> far enough apart (eg when the inputs are integers as in this case), and c) >>> smoothly changing values if you use very small step sizes. >>> >>> Just one point about changing radius and its effect on the encoding. I'm >>> almost certain (but will have some tests to prove it one way or the other) >>> that the SDR is very robust to changes in radius, due to the sparsity of >>> the SDRs. In other words, the overlap in an SDR at radius r with that at >>> radius r' (at the same GPS position) will be high, because you are only >>> adding or removing an annulus around the same position (this will be >>> similar to adding or removing a strip of squares when a small position >>> change occurs). >>> >>> Regards, >>> >>> Fergal Byrne >>> >>> >>> On Fri, Aug 8, 2014 at 2:52 AM, Matthew Taylor <[email protected]> wrote: >>> >>>> In this video, Numenta engineer Chetan Surpur explains NuPIC's >>>> Geospatial Coordinate Encoder, which is used to generate predictions >>>> and anomaly indications from temporal geospatial data (like you get >>>> from GPS). >>>> >>>> https://www.youtube.com/watch?v=KxxHo-FtKRo >>>> >>>> This video describes the NuPIC Geospatial Tracking application and >>>> shows how it can be used to analyze GPS data. >>>> >>>> https://www.youtube.com/watch?v=M4dD9wCQLkA >>>> --------- >>>> Matt Taylor >>>> OS Community Flag-Bearer >>>> Numenta >>>> >>>> _______________________________________________ >>>> nupic mailing list >>>> [email protected] >>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>> >>> >>> >>> >>> -- >>> >>> Fergal Byrne, Brenter IT >>> >>> Author, Real Machine Intelligence with Clortex and NuPIC >>> https://leanpub.com/realsmartmachines >>> >>> Speaking on Clortex and HTM/CLA at euroClojure Krakow, June 2014: >>> http://euroclojure.com/2014/ >>> and at LambdaJam Chicago, July 2014: http://www.lambdajam.com >>> >>> http://inbits.com - Better Living through Thoughtful Technology >>> http://ie.linkedin.com/in/fergbyrne/ - https://github.com/fergalbyrne >>> >>> e:[email protected] t:+353 83 4214179 >>> Join the quest for Machine Intelligence at http://numenta.org >>> Formerly of Adnet [email protected] http://www.adnet.ie >>> _______________________________________________ >>> nupic mailing list >>> [email protected] >>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>> >>> >>> _______________________________________________ >>> nupic mailing list >>> [email protected] >>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>> >>> >> >> _______________________________________________ >> nupic mailing list >> [email protected] >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> >> > > > -- > > Fergal Byrne, Brenter IT > > Author, Real Machine Intelligence with Clortex and NuPIC > https://leanpub.com/realsmartmachines > > Speaking on Clortex and HTM/CLA at euroClojure Krakow, June 2014: > http://euroclojure.com/2014/ > and at LambdaJam Chicago, July 2014: http://www.lambdajam.com > > http://inbits.com - Better Living through Thoughtful Technology > http://ie.linkedin.com/in/fergbyrne/ - https://github.com/fergalbyrne > > e:[email protected] t:+353 83 4214179 > Join the quest for Machine Intelligence at http://numenta.org > Formerly of Adnet [email protected] http://www.adnet.ie > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
