Hi All, Thank you for your comments and suggestions. I incorporate your input into my design, that will likely change as I dig deeper into the codebase and understand it better.
For example, one of the reasons I proposed separating the global inhibition vs. local inhibition is that global inhibition can be implemented much more efficiently. All it takes is to sort the firing rates in the entire region and return the top K rates. However, digging deeper into the Inhibition functionality I learned that inhibition is indeed implemented in a separate class called "Inhibition2" which can be found in "nupic/nta/algorithms/FDRCSpatial.hpp". This class already handles the special case of global inhibition in an efficient manner. Mark - it's your lucky day! However, currently the Inhibition2 class can only support 2D topology or less. If anyone would ever be interested in creating higher order topologies as Jeff suggested, be aware that it would be necessary to change this class. The "Flat" spatial pooler wasn't a good name. My intent was to separate specific, ad-hoc functionality that we use in Grok, such as 'highTiering' which might not be relevant for other uses. (Don't worry, I will document all those features). So as Subutai suggested, I think it makes more sense to implement a base "SpatialPooler" class that handles a 2D topology (that can easily degenerate into a 1D topology), as well as local inhibition (that can easily degenerate into global inhibition) and a sub class that hides all the quirks that we use for Grok. Since it would be a lot of work changing the implement ion to support higher order topologies, I plan to postpone it for the time being. Please let me know what you guys think. Gil. On 7/15/13 8:36 AM, "Matthew Taylor" <[email protected]> wrote: >Moving an internal discussion about this topic into the public mailing >list. See below for details and please continue this discussion here. > >-- >Matt Taylor >OS Community Flag-Bearer >Numenta > >-------- Original Message -------- >Subject: Re: New design for spatial pooler >Date: Sat, 13 Jul 2013 17:53:27 -0700 >From: Subutai Ahmad <[email protected]> >To: Rahul Agarwal <[email protected]>, Scott Purdy ><[email protected]> >CC: Gil Shotan <[email protected]>, Engineering ><[email protected]> > > > > >I'm a bit confused here. My understanding was also similar to Scott's, >in that the base class would handle arbitrary topologies. A flat spatial >pooler is simply a parameter setting such that the receptive fields of >columns include the whole input space and there is no specific potential >field for the synapses. Global inhibition as we have it today would >automatically work since "local" includes everything. As such there is >no code for a "flat spatial pooler" to implement and (I believe) there >is no real efficiency loss. What would this class contain? > >Similarly, what would the "topology spatial pooler" implement that is >different from the base class? > >In addition, I don't think we should spend time implementing cloning >right now. > >So, why do we need more than one class right now?? > >-- Subutai > > >From: Rahul Agarwal <[email protected] ><mailto:[email protected]>> >Date: Friday, July 12, 2013 4:35 PM >To: Scott Purdy <[email protected] ><mailto:[email protected]>> >Cc: Gil Shotan <[email protected] ><mailto:[email protected]>>, Engineering ><[email protected] <mailto:[email protected]>> >Subject: Re: New design for spatial pooler > >Oh ok, then that was a misunderstanding I had. > >If that is the case, however, then we should re-think where the >"CloningSpatialPooler" would fall in the hierarchy. I don't think the >cloning functionality is specific to vision use cases. It was a >technique for speeding up learning, and it could be used in multiple use >cases that have topology built in. > >-Rahul > >From: Scott Purdy <[email protected] ><mailto:[email protected]>> >Date: Friday, July 12, 2013 4:14 PM >To: Rahul Agarwal <[email protected] ><mailto:[email protected]>> >Cc: Gil Shotan <[email protected] ><mailto:[email protected]>>, Engineering ><[email protected] <mailto:[email protected]>> >Subject: Re: New design for spatial pooler > >Rahul, my impression was that the "SpatialPooler" base class would be >designed for arbitrary topologies. Then "FlatSpatialPooler" would >specifically implement our current topology and "VisionSpatialPooler" >would have a 2D topology and potentially other vision-specific changes. > > >On Fri, Jul 12, 2013 at 3:48 PM, Rahul Agarwal ><[email protected] <mailto:[email protected]>> wrote: > > Gil, > That breakdown looks good. The only suggestion I would have would be > to rename the "VisionSpatialPooler" to the "TopologySpatialPooler". > This would more accurate reflect the real difference between it and > the FlatSpatialPooler, and make it clear that it's uses are not > restricted to vision use cases. > > -Rahul > > From: Gil Shotan <[email protected] > <mailto:[email protected]>> > Date: Friday, July 12, 2013 10:32 AM > To: Engineering <[email protected] > <mailto:[email protected]>> > Subject: New design for spatial pooler > > > Hey Guys, > > For those of you who are interested, here is the design I came up > with for the spatial pooler (see image attached). Basically there > will be one common ancestor class called SpatialPooler that will > handle the implementation. It will handle all the calls to the > Sparse Matrix functions. > > The FlatSpatialPooler will include only the bare functionality that > we currently use, I.e. No topology and global inhibition. > > The VisionSpatialPooler will include local inhibition, a 2D > topology, Gaussian-initialized permanence option, etc. > > The ClonedVisionSpatialPooler will include cloning functionality. > > This design will tease apart all these separate functionalities, > while under the hood use a fast and efficient implementation using a > sparse matrix. It would also allow us to change the underlying > implementation (the SpatialPooler class) without affecting the > public interface exposed through the FlatSpatialPooler class. > > If you have any questions or comments please send them my way! > > Gil. > > > > > > >_______________________________________________ >nupic mailing list >[email protected] >http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org _______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
