On Saturday, July 13, 2013, Hideaki Suzuki <[email protected]> wrote: > Dear Gil san, > > Thank you very much for this. I will be very happy once I start reading the code. > As suggested by others, I also like 1D, 2D, ...,etc. > Probably the oppsite term of global is local, so we can think of 2D global and 1D local. > > In fact my CLA gadget I'm playing with for my learning has global inhibition and local inhibition methods, > which chooses Kth highest active columns globally and locally in 2D. > I can see that local inhibition lets the active columns distribute more evenly over the input bits at the beginning. > It's really fun to observe various aspects of CLA. > > Ah, slightly off topic. > > Thank you for your refactering. :-) > > Best Regards, > Hideaki Suzuki. > > > 2013/7/13 Gil Shotan <[email protected]> > > Hey Everyone, > > For those of you who had a chance to check out the spatial pooler > implementation (it's located in nupic/py/nupic/research/FDRCSpatial2.py) > you'll notice that its a very long file containing some obsolete code. I > took it upon myself to rewrite it to make > it more readable and accessible. > > To understand the changes I am proposing, a little background is in order > (this will require having read the white paper): > Basically, the spatial pooler uses a "sparse matrix" implementation under > the hood to keep track of the permanences of each column (referred to as > "coincidence" in the code). Each row in the matrix represents a cortical > column, and each column in the matrix > represents an input bit. Since a column is only connected to a subset of > the inputs, using sparse matrix implementation makes operations such as > computing the overlap more efficient while at the same time reducing the > memory footprint. > > In the past, Numenta was experimenting with vision related problems, so > each region had a 2D-topology of columns, as shown in the white paper. > This topology determined a columns' neighbors for the purpose of > inhibition, as well as the "natural center" > over the inputs that each column is biased towards. > > With vision related inputs, each sub-region in lower regions might learn > to represent the same patterns. For example, in the V1 region there are > multiple "edge detectors" responsible for detecting edges at different > areas of an eye's the field of vision. > Since these "edge detectors" are very similar, there is an optimization > in the code that allows for training a small number of columns in a region > over a subset of the inputs and "cloning" the learned permanences across > the entire region. > > Currently all this functionality is combined into one giant class (3000+) > lines that is very difficult to understand. Furthermore, as Grok shifted > away from dealing with vision, most of this functionality is not being > used. For time-series data, Grok uses > a subset of the aforementioned functionality that doesn't assume any > topology or structure to the columns. Basically, each column can be > connected to every input and the "winning" columns after the overlap has > been computed are selected to be columns with > highest overlap of the entire region (This is referred to as "global > inhibition"). > > My aim is to refactor the spatial pooler class (which for some reason is > called FDRCSpatial2.py) to make it more manageable and easier to change, > while also supporting the aforementioned functionality, that even though > outdated, might serve some of you > who are interested in applying the NuPIC framework to interesting and > diverse problems.Therefore I am planning on breaking down the spatial > pooler class into several classes, with the following inheritance > structure: > > +-------------------+ > | SpatialPooler | > +---+------------+--+ > | | > v v > +--------------------+ +---------------------+ > | FlatSpatialPooler | | VisionSpatialPooler | > +--------------------+ +--------+------------+ > | > v > +-----------------------+ > | ClonedSpatialPooler | > +-----------------------+ > > > Where the "SpatialPooler" class will contain the underlying implementation > and all the sparse matrix operations. The "FlatSpatialPooler" will contain > only the minimal adjustments needed to work for streaming time series data > (which is what is currently > being used). The "VisionSpatialPooler" will contain additional structure > data, allowing for local inhibition in a 2D region, and the > "ClonedSpatialPooler" class will allow for training only a subset of a > region and copying the learned permanence values across > the rest of the region. > In doing so, I will aim to rename some of the variables and methods to > conform to the terminology used in the white paper. > > Please feel free to share any comments or suggestions you may have! > > Gil Shotan > Algorithms Intern, Grok Solutions >
-- Marek Otahal :o)
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
