Dear Gil san,
Thank you very much for this. I will be very happy once I start reading
the code.
As suggested by others, I also like 1D, 2D, ...,etc.
Probably the oppsite term of global is local, so we can think of 2D global
and 1D local.
In fact my CLA gadget I'm playing with for my learning has global
inhibition and local inhibition methods,
which chooses Kth highest active columns globally and locally in 2D.
I can see that local inhibition lets the active columns distribute more
evenly over the input bits at the beginning.
It's really fun to observe various aspects of CLA.
Ah, slightly off topic.
Thank you for your refactering. :-)
Best Regards,
Hideaki Suzuki.
2013/7/13 Gil Shotan <[email protected]>
>
> Hey Everyone,
>
> For those of you who had a chance to check out the spatial pooler
> implementation (it's located in nupic/py/nupic/research/FDRCSpatial2.py)
> you'll notice that its a very long file containing some obsolete code. I
> took it upon myself to rewrite it to make
> it more readable and accessible.
>
> To understand the changes I am proposing, a little background is in order
> (this will require having read the white paper):
> Basically, the spatial pooler uses a "sparse matrix" implementation under
> the hood to keep track of the permanences of each column (referred to as
> "coincidence" in the code). Each row in the matrix represents a cortical
> column, and each column in the matrix
> represents an input bit. Since a column is only connected to a subset of
> the inputs, using sparse matrix implementation makes operations such as
> computing the overlap more efficient while at the same time reducing the
> memory footprint.
>
> In the past, Numenta was experimenting with vision related problems, so
> each region had a 2D-topology of columns, as shown in the white paper.
> This topology determined a columns' neighbors for the purpose of
> inhibition, as well as the "natural center"
> over the inputs that each column is biased towards.
>
> With vision related inputs, each sub-region in lower regions might learn
> to represent the same patterns. For example, in the V1 region there are
> multiple "edge detectors" responsible for detecting edges at different
> areas of an eye's the field of vision.
> Since these "edge detectors" are very similar, there is an optimization
> in the code that allows for training a small number of columns in a region
> over a subset of the inputs and "cloning" the learned permanences across
> the entire region.
>
> Currently all this functionality is combined into one giant class (3000+)
> lines that is very difficult to understand. Furthermore, as Grok shifted
> away from dealing with vision, most of this functionality is not being
> used. For time-series data, Grok uses
> a subset of the aforementioned functionality that doesn't assume any
> topology or structure to the columns. Basically, each column can be
> connected to every input and the "winning" columns after the overlap has
> been computed are selected to be columns with
> highest overlap of the entire region (This is referred to as "global
> inhibition").
>
> My aim is to refactor the spatial pooler class (which for some reason is
> called FDRCSpatial2.py) to make it more manageable and easier to change,
> while also supporting the aforementioned functionality, that even though
> outdated, might serve some of you
> who are interested in applying the NuPIC framework to interesting and
> diverse problems.Therefore I am planning on breaking down the spatial
> pooler class into several classes, with the following inheritance
> structure:
>
> +-------------------+
> | SpatialPooler |
> +---+------------+--+
> | |
> v v
> +--------------------+ +---------------------+
> | FlatSpatialPooler | | VisionSpatialPooler |
> +--------------------+ +--------+------------+
> |
> v
> +-----------------------+
> | ClonedSpatialPooler |
> +-----------------------+
>
>
> Where the "SpatialPooler" class will contain the underlying implementation
> and all the sparse matrix operations. The "FlatSpatialPooler" will contain
> only the minimal adjustments needed to work for streaming time series data
> (which is what is currently
> being used). The "VisionSpatialPooler" will contain additional structure
> data, allowing for local inhibition in a 2D region, and the
> "ClonedSpatialPooler" class will allow for training only a subset of a
> region and copying the learned permanence values across
> the rest of the region.
> In doing so, I will aim to rename some of the variables and methods to
> conform to the terminology used in the white paper.
>
> Please feel free to share any comments or suggestions you may have!
>
> Gil Shotan
> Algorithms Intern, Grok Solutions
> [email protected]
>
>
>
>
> _______________________________________________
> nupic mailing list
> [email protected]
> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
>
_______________________________________________
nupic mailing list
[email protected]
http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
