I hope to clarify what topology is. I agree with some of what Fergal wrote but disagree with other parts, so maybe we are thinking different things.
In a brain, the cortex is a sheet of tissue. Everywhere you look, it is about 2mm thick and consists of 4 or 5 layers of cells (depends on how you count). The total area of human neocortex is about 1000 cm2. Cortical regions are 2D areas. Some of the regions, such as the primary visual or touch regions are large. The retina sends about a million axons to the primarily visual cortex, or V1. Each axon represents a small part of the visual space, sort of like a pixel in an image. When this bundle fibers reaches V1 the 2D topology of the axons is maintained. It is as if a 1M pixel image were projected on to the area of V1. (The "image" is inverted, split, and highly distorted, but in all cases any two adjacent areas of V1 receive input from adjacent areas of the retina.) Scientists would say V1 has a "retinotopic" orientation. Now comes the interesting part. The cells in V1 learn common patterns coming from the retina. The way the SP works is each bit in the SP tries to learn the largest spatial pattern it can. And each bit competes with the other bits. The number of patterns coming from the retina is so large that the SP bits are forced to learn only small spatial patterns that are localized on the retina. So cells on the right side of V1 learn small patterns on the left side of the retina (remember the image is inverted). In biological V1 all the learning rules are local. Inhibition between columns is restricted in radius. The axons that a cell can possibly connect to are restricted to a radius. When we say a region has topology it means that inhibition between columns is limited to a local radius and not the entire region. It means that the set of potential connections that a cell or column can make are also within a radius and not all possible input bits or all possible cells in the region. This is the way real neocortex works. When we were applying the CLA to vision-like problems we implemented it with topology. It worked as expected but it is more complicated to test. The current implementation of the CLA has no topology. This means that each column competes with all the other columns and the set of potential connections between cells is a subset of all the cells in the region. There is no ordering of the columns, nothing is "adjacent" to anything else. For applications that are small no topology works great. If the application had a large 2D sensory array then topology would likely be required. I hope that helped. Jeff From: nupic [mailto:[email protected]] On Behalf Of Fergal Byrne Sent: Wednesday, February 05, 2014 5:54 AM To: NuPIC general mailing list. Subject: Re: [nupic-discuss] Tests for spatial pooling HI Kevin, >> The following ideas are not things I think relate to "topology". Topology is the jargon we use to describe the semantic meaning (or lack of it) of a column (or input bit) position, and for connections between them. For example, a typical encoding of category data randomly picks 21 bits on out of 128. Each bit position has no real connection with the input category it encodes. This representation has no topology at all. Conversely, the sliding window of on-bits in a scalar encoding has one-dimensional topology. In this case, bit 0 might mean the input is < 10, while bit 23 might mean an input between 11 and 33. >> These ideas are things I think relate to "topology". An example of a 2d topology in an SDR is a retina-like sensor. In this case the light levels or presence of particular features in a certain spatial position would be represented by bits in the corresponding position. Topology in the SP relates to how each column position is connected with its input bits. An SP which is learning to recognise features in an SDR coming in from a retina would have columns "looking at" a set of input bits directly "below it" in the retina. The V1 primary visual cortex has exactly this property. If you have topology, you'll usually use local inhibition to pick winning columns, where local is defined according to the topology (1-d, 2-d distance). In the absence of topology, it's more efficient to just pick the highest activation potentials globally. Regards Fergal Byrne On Wed, Feb 5, 2014 at 11:25 AM, Kevin Martin <[email protected]> wrote: Hi all, Thanks for the help. But I'm not sure if I understand what you mean by topology. As far as I understand, the spatial pooler has 2 functions : Return a set of columns that are 1)sparse and 2)distributed. Sparsity can be maintained by making sure that the pooler returns only n% of columns and this criterion can be made into a test. And distribution can be tested by computing the average distance between winning columns. What is topology? Is it another way of referring to a distributed set of columns? I might need some help with local learning too. I'm afraid I have never heard the term before (its in the white paper? A quick search on the pdf did not see any hits for "local learning") Thanks, Kevin Martin On Wed, Feb 5, 2014 at 7:37 AM, Scott Purdy <[email protected]> wrote: I would go over the properties of the SP and create tests accordingly. The output, for instance, should be tolerant to spatial noise. So show an instance a set of very different spatial patterns over and over a number of times. Then show one of the patterns with a single bit flipped and make sure the output is very close (possibly identical if columns are saturated) to the same. There are a bunch of similar things you could check. Ian had a hackathon demo showing the connected bits for each column. He was able to validate different aspects of the SP visually by using different numbers of columns and % overlap. You could do something similar to make sure you see the right behavior but I don't know if you could make an automated test out of it. http://numenta.org/blog/2013/11/06/2013-fall-hackathon-outcome.html#sp_viewe r On Feb 4, 2014 11:23 AM, "Jeff Hawkins" <[email protected]> wrote: Kevin, Adding on to what Fergal said.. Here is what we did. We started with a fixed number of columns and no topology. In this situation we picked the n% most innervated columns as the output of the SP. Thus we were guaranteed to get the correct sparsity of active columns. With this setup we did a lot of other tests to verify that the SP was forming good representations. The tricky part is if you want to do this with topology and local learning rules. Is that your concern? With topology we didn't try to guarantee the entire layer of cells will have exactly n% active columns. Exact numbers are not essential. We tried several different local learning rules, all of them produced about the right level of sparsity. We than ran the other tests to make sure that local learning rules didn't cause the SP to form poor representations. You won't get exactly the same results with topology and without topology but my recollection was we were able to come up with a set of local inhibition rules that produced good results. Jeff From: nupic [mailto:[email protected]] On Behalf Of Fergal Byrne Sent: Tuesday, February 04, 2014 10:43 AM To: NuPIC general mailing list. Subject: Re: [nupic-discuss] Tests for spatial pooling Hi Kevin, If you're using no topology (and no local inhibition) then the fraction of active columns (2% in NuPIC) is the only important factor. With topology, you should choose winners relative to their neighbours. In either case, the first test is to count the fraction of active columns. In the second case, measure the average distance between active columns. Regards, Fergal Byrne On Tue, Feb 4, 2014 at 6:05 PM, Kevin Martin <[email protected]> wrote: Hi, I'm writing my own version of the CLA. I have not reached anywhere significant and I have been playing with a very small number of columns until now. I am starting by implementing a spatial pooler and spatial pooling requires a large number of columns. So far it has been easy to manually check if the synapses are distributed throughout the input, if the winning columns are really sparse etc. But it is going to be impossible when, say,I use 1000 columns. Have somebody used a test to check if the spatial pooler outputs a sparse distribution? Any suggestions on how to write a test? Thanks, Kevin Martin Jose _______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org -- Fergal Byrne, Brenter IT http://inbits.com - Better Living through Thoughtful Technology e:[email protected] t:+353 83 4214179 <tel:%2B353%2083%204214179> Formerly of Adnet [email protected] http://www.adnet.ie <http://www.adnet.ie/> _______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org _______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org _______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org -- Fergal Byrne, Brenter IT http://inbits.com - Better Living through Thoughtful Technology e:[email protected] t:+353 83 4214179 Formerly of Adnet [email protected] http://www.adnet.ie <http://www.adnet.ie/>
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
