In the current SP each column is initialized with potential connections to some % of all input bits. Through the learning rule, which is basically Hebbian, each column will learn represent a common spatial pattern in the input. The fewer the number of different input patterns the SP is exposed to the larger the spatial pattern each column will learn to represent. If you fed random noise to the SP the columns wouldn’t be able to learn anything. One simple way to detect how good a job the spatial pooler is doing is to see how many active synapses are active for each column. The more active synapses the bigger the spatial pattern is being recognized. Also, the more active synapses the more stable the SP will be. More active synapses are better. If you add more columns to the SP then each column will learn larger patterns.
Jeff From: nupic [mailto:[email protected]] On Behalf Of Christian Cleber Masdeval Braz Sent: Wednesday, February 05, 2014 10:27 AM To: NuPIC general mailing list. Subject: Re: [nupic-discuss] Tests for spatial pooling Fergal, the conclusion i got from inhibition was that its role is to control how many columns will be, in fact, returned as active, making sure to yield a good sparse representation. "Good" means that it will be concerned in picking columns distributed all over the input and not, for example, all of them concentrated in just one area of the input, what could yield a poor representation of the pattern. Is this correct? How inhibition algorithm (as described in the CLA paper) influence the way each column is connect with its input bits? Doesn't it random? If we just pick the highest activation potentials globally, in a sort overlaps like method, couldn't we get not realistic representations of the patterns (independently of the topology of the input)? Regards, Christian 2014-02-05 Fergal Byrne <[email protected]>: HI Kevin, Topology is the jargon we use to describe the semantic meaning (or lack of it) of a column (or input bit) position, and for connections between them. For example, a typical encoding of category data randomly picks 21 bits on out of 128. Each bit position has no real connection with the input category it encodes. This representation has no topology at all. Conversely, the sliding window of on-bits in a scalar encoding has one-dimensional topology. In this case, bit 0 might mean the input is < 10, while bit 23 might mean an input between 11 and 33. An example of a 2d topology in an SDR is a retina-like sensor. In this case the light levels or presence of particular features in a certain spatial position would be represented by bits in the corresponding position. Topology in the SP relates to how each column position is connected with its input bits. An SP which is learning to recognise features in an SDR coming in from a retina would have columns "looking at" a set of input bits directly "below it" in the retina. The V1 primary visual cortex has exactly this property. If you have topology, you'll usually use local inhibition to pick winning columns, where local is defined according to the topology (1-d, 2-d distance). In the absence of topology, it's more efficient to just pick the highest activation potentials globally. Regards Fergal Byrne On Wed, Feb 5, 2014 at 11:25 AM, Kevin Martin <[email protected]> wrote: Hi all, Thanks for the help. But I'm not sure if I understand what you mean by topology. As far as I understand, the spatial pooler has 2 functions : Return a set of columns that are 1)sparse and 2)distributed. Sparsity can be maintained by making sure that the pooler returns only n% of columns and this criterion can be made into a test. And distribution can be tested by computing the average distance between winning columns. What is topology? Is it another way of referring to a distributed set of columns? I might need some help with local learning too. I'm afraid I have never heard the term before (its in the white paper? A quick search on the pdf did not see any hits for "local learning") Thanks, Kevin Martin On Wed, Feb 5, 2014 at 7:37 AM, Scott Purdy <[email protected]> wrote: I would go over the properties of the SP and create tests accordingly. The output, for instance, should be tolerant to spatial noise. So show an instance a set of very different spatial patterns over and over a number of times. Then show one of the patterns with a single bit flipped and make sure the output is very close (possibly identical if columns are saturated) to the same. There are a bunch of similar things you could check. Ian had a hackathon demo showing the connected bits for each column. He was able to validate different aspects of the SP visually by using different numbers of columns and % overlap. You could do something similar to make sure you see the right behavior but I don't know if you could make an automated test out of it. http://numenta.org/blog/2013/11/06/2013-fall-hackathon-outcome.html#sp_viewer On Feb 4, 2014 11:23 AM, "Jeff Hawkins" <[email protected]> wrote: Kevin, Adding on to what Fergal said…. Here is what we did. We started with a fixed number of columns and no topology. In this situation we picked the n% most innervated columns as the output of the SP. Thus we were guaranteed to get the correct sparsity of active columns. With this setup we did a lot of other tests to verify that the SP was forming good representations. The tricky part is if you want to do this with topology and local learning rules. Is that your concern? With topology we didn’t try to guarantee the entire layer of cells will have exactly n% active columns. Exact numbers are not essential. We tried several different local learning rules, all of them produced about the right level of sparsity. We than ran the other tests to make sure that local learning rules didn’t cause the SP to form poor representations. You won’t get exactly the same results with topology and without topology but my recollection was we were able to come up with a set of local inhibition rules that produced good results. Jeff From: nupic [mailto:[email protected]] On Behalf Of Fergal Byrne Sent: Tuesday, February 04, 2014 10:43 AM To: NuPIC general mailing list. Subject: Re: [nupic-discuss] Tests for spatial pooling Hi Kevin, If you're using no topology (and no local inhibition) then the fraction of active columns (2% in NuPIC) is the only important factor. With topology, you should choose winners relative to their neighbours. In either case, the first test is to count the fraction of active columns. In the second case, measure the average distance between active columns. Regards, Fergal Byrne On Tue, Feb 4, 2014 at 6:05 PM, Kevin Martin <[email protected]> wrote: Hi, I'm writing my own version of the CLA. I have not reached anywhere significant and I have been playing with a very small number of columns until now. I am starting by implementing a spatial pooler and spatial pooling requires a large number of columns. So far it has been easy to manually check if the synapses are distributed throughout the input, if the winning columns are really sparse etc. But it is going to be impossible when, say,I use 1000 columns. Have somebody used a test to check if the spatial pooler outputs a sparse distribution? Any suggestions on how to write a test? Thanks, Kevin Martin Jose _______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org -- Fergal Byrne, Brenter IT http://inbits.com - Better Living through Thoughtful Technology e:[email protected] t:+353 83 4214179 <tel:%2B353%2083%204214179> Formerly of Adnet [email protected] http://www.adnet.ie <http://www.adnet.ie/> _______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org _______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org _______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org -- Fergal Byrne, Brenter IT http://inbits.com - Better Living through Thoughtful Technology e:[email protected] t:+353 83 4214179 <tel:%2B353%2083%204214179> Formerly of Adnet [email protected] http://www.adnet.ie <http://www.adnet.ie/> _______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
