Fergal, the conclusion i got from inhibition was that its role is to control how many columns will be, in fact, returned as active, making sure to yield a good sparse representation. "Good" means that it will be concerned in picking columns distributed all over the input and not, for example, all of them concentrated in just one area of the input, what could yield a poor representation of the pattern.
Is this correct? How inhibition algorithm (as described in the CLA paper) influence the way each column is connect with its input bits? Doesn't it random? If we just pick the highest activation potentials globally, in a sort overlaps like method, couldn't we get not realistic representations of the patterns (independently of the topology of the input)? Regards, Christian 2014-02-05 Fergal Byrne <[email protected]>: > HI Kevin, > > Topology is the jargon we use to describe the semantic meaning (or lack of > it) of a column (or input bit) position, and for connections between them. > > For example, a typical encoding of category data randomly picks 21 bits on > out of 128. Each bit position has no real connection with the input > category it encodes. This representation has no topology at all. > > Conversely, the sliding window of on-bits in a scalar encoding has > one-dimensional topology. In this case, bit 0 might mean the input is < 10, > while bit 23 might mean an input between 11 and 33. > > An example of a 2d topology in an SDR is a retina-like sensor. In this > case the light levels or presence of particular features in a certain > spatial position would be represented by bits in the corresponding position. > > Topology in the SP relates to how each column position is connected with > its input bits. An SP which is learning to recognise features in an SDR > coming in from a retina would have columns "looking at" a set of input bits > directly "below it" in the retina. The V1 primary visual cortex has exactly > this property. > > If you have topology, you'll usually use local inhibition to pick winning > columns, where local is defined according to the topology (1-d, 2-d > distance). In the absence of topology, it's more efficient to just pick the > highest activation potentials globally. > > Regards > > Fergal Byrne > > > > > On Wed, Feb 5, 2014 at 11:25 AM, Kevin Martin <[email protected] > > wrote: > >> Hi all, >> >> >> Thanks for the help. But I'm not sure if I understand what you mean by >> topology. As far as I understand, the spatial pooler has 2 functions : >> Return a set of columns that are 1)sparse and 2)distributed. >> >> Sparsity can be maintained by making sure that the pooler returns only n% >> of columns and this criterion can be made into a test. And distribution can >> be tested by computing the average distance between winning columns. What >> is topology? Is it another way of referring to a distributed set of >> columns? I might need some help with local learning too. I'm afraid I have >> never heard the term before (its in the white paper? A quick search on the >> pdf did not see any hits for "local learning") >> >> Thanks, >> >> Kevin Martin >> >> >> On Wed, Feb 5, 2014 at 7:37 AM, Scott Purdy <[email protected]> wrote: >> >>> I would go over the properties of the SP and create tests accordingly. >>> The output, for instance, should be tolerant to spatial noise. So show an >>> instance a set of very different spatial patterns over and over a number of >>> times. Then show one of the patterns with a single bit flipped and make >>> sure the output is very close (possibly identical if columns are saturated) >>> to the same. >>> >>> There are a bunch of similar things you could check. >>> >>> Ian had a hackathon demo showing the connected bits for each column. He >>> was able to validate different aspects of the SP visually by using >>> different numbers of columns and % overlap. >>> >>> You could do something similar to make sure you see the right behavior >>> but I don't know if you could make an automated test out of it. >>> >>> >>> http://numenta.org/blog/2013/11/06/2013-fall-hackathon-outcome.html#sp_viewer >>> On Feb 4, 2014 11:23 AM, "Jeff Hawkins" <[email protected]> wrote: >>> >>>> Kevin, >>>> >>>> Adding on to what Fergal said…. >>>> >>>> >>>> >>>> Here is what we did. We started with a fixed number of columns and no >>>> topology. In this situation we picked the n% most innervated columns as >>>> the output of the SP. Thus we were *guaranteed* to get the correct >>>> sparsity of active columns. With this setup we did a lot of other tests to >>>> verify that the SP was forming good representations. >>>> >>>> >>>> >>>> The tricky part is if you want to do this with topology and local >>>> learning rules. Is that your concern? >>>> >>>> >>>> >>>> With topology we didn’t try to guarantee the entire layer of cells will >>>> have exactly n% active columns. Exact numbers are not essential. We tried >>>> several different local learning rules, all of them produced about the >>>> right level of sparsity. We than ran the other tests to make sure that >>>> local learning rules didn’t cause the SP to form poor representations. You >>>> won’t get exactly the same results with topology and without topology but >>>> my recollection was we were able to come up with a set of local inhibition >>>> rules that produced good results. >>>> >>>> Jeff >>>> >>>> >>>> >>>> *From:* nupic [mailto:[email protected]] *On Behalf Of >>>> *Fergal >>>> Byrne >>>> *Sent:* Tuesday, February 04, 2014 10:43 AM >>>> *To:* NuPIC general mailing list. >>>> *Subject:* Re: [nupic-discuss] Tests for spatial pooling >>>> >>>> >>>> >>>> Hi Kevin, >>>> >>>> >>>> >>>> If you're using no topology (and no local inhibition) then the fraction >>>> of active columns (2% in NuPIC) is the only important factor. With >>>> topology, you should choose winners relative to their neighbours. In either >>>> case, the first test is to count the fraction of active columns. In the >>>> second case, measure the average distance between active columns. >>>> >>>> >>>> >>>> Regards, >>>> >>>> >>>> >>>> Fergal Byrne >>>> >>>> >>>> >>>> On Tue, Feb 4, 2014 at 6:05 PM, Kevin Martin < >>>> [email protected]> wrote: >>>> >>>> Hi, >>>> >>>> I'm writing my own version of the CLA. I have not reached anywhere >>>> significant and I have been playing with a very small number of columns >>>> until now. I am starting by implementing a spatial pooler and spatial >>>> pooling requires a large number of columns. So far it has been easy to >>>> manually check if the synapses are distributed throughout the input, if the >>>> winning columns are really sparse etc. But it is going to be impossible >>>> when, say,I use 1000 columns. Have somebody used a test to check if the >>>> spatial pooler outputs a sparse distribution? Any suggestions on how to >>>> write a test? >>>> >>>> Thanks, >>>> >>>> Kevin Martin Jose >>>> >>>> >>>> _______________________________________________ >>>> nupic mailing list >>>> [email protected] >>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>> >>>> >>>> >>>> >>>> >>>> -- >>>> >>>> >>>> Fergal Byrne, Brenter IT >>>> >>>> >>>> >>>> http://inbits.com - Better Living through Thoughtful Technology >>>> >>>> >>>> >>>> e:[email protected] t:+353 83 4214179 >>>> >>>> Formerly of Adnet [email protected] http://www.adnet.ie >>>> >>>> _______________________________________________ >>>> nupic mailing list >>>> [email protected] >>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>> >>>> >>> _______________________________________________ >>> nupic mailing list >>> [email protected] >>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>> >>> >> >> _______________________________________________ >> nupic mailing list >> [email protected] >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> >> > > > -- > > Fergal Byrne, Brenter IT > > <http://www.examsupport.ie>http://inbits.com - Better Living through > Thoughtful Technology > > e:[email protected] t:+353 83 4214179 > Formerly of Adnet [email protected] http://www.adnet.ie > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
