Your rationale is a good one on the whole, but I used a very simplified setup for this experiment that does not have this property. Each column in the spatial pooler I used here can see the entire image so neighboring columns can form representations of completely different images. I believe a more biologically faithful setup would have columns that only see part of the input image so that columns in a particular region would form representations of a particular part of the input image and that would produce the property to which you are referring.
I started working with this simplified version and used it while I was developing some tools for working on visual tasks. These are some of the early results from using these tools so I have simply not yet gotten to trying out local inhibition or limiting the number of inputs to which a column can connect. In any case thanks for asking the questions. You got me thinking about some more good stuff. On Mon, Jul 7, 2014 at 2:51 PM, cogmission1 . <[email protected]> wrote: > First, let me thank you for tolerating my unlearned intrusion :P My initial > knowledge is from watching some of the videos on the CLA architecture and > configuration; and reading the CLA white paper (I'm on my second pass). > Asking these questions was both a way for me to attempt to contribute to > your tasks at hand (which I find exhilarating!) and test/verify my > understanding at the same time. > >> ...Can you share you thought process on this? > > If I understood correctly, the geometrical location of columns is closely > related to their semantic meaning. This would mean that columns carrying > similar semantic data would be more proximal to each other than those that > are not. The fuzziness between a "0" and a "2" is due to the degree of > similarity between them, therefore a larger inhibitory radius would mean > that predictions arising from more similar columns would be more inhibited > (less likely to contribute a prediction) therefore the fuzziness between "0" > and "2" would diminish? At least that is the way I reason it out at my > present level of understanding...? > > David > > > On Mon, Jul 7, 2014 at 4:05 PM, Jim Bridgewater <[email protected]> wrote: >> >> Hi David, >> >> You ask some good questions. >> >> >> On Mon, Jul 7, 2014 at 6:06 AM, David Ray <[email protected]> >> wrote: >> > I'm still in the beginning learning phase, but doesn't the inhibitory >> > radius influence the number of shared bits distinguishing one stable SDR >> > from another? >> >> This is an interesting question. Currently I'm using global >> inhibition so if I set numActiveColumnsPerInhArea = 4 then I know >> there are four active columns for each input pattern. If I used local >> inhibition and set the inhibition radius such that I ended up with 4 >> active columns per input I can see how that would affect where the >> active bits in the SDR were, but would that necessarily reduce the >> number of bits that different SDRs share? Can you share you thought >> process on this? >> >> >> > And wouldn't fewer common bits make it more likely that "0" would be >> > more easily distinguishable from a "2"? >> >> This depends on what classification scheme is being used and right now >> I'm using what I refer to as a classification hack that I wrote >> myself. It doesn't care how many common bits there are, just that >> there is at least one bit that is different. It works fine for simple >> cases, but is not tolerant to noise. I should / will move to a >> probabilistic classification scheme, but even then it's not clear to >> me that this will matter unless some noise is introduced into the >> system. >> >> >> > The trade off being more memory required to represent a given number of >> > inputs? >> >> This is a trade off I have fully embraced. It only takes 4 bits to >> represent 16 characters and my example uses anywhere from 16 to 1024 >> columns. The challenge is getting the SP to quickly find those SDRs >> with fewer shared bits. >> >> >> > Also is it not true that the inhibitory radius is inversely related to >> > the number of columns necessary to represent a finite set of inputs? >> >> This sounds correct to me. Smaller inhibition radius means more >> active columns per input pattern which means fewer unrelated input >> patterns that can be stored by a given number of columns. >> >> >> > >> > Hi btw, :-) >> > David Ray >> > >> > >> > Sent from my iPhone >> > >> >> On Jul 7, 2014, at 12:43 AM, Jim Bridgewater <[email protected]> >> >> wrote: >> >> >> >> Hi Yuwei, >> >> >> >> Thanks for the feedback, I'm glad you found it interesting. >> >> >> >> I don't know why the training for the 3 character data set takes so >> >> long, but it just occurred to me that it may have something to do with >> >> the particular characters involved. The one character data set is an >> >> image of a zero, 0. The two character data set contains images of 0 >> >> and 1 which are pretty easy to distinguish from one another. The >> >> three character data set contains images of 0,1, and 2. The 0 and >> >> the 2 are much harder to distinguish from one another, in fact when I >> >> ran Ian's spviewer demo on a hexadecimal data set (0-9, A-F), the >> >> column that represents 2 is the last one to stabilize. >> >> >> >> Just tried a three character data set consisting of O, X, and I and it >> >> only took 2 cycles to learn it so it does appear to be related to the >> >> characters involved, but I'm still not sure why the number of training >> >> cycles decreased for the data sets containing between 4 and 13 >> >> characters because the 2 and 0 are still included. >> >> >> >> By no SDR collisions I mean that two different characters cannot have >> >> the exact same SDR, but their SDRs can share bits as long as the two >> >> SDRs are different by at least one bit. >> >> >> >> I put "plot accuracy as a function of training cycles" on my ToDo >> >> list. I'll let you know when I check that one off. >> >> >> >> >> >> >> >> >> >> >> >> >> >>> On Sun, Jul 6, 2014 at 2:29 PM, Yuwei Cui <[email protected]> wrote: >> >>> Hi Jim, >> >>> >> >>> This is an interesting task. I am confused about how the amount of >> >>> required >> >>> training varies with the data set size (Fig. 3). Do you have any >> >>> intuition >> >>> on this (why it first increase and then decreases). I am also having >> >>> trouble >> >>> to understand the criterion for stop training sounds. You said there >> >>> must be >> >>> "no collisions" for images of different characters. But for SDR, a >> >>> small >> >>> number of collisions should not affect the performance much (since the >> >>> representation is distributed). It would be interesting if you could >> >>> plot >> >>> the "error rate" as a function of iteration number during learning. >> >>> >> >>> Yuwei >> >>> >> >>> >> >>>> On Sun, Jul 6, 2014 at 12:20 AM, Jim Bridgewater <[email protected]> >> >>>> wrote: >> >>>> >> >>>> Hi everyone, >> >>>> >> >>>> I wanted to make sure I know how to size the spatial pooler properly >> >>>> for a given task so I ran a few tests to measure its image >> >>>> recognition >> >>>> accuracy on data sets of different sizes. One of the surprising >> >>>> results was how the amount of training required varies with the size >> >>>> of the data set. The results are shown in the attached pdf. >> >>>> >> >>>> -- >> >>>> Jim Bridgewater, PhD >> >>>> Arizona State University >> >>>> 480-227-9592 >> >>>> >> >>>> _______________________________________________ >> >>>> nupic mailing list >> >>>> [email protected] >> >>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> >>> >> >>> >> >>> >> >>> -- >> >>> -- >> >>> Yuwei Cui >> >>> >> >>> Algorithm Internship, Numenta Inc. >> >>> >> >>> PhD Candidate, Neuroscience and Cognitive Science >> >>> >> >>> University of Maryland, College Park, MD, 20742 >> >>> >> >>> Homepage: http://terpconnect.umd.edu/~ywcui/ >> >>> >> >>> >> >>> _______________________________________________ >> >>> nupic mailing list >> >>> [email protected] >> >>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> >> >> >> >> >> >> >> -- >> >> Jim Bridgewater, PhD >> >> Arizona State University >> >> 480-227-9592 >> >> >> >> _______________________________________________ >> >> nupic mailing list >> >> [email protected] >> >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> > >> > _______________________________________________ >> > nupic mailing list >> > [email protected] >> > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> >> >> >> -- >> Jim Bridgewater, PhD >> Arizona State University >> 480-227-9592 >> >> _______________________________________________ >> nupic mailing list >> [email protected] >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > -- Jim Bridgewater, PhD Arizona State University 480-227-9592 _______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
