No problem. Thank you for taking the time to answer my questions and give me feedback on my understanding so far.
Cheers! David On Mon, Jul 7, 2014 at 6:11 PM, Jim Bridgewater <[email protected]> wrote: > Your rationale is a good one on the whole, but I used a very > simplified setup for this experiment that does not have this property. > Each column in the spatial pooler I used here can see the entire image > so neighboring columns can form representations of completely > different images. I believe a more biologically faithful setup would > have columns that only see part of the input image so that columns in > a particular region would form representations of a particular part of > the input image and that would produce the property to which you are > referring. > > I started working with this simplified version and used it while I was > developing some tools for working on visual tasks. These are some of > the early results from using these tools so I have simply not yet > gotten to trying out local inhibition or limiting the number of inputs > to which a column can connect. > > In any case thanks for asking the questions. You got me thinking > about some more good stuff. > > > On Mon, Jul 7, 2014 at 2:51 PM, cogmission1 . > <[email protected]> wrote: > > First, let me thank you for tolerating my unlearned intrusion :P My > initial > > knowledge is from watching some of the videos on the CLA architecture and > > configuration; and reading the CLA white paper (I'm on my second pass). > > Asking these questions was both a way for me to attempt to contribute to > > your tasks at hand (which I find exhilarating!) and test/verify my > > understanding at the same time. > > > >> ...Can you share you thought process on this? > > > > If I understood correctly, the geometrical location of columns is closely > > related to their semantic meaning. This would mean that columns carrying > > similar semantic data would be more proximal to each other than those > that > > are not. The fuzziness between a "0" and a "2" is due to the degree of > > similarity between them, therefore a larger inhibitory radius would mean > > that predictions arising from more similar columns would be more > inhibited > > (less likely to contribute a prediction) therefore the fuzziness between > "0" > > and "2" would diminish? At least that is the way I reason it out at my > > present level of understanding...? > > > > David > > > > > > On Mon, Jul 7, 2014 at 4:05 PM, Jim Bridgewater <[email protected]> > wrote: > >> > >> Hi David, > >> > >> You ask some good questions. > >> > >> > >> On Mon, Jul 7, 2014 at 6:06 AM, David Ray <[email protected]> > >> wrote: > >> > I'm still in the beginning learning phase, but doesn't the inhibitory > >> > radius influence the number of shared bits distinguishing one stable > SDR > >> > from another? > >> > >> This is an interesting question. Currently I'm using global > >> inhibition so if I set numActiveColumnsPerInhArea = 4 then I know > >> there are four active columns for each input pattern. If I used local > >> inhibition and set the inhibition radius such that I ended up with 4 > >> active columns per input I can see how that would affect where the > >> active bits in the SDR were, but would that necessarily reduce the > >> number of bits that different SDRs share? Can you share you thought > >> process on this? > >> > >> > >> > And wouldn't fewer common bits make it more likely that "0" would be > >> > more easily distinguishable from a "2"? > >> > >> This depends on what classification scheme is being used and right now > >> I'm using what I refer to as a classification hack that I wrote > >> myself. It doesn't care how many common bits there are, just that > >> there is at least one bit that is different. It works fine for simple > >> cases, but is not tolerant to noise. I should / will move to a > >> probabilistic classification scheme, but even then it's not clear to > >> me that this will matter unless some noise is introduced into the > >> system. > >> > >> > >> > The trade off being more memory required to represent a given number > of > >> > inputs? > >> > >> This is a trade off I have fully embraced. It only takes 4 bits to > >> represent 16 characters and my example uses anywhere from 16 to 1024 > >> columns. The challenge is getting the SP to quickly find those SDRs > >> with fewer shared bits. > >> > >> > >> > Also is it not true that the inhibitory radius is inversely related to > >> > the number of columns necessary to represent a finite set of inputs? > >> > >> This sounds correct to me. Smaller inhibition radius means more > >> active columns per input pattern which means fewer unrelated input > >> patterns that can be stored by a given number of columns. > >> > >> > >> > > >> > Hi btw, :-) > >> > David Ray > >> > > >> > > >> > Sent from my iPhone > >> > > >> >> On Jul 7, 2014, at 12:43 AM, Jim Bridgewater <[email protected]> > >> >> wrote: > >> >> > >> >> Hi Yuwei, > >> >> > >> >> Thanks for the feedback, I'm glad you found it interesting. > >> >> > >> >> I don't know why the training for the 3 character data set takes so > >> >> long, but it just occurred to me that it may have something to do > with > >> >> the particular characters involved. The one character data set is an > >> >> image of a zero, 0. The two character data set contains images of 0 > >> >> and 1 which are pretty easy to distinguish from one another. The > >> >> three character data set contains images of 0,1, and 2. The 0 and > >> >> the 2 are much harder to distinguish from one another, in fact when I > >> >> ran Ian's spviewer demo on a hexadecimal data set (0-9, A-F), the > >> >> column that represents 2 is the last one to stabilize. > >> >> > >> >> Just tried a three character data set consisting of O, X, and I and > it > >> >> only took 2 cycles to learn it so it does appear to be related to the > >> >> characters involved, but I'm still not sure why the number of > training > >> >> cycles decreased for the data sets containing between 4 and 13 > >> >> characters because the 2 and 0 are still included. > >> >> > >> >> By no SDR collisions I mean that two different characters cannot have > >> >> the exact same SDR, but their SDRs can share bits as long as the two > >> >> SDRs are different by at least one bit. > >> >> > >> >> I put "plot accuracy as a function of training cycles" on my ToDo > >> >> list. I'll let you know when I check that one off. > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >>> On Sun, Jul 6, 2014 at 2:29 PM, Yuwei Cui <[email protected]> > wrote: > >> >>> Hi Jim, > >> >>> > >> >>> This is an interesting task. I am confused about how the amount of > >> >>> required > >> >>> training varies with the data set size (Fig. 3). Do you have any > >> >>> intuition > >> >>> on this (why it first increase and then decreases). I am also having > >> >>> trouble > >> >>> to understand the criterion for stop training sounds. You said there > >> >>> must be > >> >>> "no collisions" for images of different characters. But for SDR, a > >> >>> small > >> >>> number of collisions should not affect the performance much (since > the > >> >>> representation is distributed). It would be interesting if you could > >> >>> plot > >> >>> the "error rate" as a function of iteration number during learning. > >> >>> > >> >>> Yuwei > >> >>> > >> >>> > >> >>>> On Sun, Jul 6, 2014 at 12:20 AM, Jim Bridgewater < > [email protected]> > >> >>>> wrote: > >> >>>> > >> >>>> Hi everyone, > >> >>>> > >> >>>> I wanted to make sure I know how to size the spatial pooler > properly > >> >>>> for a given task so I ran a few tests to measure its image > >> >>>> recognition > >> >>>> accuracy on data sets of different sizes. One of the surprising > >> >>>> results was how the amount of training required varies with the > size > >> >>>> of the data set. The results are shown in the attached pdf. > >> >>>> > >> >>>> -- > >> >>>> Jim Bridgewater, PhD > >> >>>> Arizona State University > >> >>>> 480-227-9592 > >> >>>> > >> >>>> _______________________________________________ > >> >>>> nupic mailing list > >> >>>> [email protected] > >> >>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >> >>> > >> >>> > >> >>> > >> >>> -- > >> >>> -- > >> >>> Yuwei Cui > >> >>> > >> >>> Algorithm Internship, Numenta Inc. > >> >>> > >> >>> PhD Candidate, Neuroscience and Cognitive Science > >> >>> > >> >>> University of Maryland, College Park, MD, 20742 > >> >>> > >> >>> Homepage: http://terpconnect.umd.edu/~ywcui/ > >> >>> > >> >>> > >> >>> _______________________________________________ > >> >>> nupic mailing list > >> >>> [email protected] > >> >>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >> >> > >> >> > >> >> > >> >> -- > >> >> Jim Bridgewater, PhD > >> >> Arizona State University > >> >> 480-227-9592 > >> >> > >> >> _______________________________________________ > >> >> nupic mailing list > >> >> [email protected] > >> >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >> > > >> > _______________________________________________ > >> > nupic mailing list > >> > [email protected] > >> > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >> > >> > >> > >> -- > >> Jim Bridgewater, PhD > >> Arizona State University > >> 480-227-9592 > >> > >> _______________________________________________ > >> nupic mailing list > >> [email protected] > >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > > > > > > > _______________________________________________ > > nupic mailing list > > [email protected] > > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > > > > > -- > Jim Bridgewater, PhD > Arizona State University > 480-227-9592 > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
