Hi David, You ask some good questions.
On Mon, Jul 7, 2014 at 6:06 AM, David Ray <[email protected]> wrote: > I'm still in the beginning learning phase, but doesn't the inhibitory radius > influence the number of shared bits distinguishing one stable SDR from > another? This is an interesting question. Currently I'm using global inhibition so if I set numActiveColumnsPerInhArea = 4 then I know there are four active columns for each input pattern. If I used local inhibition and set the inhibition radius such that I ended up with 4 active columns per input I can see how that would affect where the active bits in the SDR were, but would that necessarily reduce the number of bits that different SDRs share? Can you share you thought process on this? > And wouldn't fewer common bits make it more likely that "0" would be more > easily distinguishable from a "2"? This depends on what classification scheme is being used and right now I'm using what I refer to as a classification hack that I wrote myself. It doesn't care how many common bits there are, just that there is at least one bit that is different. It works fine for simple cases, but is not tolerant to noise. I should / will move to a probabilistic classification scheme, but even then it's not clear to me that this will matter unless some noise is introduced into the system. > The trade off being more memory required to represent a given number of > inputs? This is a trade off I have fully embraced. It only takes 4 bits to represent 16 characters and my example uses anywhere from 16 to 1024 columns. The challenge is getting the SP to quickly find those SDRs with fewer shared bits. > Also is it not true that the inhibitory radius is inversely related to the > number of columns necessary to represent a finite set of inputs? This sounds correct to me. Smaller inhibition radius means more active columns per input pattern which means fewer unrelated input patterns that can be stored by a given number of columns. > > Hi btw, :-) > David Ray > > > Sent from my iPhone > >> On Jul 7, 2014, at 12:43 AM, Jim Bridgewater <[email protected]> wrote: >> >> Hi Yuwei, >> >> Thanks for the feedback, I'm glad you found it interesting. >> >> I don't know why the training for the 3 character data set takes so >> long, but it just occurred to me that it may have something to do with >> the particular characters involved. The one character data set is an >> image of a zero, 0. The two character data set contains images of 0 >> and 1 which are pretty easy to distinguish from one another. The >> three character data set contains images of 0,1, and 2. The 0 and >> the 2 are much harder to distinguish from one another, in fact when I >> ran Ian's spviewer demo on a hexadecimal data set (0-9, A-F), the >> column that represents 2 is the last one to stabilize. >> >> Just tried a three character data set consisting of O, X, and I and it >> only took 2 cycles to learn it so it does appear to be related to the >> characters involved, but I'm still not sure why the number of training >> cycles decreased for the data sets containing between 4 and 13 >> characters because the 2 and 0 are still included. >> >> By no SDR collisions I mean that two different characters cannot have >> the exact same SDR, but their SDRs can share bits as long as the two >> SDRs are different by at least one bit. >> >> I put "plot accuracy as a function of training cycles" on my ToDo >> list. I'll let you know when I check that one off. >> >> >> >> >> >> >>> On Sun, Jul 6, 2014 at 2:29 PM, Yuwei Cui <[email protected]> wrote: >>> Hi Jim, >>> >>> This is an interesting task. I am confused about how the amount of required >>> training varies with the data set size (Fig. 3). Do you have any intuition >>> on this (why it first increase and then decreases). I am also having trouble >>> to understand the criterion for stop training sounds. You said there must be >>> "no collisions" for images of different characters. But for SDR, a small >>> number of collisions should not affect the performance much (since the >>> representation is distributed). It would be interesting if you could plot >>> the "error rate" as a function of iteration number during learning. >>> >>> Yuwei >>> >>> >>>> On Sun, Jul 6, 2014 at 12:20 AM, Jim Bridgewater <[email protected]> >>>> wrote: >>>> >>>> Hi everyone, >>>> >>>> I wanted to make sure I know how to size the spatial pooler properly >>>> for a given task so I ran a few tests to measure its image recognition >>>> accuracy on data sets of different sizes. One of the surprising >>>> results was how the amount of training required varies with the size >>>> of the data set. The results are shown in the attached pdf. >>>> >>>> -- >>>> Jim Bridgewater, PhD >>>> Arizona State University >>>> 480-227-9592 >>>> >>>> _______________________________________________ >>>> nupic mailing list >>>> [email protected] >>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>> >>> >>> >>> -- >>> -- >>> Yuwei Cui >>> >>> Algorithm Internship, Numenta Inc. >>> >>> PhD Candidate, Neuroscience and Cognitive Science >>> >>> University of Maryland, College Park, MD, 20742 >>> >>> Homepage: http://terpconnect.umd.edu/~ywcui/ >>> >>> >>> _______________________________________________ >>> nupic mailing list >>> [email protected] >>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> >> >> >> -- >> Jim Bridgewater, PhD >> Arizona State University >> 480-227-9592 >> >> _______________________________________________ >> nupic mailing list >> [email protected] >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org -- Jim Bridgewater, PhD Arizona State University 480-227-9592 _______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
