Hi Subutai, maxBoost was actually at 1, but I failed to update the code listing in the document properly. I ran it with maxBoost = 3 first, by mistake and those results were not as good.
On Wed, Jul 9, 2014 at 8:08 AM, Subutai Ahmad <[email protected]> wrote: > > Thanks for running the experiment! Without looking at the script in detail > I'm not sure what is going on. (BTW, looks like maxBoost was still at 3, not > 1. Not sure if that will affect anything though.) > > My hunch is still that the SP should be able to perform perfectly on this > task. > > --Subutai > > > On Wed, Jul 9, 2014 at 12:20 AM, Jim Bridgewater <[email protected]> wrote: >> >> Subutai, >> >> Here's the first go at it, the accuracies are still less than 100% for >> most of the data sets, but many of them hit the 30 cycle limit. I'll >> run it again with a higher limit. >> >> On Tue, Jul 8, 2014 at 9:13 PM, Subutai Ahmad <[email protected]> wrote: >> > >> > Thanks, Jim. Yes, as long as the inputs are pretty distinct, I think >> > learning the training set should work perfectly for much larger data >> > sets. >> > In theory the limit here should be extremely high (well over a billion >> > patterns). >> > >> > I'm also a bit unsure about how you are classifying each letter during >> > testing. I am assuming you are essentially finding the letter whose SP >> > output has the most overlap with the actual current SP output? >> > >> > As a control experiment, could you try the following parameter changes >> > with >> > you 1024:64 SP and see if it has any effect? >> > >> > potentialPct = 0.8, >> > synPermInactiveDec = 0.001 >> > synPermActiveInc = 0.001 >> > maxBoost = 1.0 >> > >> > The above would have a potential pool less than 100%, slow learning and >> > no >> > boosting. >> > >> > Thanks, >> > >> > --Subutai >> > >> > >> > >> > On Tue, Jul 8, 2014 at 8:34 PM, Jim Bridgewater <[email protected]> >> > wrote: >> >> >> >> Hi Subutai, >> >> >> >> The training and testing data sets are the same. I tested 4 different >> >> configurations and all configurations achieve perfect results for the >> >> data sets that contain between 1 and 13 characters. Two >> >> configurations get perfect results for the data set with 14 characters >> >> and one achieves it for 15 and 16 characters. Are you saying you >> >> think it should be perfect for larger data sets? >> >> >> >> >> >> >> >> On Tue, Jul 8, 2014 at 9:52 AM, Subutai Ahmad <[email protected]> >> >> wrote: >> >> > Hi Jim, >> >> > >> >> > Thanks for doing this experiment. I have one quick question: are the >> >> > training and test sets the same? If so, I think the SP should >> >> > perform >> >> > perfectly in this task. >> >> > >> >> > Thanks, >> >> > >> >> > --Subutai >> >> > >> >> > >> >> > On Sun, Jul 6, 2014 at 10:43 PM, Jim Bridgewater <[email protected]> >> >> > wrote: >> >> >> >> >> >> Hi Yuwei, >> >> >> >> >> >> Thanks for the feedback, I'm glad you found it interesting. >> >> >> >> >> >> I don't know why the training for the 3 character data set takes so >> >> >> long, but it just occurred to me that it may have something to do >> >> >> with >> >> >> the particular characters involved. The one character data set is >> >> >> an >> >> >> image of a zero, 0. The two character data set contains images of 0 >> >> >> and 1 which are pretty easy to distinguish from one another. The >> >> >> three character data set contains images of 0,1, and 2. The 0 and >> >> >> the 2 are much harder to distinguish from one another, in fact when >> >> >> I >> >> >> ran Ian's spviewer demo on a hexadecimal data set (0-9, A-F), the >> >> >> column that represents 2 is the last one to stabilize. >> >> >> >> >> >> Just tried a three character data set consisting of O, X, and I and >> >> >> it >> >> >> only took 2 cycles to learn it so it does appear to be related to >> >> >> the >> >> >> characters involved, but I'm still not sure why the number of >> >> >> training >> >> >> cycles decreased for the data sets containing between 4 and 13 >> >> >> characters because the 2 and 0 are still included. >> >> >> >> >> >> By no SDR collisions I mean that two different characters cannot >> >> >> have >> >> >> the exact same SDR, but their SDRs can share bits as long as the two >> >> >> SDRs are different by at least one bit. >> >> >> >> >> >> I put "plot accuracy as a function of training cycles" on my ToDo >> >> >> list. I'll let you know when I check that one off. >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> On Sun, Jul 6, 2014 at 2:29 PM, Yuwei Cui <[email protected]> >> >> >> wrote: >> >> >> > Hi Jim, >> >> >> > >> >> >> > This is an interesting task. I am confused about how the amount of >> >> >> > required >> >> >> > training varies with the data set size (Fig. 3). Do you have any >> >> >> > intuition >> >> >> > on this (why it first increase and then decreases). I am also >> >> >> > having >> >> >> > trouble >> >> >> > to understand the criterion for stop training sounds. You said >> >> >> > there >> >> >> > must be >> >> >> > "no collisions" for images of different characters. But for SDR, >> >> >> > a >> >> >> > small >> >> >> > number of collisions should not affect the performance much (since >> >> >> > the >> >> >> > representation is distributed). It would be interesting if you >> >> >> > could >> >> >> > plot >> >> >> > the "error rate" as a function of iteration number during >> >> >> > learning. >> >> >> > >> >> >> > Yuwei >> >> >> > >> >> >> > >> >> >> > On Sun, Jul 6, 2014 at 12:20 AM, Jim Bridgewater >> >> >> > <[email protected]> >> >> >> > wrote: >> >> >> >> >> >> >> >> Hi everyone, >> >> >> >> >> >> >> >> I wanted to make sure I know how to size the spatial pooler >> >> >> >> properly >> >> >> >> for a given task so I ran a few tests to measure its image >> >> >> >> recognition >> >> >> >> accuracy on data sets of different sizes. One of the surprising >> >> >> >> results was how the amount of training required varies with the >> >> >> >> size >> >> >> >> of the data set. The results are shown in the attached pdf. >> >> >> >> >> >> >> >> -- >> >> >> >> Jim Bridgewater, PhD >> >> >> >> Arizona State University >> >> >> >> 480-227-9592 >> >> >> >> >> >> >> >> _______________________________________________ >> >> >> >> nupic mailing list >> >> >> >> [email protected] >> >> >> >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> >> >> >> >> >> >> > >> >> >> > >> >> >> > >> >> >> > -- >> >> >> > -- >> >> >> > Yuwei Cui >> >> >> > >> >> >> > Algorithm Internship, Numenta Inc. >> >> >> > >> >> >> > PhD Candidate, Neuroscience and Cognitive Science >> >> >> > >> >> >> > University of Maryland, College Park, MD, 20742 >> >> >> > >> >> >> > Homepage: http://terpconnect.umd.edu/~ywcui/ >> >> >> > >> >> >> > >> >> >> > _______________________________________________ >> >> >> > nupic mailing list >> >> >> > [email protected] >> >> >> > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> >> >> > >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> Jim Bridgewater, PhD >> >> >> Arizona State University >> >> >> 480-227-9592 >> >> >> >> >> >> _______________________________________________ >> >> >> nupic mailing list >> >> >> [email protected] >> >> >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> >> > >> >> > >> >> > >> >> > _______________________________________________ >> >> > nupic mailing list >> >> > [email protected] >> >> > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> >> > >> >> >> >> >> >> >> >> -- >> >> Jim Bridgewater, PhD >> >> Arizona State University >> >> 480-227-9592 >> >> >> >> _______________________________________________ >> >> nupic mailing list >> >> [email protected] >> >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> > >> > >> > >> > _______________________________________________ >> > nupic mailing list >> > [email protected] >> > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> > >> >> >> >> -- >> Jim Bridgewater, PhD >> Arizona State University >> 480-227-9592 >> >> _______________________________________________ >> nupic mailing list >> [email protected] >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> > > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > -- Jim Bridgewater, PhD Arizona State University 480-227-9592 _______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
