Great, good news! --Subutai
On Wed, Jul 9, 2014 at 9:52 PM, Jim Bridgewater <[email protected]> wrote: > Subutai, > > The accuracies are 100% for all data sets using the KNN classifier. > Pretty cool! > > I also tried testing on the same images it was trained on, but > reversing the order of the images. Those accuracies are less than > 100%, but maybe just because the SP really hasn't been trained much. > One training cycle was enough for all data sets to hit 100% accuracy > when the order is not reversed. > > > > On Wed, Jul 9, 2014 at 11:39 AM, Subutai Ahmad <[email protected]> > wrote: > > > > Thanks, Jim. That looks like good progress! The area under the curve is > > much better now. I'll be curious as to what you get with the KNN. > > > > --Subutai > > > > > > On Wed, Jul 9, 2014 at 11:33 AM, Jim Bridgewater <[email protected]> > wrote: > >> > >> Subutai, > >> > >> Here's an updated document that shows results for maxBoost = 3 and > >> maxBoost = 1. The two cases are only different for data sets that > >> contain between 7 and 17 characters. Their accuracies are the same > >> for all other data sets. This document also shows results where I > >> increased the maximum number of training cycles to 100 which improved > >> the accuracy quite a bit. I'm running another test right now that > >> runs 100 cycles on all data sets rather than determining the # of > >> cycles by looking for SDR collisions. > >> > >> My classifier is even simpler / cruder than you suppose, it looks for > >> an exact match between training and testing SDRs. This likely reduces > >> the accuracy so I'm working on switching to the KNN classifier. > >> > >> On Wed, Jul 9, 2014 at 11:13 AM, Jim Bridgewater <[email protected]> > >> wrote: > >> > Hi Subutai, > >> > > >> > maxBoost was actually at 1, but I failed to update the code listing in > >> > the document properly. I ran it with maxBoost = 3 first, by mistake > >> > and those results were not as good. > >> > > >> > > >> > > >> > On Wed, Jul 9, 2014 at 8:08 AM, Subutai Ahmad <[email protected]> > >> > wrote: > >> >> > >> >> Thanks for running the experiment! Without looking at the script in > >> >> detail > >> >> I'm not sure what is going on. (BTW, looks like maxBoost was still at > >> >> 3, not > >> >> 1. Not sure if that will affect anything though.) > >> >> > >> >> My hunch is still that the SP should be able to perform perfectly on > >> >> this > >> >> task. > >> >> > >> >> --Subutai > >> >> > >> >> > >> >> On Wed, Jul 9, 2014 at 12:20 AM, Jim Bridgewater <[email protected] > > > >> >> wrote: > >> >>> > >> >>> Subutai, > >> >>> > >> >>> Here's the first go at it, the accuracies are still less than 100% > for > >> >>> most of the data sets, but many of them hit the 30 cycle limit. > I'll > >> >>> run it again with a higher limit. > >> >>> > >> >>> On Tue, Jul 8, 2014 at 9:13 PM, Subutai Ahmad <[email protected]> > >> >>> wrote: > >> >>> > > >> >>> > Thanks, Jim. Yes, as long as the inputs are pretty distinct, I > think > >> >>> > learning the training set should work perfectly for much larger > data > >> >>> > sets. > >> >>> > In theory the limit here should be extremely high (well over a > >> >>> > billion > >> >>> > patterns). > >> >>> > > >> >>> > I'm also a bit unsure about how you are classifying each letter > >> >>> > during > >> >>> > testing. I am assuming you are essentially finding the letter > whose > >> >>> > SP > >> >>> > output has the most overlap with the actual current SP output? > >> >>> > > >> >>> > As a control experiment, could you try the following parameter > >> >>> > changes > >> >>> > with > >> >>> > you 1024:64 SP and see if it has any effect? > >> >>> > > >> >>> > potentialPct = 0.8, > >> >>> > synPermInactiveDec = 0.001 > >> >>> > synPermActiveInc = 0.001 > >> >>> > maxBoost = 1.0 > >> >>> > > >> >>> > The above would have a potential pool less than 100%, slow > learning > >> >>> > and > >> >>> > no > >> >>> > boosting. > >> >>> > > >> >>> > Thanks, > >> >>> > > >> >>> > --Subutai > >> >>> > > >> >>> > > >> >>> > > >> >>> > On Tue, Jul 8, 2014 at 8:34 PM, Jim Bridgewater < > [email protected]> > >> >>> > wrote: > >> >>> >> > >> >>> >> Hi Subutai, > >> >>> >> > >> >>> >> The training and testing data sets are the same. I tested 4 > >> >>> >> different > >> >>> >> configurations and all configurations achieve perfect results for > >> >>> >> the > >> >>> >> data sets that contain between 1 and 13 characters. Two > >> >>> >> configurations get perfect results for the data set with 14 > >> >>> >> characters > >> >>> >> and one achieves it for 15 and 16 characters. Are you saying you > >> >>> >> think it should be perfect for larger data sets? > >> >>> >> > >> >>> >> > >> >>> >> > >> >>> >> On Tue, Jul 8, 2014 at 9:52 AM, Subutai Ahmad < > [email protected]> > >> >>> >> wrote: > >> >>> >> > Hi Jim, > >> >>> >> > > >> >>> >> > Thanks for doing this experiment. I have one quick question: > are > >> >>> >> > the > >> >>> >> > training and test sets the same? If so, I think the SP should > >> >>> >> > perform > >> >>> >> > perfectly in this task. > >> >>> >> > > >> >>> >> > Thanks, > >> >>> >> > > >> >>> >> > --Subutai > >> >>> >> > > >> >>> >> > > >> >>> >> > On Sun, Jul 6, 2014 at 10:43 PM, Jim Bridgewater > >> >>> >> > <[email protected]> > >> >>> >> > wrote: > >> >>> >> >> > >> >>> >> >> Hi Yuwei, > >> >>> >> >> > >> >>> >> >> Thanks for the feedback, I'm glad you found it interesting. > >> >>> >> >> > >> >>> >> >> I don't know why the training for the 3 character data set > takes > >> >>> >> >> so > >> >>> >> >> long, but it just occurred to me that it may have something to > >> >>> >> >> do > >> >>> >> >> with > >> >>> >> >> the particular characters involved. The one character data > set > >> >>> >> >> is > >> >>> >> >> an > >> >>> >> >> image of a zero, 0. The two character data set contains > images > >> >>> >> >> of 0 > >> >>> >> >> and 1 which are pretty easy to distinguish from one another. > >> >>> >> >> The > >> >>> >> >> three character data set contains images of 0,1, and 2. The > 0 > >> >>> >> >> and > >> >>> >> >> the 2 are much harder to distinguish from one another, in fact > >> >>> >> >> when > >> >>> >> >> I > >> >>> >> >> ran Ian's spviewer demo on a hexadecimal data set (0-9, A-F), > >> >>> >> >> the > >> >>> >> >> column that represents 2 is the last one to stabilize. > >> >>> >> >> > >> >>> >> >> Just tried a three character data set consisting of O, X, and > I > >> >>> >> >> and > >> >>> >> >> it > >> >>> >> >> only took 2 cycles to learn it so it does appear to be related > >> >>> >> >> to > >> >>> >> >> the > >> >>> >> >> characters involved, but I'm still not sure why the number of > >> >>> >> >> training > >> >>> >> >> cycles decreased for the data sets containing between 4 and 13 > >> >>> >> >> characters because the 2 and 0 are still included. > >> >>> >> >> > >> >>> >> >> By no SDR collisions I mean that two different characters > cannot > >> >>> >> >> have > >> >>> >> >> the exact same SDR, but their SDRs can share bits as long as > the > >> >>> >> >> two > >> >>> >> >> SDRs are different by at least one bit. > >> >>> >> >> > >> >>> >> >> I put "plot accuracy as a function of training cycles" on my > >> >>> >> >> ToDo > >> >>> >> >> list. I'll let you know when I check that one off. > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> On Sun, Jul 6, 2014 at 2:29 PM, Yuwei Cui < > [email protected]> > >> >>> >> >> wrote: > >> >>> >> >> > Hi Jim, > >> >>> >> >> > > >> >>> >> >> > This is an interesting task. I am confused about how the > >> >>> >> >> > amount of > >> >>> >> >> > required > >> >>> >> >> > training varies with the data set size (Fig. 3). Do you have > >> >>> >> >> > any > >> >>> >> >> > intuition > >> >>> >> >> > on this (why it first increase and then decreases). I am > also > >> >>> >> >> > having > >> >>> >> >> > trouble > >> >>> >> >> > to understand the criterion for stop training sounds. You > said > >> >>> >> >> > there > >> >>> >> >> > must be > >> >>> >> >> > "no collisions" for images of different characters. But for > >> >>> >> >> > SDR, > >> >>> >> >> > a > >> >>> >> >> > small > >> >>> >> >> > number of collisions should not affect the performance much > >> >>> >> >> > (since > >> >>> >> >> > the > >> >>> >> >> > representation is distributed). It would be interesting if > you > >> >>> >> >> > could > >> >>> >> >> > plot > >> >>> >> >> > the "error rate" as a function of iteration number during > >> >>> >> >> > learning. > >> >>> >> >> > > >> >>> >> >> > Yuwei > >> >>> >> >> > > >> >>> >> >> > > >> >>> >> >> > On Sun, Jul 6, 2014 at 12:20 AM, Jim Bridgewater > >> >>> >> >> > <[email protected]> > >> >>> >> >> > wrote: > >> >>> >> >> >> > >> >>> >> >> >> Hi everyone, > >> >>> >> >> >> > >> >>> >> >> >> I wanted to make sure I know how to size the spatial pooler > >> >>> >> >> >> properly > >> >>> >> >> >> for a given task so I ran a few tests to measure its image > >> >>> >> >> >> recognition > >> >>> >> >> >> accuracy on data sets of different sizes. One of the > >> >>> >> >> >> surprising > >> >>> >> >> >> results was how the amount of training required varies with > >> >>> >> >> >> the > >> >>> >> >> >> size > >> >>> >> >> >> of the data set. The results are shown in the attached > pdf. > >> >>> >> >> >> > >> >>> >> >> >> -- > >> >>> >> >> >> Jim Bridgewater, PhD > >> >>> >> >> >> Arizona State University > >> >>> >> >> >> 480-227-9592 > >> >>> >> >> >> > >> >>> >> >> >> _______________________________________________ > >> >>> >> >> >> nupic mailing list > >> >>> >> >> >> [email protected] > >> >>> >> >> >> > >> >>> >> >> >> > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >> >>> >> >> >> > >> >>> >> >> > > >> >>> >> >> > > >> >>> >> >> > > >> >>> >> >> > -- > >> >>> >> >> > -- > >> >>> >> >> > Yuwei Cui > >> >>> >> >> > > >> >>> >> >> > Algorithm Internship, Numenta Inc. > >> >>> >> >> > > >> >>> >> >> > PhD Candidate, Neuroscience and Cognitive Science > >> >>> >> >> > > >> >>> >> >> > University of Maryland, College Park, MD, 20742 > >> >>> >> >> > > >> >>> >> >> > Homepage: http://terpconnect.umd.edu/~ywcui/ > >> >>> >> >> > > >> >>> >> >> > > >> >>> >> >> > _______________________________________________ > >> >>> >> >> > nupic mailing list > >> >>> >> >> > [email protected] > >> >>> >> >> > > >> >>> >> >> > > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >> >>> >> >> > > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> -- > >> >>> >> >> Jim Bridgewater, PhD > >> >>> >> >> Arizona State University > >> >>> >> >> 480-227-9592 > >> >>> >> >> > >> >>> >> >> _______________________________________________ > >> >>> >> >> nupic mailing list > >> >>> >> >> [email protected] > >> >>> >> >> > >> >>> >> >> > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >> >>> >> > > >> >>> >> > > >> >>> >> > > >> >>> >> > _______________________________________________ > >> >>> >> > nupic mailing list > >> >>> >> > [email protected] > >> >>> >> > > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >> >>> >> > > >> >>> >> > >> >>> >> > >> >>> >> > >> >>> >> -- > >> >>> >> Jim Bridgewater, PhD > >> >>> >> Arizona State University > >> >>> >> 480-227-9592 > >> >>> >> > >> >>> >> _______________________________________________ > >> >>> >> nupic mailing list > >> >>> >> [email protected] > >> >>> >> > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >> >>> > > >> >>> > > >> >>> > > >> >>> > _______________________________________________ > >> >>> > nupic mailing list > >> >>> > [email protected] > >> >>> > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >> >>> > > >> >>> > >> >>> > >> >>> > >> >>> -- > >> >>> Jim Bridgewater, PhD > >> >>> Arizona State University > >> >>> 480-227-9592 > >> >>> > >> >>> _______________________________________________ > >> >>> nupic mailing list > >> >>> [email protected] > >> >>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >> >>> > >> >> > >> >> > >> >> _______________________________________________ > >> >> nupic mailing list > >> >> [email protected] > >> >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >> >> > >> > > >> > > >> > > >> > -- > >> > Jim Bridgewater, PhD > >> > Arizona State University > >> > 480-227-9592 > >> > >> > >> > >> -- > >> Jim Bridgewater, PhD > >> Arizona State University > >> 480-227-9592 > >> > >> _______________________________________________ > >> nupic mailing list > >> [email protected] > >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >> > > > > > > _______________________________________________ > > nupic mailing list > > [email protected] > > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > > > > > -- > Jim Bridgewater, PhD > Arizona State University > 480-227-9592 > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
