Rik, Well this particular email thread was about helping Jim with his current experiment, which dealt with memorizing the training set (no generalization). In that context, I believe all my comments were accurate.
However, I agree with you that a real notion of capacity has to go well beyond that. We have to consider generalization to novel inputs, appropriate similarity metrics, and how well the system might perform in a real world scenario. Beyond the SP, we also have to incorporate the (considerable) impact of sequence learning, pooling, and hierarchy. --Subutai On Fri, Jul 11, 2014 at 4:48 AM, Rik <[email protected]> wrote: > Sorry what value can be 1 billion? You keep bringing up that number. >> No spatial pooler of any sane dimensions has a capacity of 1 billion. Not >> for any reasonable definition of "capacity". >> > > In this case we are just trying to discriminate between a fixed set of > patterns. By "discriminate" we mean the SDR output should be unique with > respect to the other patterns by at least one winning column. Since there > are 1024 columns, of which 64 are on at a time, the total number of > patterns that can be discriminated is 1024 choose 64 > 10^102. In reality > it will be less than that, but if two inputs differ by more than a few > bits, we will have at least one column that is different. As such, there is > quite a bit of room here. > > The SP output can possibly produce 1 billion+ different patterns but that > is what I would consider its "resolution", not its "capacity" or "accuracy" > which is what we're after in this thread and what is the desirable property > that one seeks to evaluate and optimize. A completely unlearned SP will > also produce 1 billion+ different output patterns so that can't possibly be > an interesting property. > > A real-world analogy: The local jeweler weighs precious metals with scales > boasting a resolution of .01g but in the interest of getting a fair deal on > my silver I'd rather know their accuracy. Scales with a resolution of .01g > can still be off by 5g and usually will be so straight off the assembly > line so they have to be calibrated, a primitive material world version of > "learning". > > Or your digital SLR camera + computer monitor boast a 24bit color depth > allowing for 4 million+ colors but that's saying nothing about how > faithfully they reproduce the exact shade of red of a flower that you > photographed and there's a calibration/"learning" process too that involves > photographing known colors off a pantone sheet. > > A good definition of SP capacity or accuracy would IMHO involve a > correlation between inputs and ouputs, along the lines of how well clusters > in the input vector space correlate to clusters in the output vector space. > Just thinking out loud here. > > -- Rik > > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
