I created a repo: https://github.com/allanino/nupic-classifier-mnist
I followed some suggestions (special thanks to Marek), and was able to achieve
an accuracy of 61.5% on the test set (composed of 20% of the data) with the
code currently in that repo. Disabling SP, the accuracy drops to 49.7%.
That is a better result than I previously reported.
But I still don't know how the pure classifier (without SP) works. I think I
will need to try harder to understand the code.
Best regards,
Allan
Em Sexta-feira, 24 de Janeiro de 2014 13:51, Matthew Taylor <[email protected]>
escreveu:
Ian Danforth had a compelling hackathon demo last year on the visualization of
the spatial pooler.
video: http://www.youtube.com/watch?v=X4XjYXFRIAQ&start=6110
code: https://github.com/iandanforth/spviewer
I'm not sure if it will help you, but he helped me understand the SP better. I
wonder if there is a way to add Ian's visualization to the work being done here
with MNIST.
---------Matt Taylor
OS Community Flag-Bearer
Numenta
On Fri, Jan 24, 2014 at 4:19 AM, Andreas Hartel <[email protected]>
wrote:
Wouldn't it also be interesting to see how the SP performs when there are
>only 10 columns with each column only containing one cell? After all
>that's the number of classes you want to put the digits into.
>
>Or are there any implementation-related problems in the way of trying
>this? Since I don't have that much experience with nupic, I'd be
>interested in hearing what more experienced users think of this.
>
>Andreas
>
>--
>
>On Wed, Jan 22, 2014 at 11:04:00AM -0800, Scott Purdy wrote:
>> I think there are three phases for doing this problem with NuPIC and each
>> subsequent one will yield much better results.
>>
>> 1. Just use the classifier. This isn't what the classifier is intended for
>> and
>> won't work very well.
>> 2. Use the spatial pooler to learn invariant spatial representations and the
>> classifier to turn the SP output into a classification of which character is
>> in
>> the image.
>> 3. Add temporal pooling and hierarchy. This wasn't intuitive to me but is
>> important for doing well at image recognition problems. The basic idea is
>> that
>> you translate the training images and feed the moving characters into the CLA
>> (with SP, TP, and classifier). This learns the transitions of the training
>> characters as they move around in the input field or are stretched or
>> otherwise
>> manipulated. With temporal pooling, higher levels in the hierarchy will form
>> stable representations of the character as it moves around and is translated.
>> During the testing phase, show it a single static image and it should
>> activate
>> the stable representation of the character at the higher levels of the
>> hierarchy and the classifier will have learned that that representation
>> matches
>> the correct character.
>>
>> I haven't done these things but I would recommend putting your code somewhere
>> that we can collaborate to get the parameters right. I would start with 2048
>> columns as Marek suggests (this has to match in both SP and TP). I don't
>> think
>> 8 gigs memory should be an issue. You may also want to use topology but I am
>> not sure if that is necessary.
>>
>>
>> On Wed, Jan 22, 2014 at 10:50 AM, Marek Otahal <[email protected]> wrote:
>>
>> Hi,
>>
>> text follows below..
>>
>>
>> On Wed, Jan 22, 2014 at 7:29 PM, Allan Inoc ncio de Souza Costa <
>
>> [email protected]> wrote:
>>
>> Of course!
>>
>> Following is my model_params. My memory can handle about 512 columns
>> in
>> SP. Also, the pixels fields are setted in the end:
>>
>> MODEL_PARAMS = {
>> # Type of model that the rest of these parameters apply to.
>> 'model': "CLA",
>>
>>
>> 'spEnable': True,
>>
>>
>> So you are using spatial pooler actually.
>>
>>
>>
>> 'spParams': {
>> # SP diagnostic output verbosity control;
>> # 0: silent; >=1: some info; >=2: more info;
>> 'spVerbosity' : 0,
>>
>> 'globalInhibition': 1,
>>
>> # Number of cell columns in the cortical region (same
>> number for
>> # SP and TP)
>> # (see also tpNCellsPerCol)
>> 'columnCount': 256,
>>
>>
>> too low number I think, make it the default 2048, or even more.
>>
>>
>> 'inputWidth': 256,
>>
>> Is this wrong? The input is much bigger (28x28x121)
>>
>> Missing here, but for speed use the cpp implementation (looks like you're
>> using a model_params file as a template, but an old one? Check hotgym
>> example.
>>
>>
>>
>> # SP inhibition control (absolute value);
>> # Maximum number of active columns in the SP region's
>> output (when
>> # there are more, the weaker ones are suppressed)
>> 'numActivePerInhArea': 40,
>>
>> 'seed': 1956,
>>
>> # coincInputPoolPct
>> # What percent of the columns's receptive field is
>> available
>> # for potential synapses. At initialization time, we will
>> # choose coincInputPoolPct * (2*coincInputRadius+1)^2
>> 'coincInputPoolPct': 0.5,
>>
>> # The default connected threshold. Any synapse whose
>> # permanence value is above the connected threshold is
>> # a "connected synapse", meaning it can contribute to the
>> # cell's firing. Typical value is 0.10. Cells whose
>> activity
>> # level before inhibition falls below
>> minDutyCycleBeforeInh
>> # will have their own internal synPermConnectedCell
>> # threshold set below this default value.
>> # (This concept applies to both SP and TP and so 'cells'
>> # is correct here as opposed to 'columns')
>> 'synPermConnected': 0.1,
>>
>> 'synPermActiveInc': 0.1,
>>
>> 'synPermInactiveDec': 0.01,
>>
>> 'randomSP': 0,
>> },
>>
>> # Controls whether TP is enabled or disabled;
>> # TP is necessary for making temporal predictions, such as
>> predicting
>> # the next inputs. Without TP, the model is only capable of
>> # reconstructing missing sensor inputs (via SP).
>> 'tpEnable' : False,
>>
>>
>>
>> 'clParams': {
>> 'regionName' : 'CLAClassifierRegion',
>>
>> # Classifier diagnostic output verbosity control;
>> # 0: silent; [1..6]: increasing levels of verbosity
>> 'clVerbosity' : 0,
>>
>> # This controls how fast the classifier learns/forgets.
>> Higher values
>> # make it adapt faster and forget older patterns faster.
>> 'alpha': 0.001,
>>
>> # This is set after the call to updateConfigFromSubConfig
>> and is
>> # computed from the aggregationInfo and predictAheadTime.
>> 'steps': '0',
>> },
>>
>> 'anomalyParams': {
>> u'anomalyCacheRecords': None,
>> u'autoDetectThreshold': None,
>> u'autoDetectWaitRecords': None
>> },
>>
>> 'trainSPNetOnlyIfRequested': False,
>> }
>> }
>>
>> for i in range(0,784):
>>
>>
>> this does work? I thought this is just a json-like file?
>>
>>
>>
>> MODEL_PARAMS['modelParams']['sensorParams']['encoders']['pixel%d' %
>> i] = {
>> 'fieldname': u'pixel%d' % i,
>> 'n': 121,
>>
>> I'm surprised yous truggle to fot this in RAM, I;ve done experiments with
>> 100k wide encoded vectors. Try the cpp implementation.
>>
>> 'name': u'pixel%d' % i,
>> 'type': 'ScalarEncoder',
>> 'minval':0,
>> 'maxval':255,
>>
>> Also, preprocessing just to b/w could help (?)
>>
>> 'w': 21}
>>
>>
>>
>> Cheers, Mark
>>
>> --
>> Marek Otahal :o)
>>
>> _______________________________________________
>> nupic mailing list
>> [email protected]
>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
>>
>>
>>
>
>> _______________________________________________
>> nupic mailing list
>> [email protected]
>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
>
>
>_______________________________________________
>nupic mailing list
>[email protected]
>http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
>
_______________________________________________
nupic mailing list
[email protected]
http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org_______________________________________________
nupic mailing list
[email protected]
http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
