Ian Danforth had a compelling hackathon demo last year on the visualization of the spatial pooler.
video: http://www.youtube.com/watch?v=X4XjYXFRIAQ&start=6110 code: https://github.com/iandanforth/spviewer I'm not sure if it will help you, but he helped me understand the SP better. I wonder if there is a way to add Ian's visualization to the work being done here with MNIST. --------- Matt Taylor OS Community Flag-Bearer Numenta On Fri, Jan 24, 2014 at 4:19 AM, Andreas Hartel < [email protected]> wrote: > Wouldn't it also be interesting to see how the SP performs when there are > only 10 columns with each column only containing one cell? After all > that's the number of classes you want to put the digits into. > > Or are there any implementation-related problems in the way of trying > this? Since I don't have that much experience with nupic, I'd be > interested in hearing what more experienced users think of this. > > Andreas > > -- > On Wed, Jan 22, 2014 at 11:04:00AM -0800, Scott Purdy wrote: > > I think there are three phases for doing this problem with NuPIC and each > > subsequent one will yield much better results. > > > > 1. Just use the classifier. This isn't what the classifier is intended > for and > > won't work very well. > > 2. Use the spatial pooler to learn invariant spatial representations and > the > > classifier to turn the SP output into a classification of which > character is in > > the image. > > 3. Add temporal pooling and hierarchy. This wasn't intuitive to me but is > > important for doing well at image recognition problems. The basic idea > is that > > you translate the training images and feed the moving characters into > the CLA > > (with SP, TP, and classifier). This learns the transitions of the > training > > characters as they move around in the input field or are stretched or > otherwise > > manipulated. With temporal pooling, higher levels in the hierarchy will > form > > stable representations of the character as it moves around and is > translated. > > During the testing phase, show it a single static image and it should > activate > > the stable representation of the character at the higher levels of the > > hierarchy and the classifier will have learned that that representation > matches > > the correct character. > > > > I haven't done these things but I would recommend putting your code > somewhere > > that we can collaborate to get the parameters right. I would start with > 2048 > > columns as Marek suggests (this has to match in both SP and TP). I don't > think > > 8 gigs memory should be an issue. You may also want to use topology but > I am > > not sure if that is necessary. > > > > > > On Wed, Jan 22, 2014 at 10:50 AM, Marek Otahal <[email protected]> > wrote: > > > > Hi, > > > > text follows below.. > > > > > > On Wed, Jan 22, 2014 at 7:29 PM, Allan Inoc ncio de Souza Costa < > > [email protected]> wrote: > > > > Of course! > > > > Following is my model_params. My memory can handle about 512 > columns in > > SP. Also, the pixels fields are setted in the end: > > > > MODEL_PARAMS = { > > # Type of model that the rest of these parameters apply to. > > 'model': "CLA", > > > > > > 'spEnable': True, > > > > > > So you are using spatial pooler actually. > > > > > > > > 'spParams': { > > # SP diagnostic output verbosity control; > > # 0: silent; >=1: some info; >=2: more info; > > 'spVerbosity' : 0, > > > > 'globalInhibition': 1, > > > > # Number of cell columns in the cortical region (same > > number for > > # SP and TP) > > # (see also tpNCellsPerCol) > > 'columnCount': 256, > > > > > > too low number I think, make it the default 2048, or even more. > > > > > > 'inputWidth': 256, > > > > Is this wrong? The input is much bigger (28x28x121) > > > > Missing here, but for speed use the cpp implementation (looks like > you're > > using a model_params file as a template, but an old one? Check hotgym > > example. > > > > > > > > # SP inhibition control (absolute value); > > # Maximum number of active columns in the SP region's > > output (when > > # there are more, the weaker ones are suppressed) > > 'numActivePerInhArea': 40, > > > > 'seed': 1956, > > > > # coincInputPoolPct > > # What percent of the columns's receptive field is > > available > > # for potential synapses. At initialization time, we > will > > # choose coincInputPoolPct * (2*coincInputRadius+1)^2 > > 'coincInputPoolPct': 0.5, > > > > # The default connected threshold. Any synapse whose > > # permanence value is above the connected threshold > is > > # a "connected synapse", meaning it can contribute > to the > > # cell's firing. Typical value is 0.10. Cells whose > > activity > > # level before inhibition falls below > minDutyCycleBeforeInh > > # will have their own internal synPermConnectedCell > > # threshold set below this default value. > > # (This concept applies to both SP and TP and so > 'cells' > > # is correct here as opposed to 'columns') > > 'synPermConnected': 0.1, > > > > 'synPermActiveInc': 0.1, > > > > 'synPermInactiveDec': 0.01, > > > > 'randomSP': 0, > > }, > > > > # Controls whether TP is enabled or disabled; > > # TP is necessary for making temporal predictions, such > as > > predicting > > # the next inputs. Without TP, the model is only > capable of > > # reconstructing missing sensor inputs (via SP). > > 'tpEnable' : False, > > > > > > > > 'clParams': { > > 'regionName' : 'CLAClassifierRegion', > > > > # Classifier diagnostic output verbosity control; > > # 0: silent; [1..6]: increasing levels of verbosity > > 'clVerbosity' : 0, > > > > # This controls how fast the classifier > learns/forgets. > > Higher values > > # make it adapt faster and forget older patterns > faster. > > 'alpha': 0.001, > > > > # This is set after the call to > updateConfigFromSubConfig > > and is > > # computed from the aggregationInfo and > predictAheadTime. > > 'steps': '0', > > }, > > > > 'anomalyParams': { > > u'anomalyCacheRecords': None, > > u'autoDetectThreshold': None, > > u'autoDetectWaitRecords': None > > }, > > > > 'trainSPNetOnlyIfRequested': False, > > } > > } > > > > for i in range(0,784): > > > > > > this does work? I thought this is just a json-like file? > > > > > > > MODEL_PARAMS['modelParams']['sensorParams']['encoders']['pixel%d' % > > i] = { > > 'fieldname': u'pixel%d' % i, > > 'n': 121, > > > > I'm surprised yous truggle to fot this in RAM, I;ve done experiments > with > > 100k wide encoded vectors. Try the cpp implementation. > > > > 'name': u'pixel%d' % i, > > 'type': 'ScalarEncoder', > > 'minval':0, > > 'maxval':255, > > > > Also, preprocessing just to b/w could help (?) > > > > 'w': 21} > > > > > > > > Cheers, Mark > > > > -- > > Marek Otahal :o) > > > > _______________________________________________ > > nupic mailing list > > [email protected] > > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > > > > > > > > _______________________________________________ > > nupic mailing list > > [email protected] > > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
