Thanks Matt, usually I'm not a model example for the official rules, so this is a warm feeling on the heart. You're making me a good person ;) ;P
Hi Subutai, I agree with you how tricky the encoders can be. And think it's useful for new(er) experimentors to ensure reasonable values. (on my old htm-cla I spent days figuring why the model CLA according to whitepaper is not working just for this regard). I also agree the we should leave open space for experimenting as much as possible when needed. An easy solution is in the ScalarEncoder, where I added a param. "forced" (defaults to False), which if set overrides all these checks. It would be easy to add it to other encoders as needed. I'd even go that far to set 21 as the default, and document the override in testing examples. PS: One more idea I think would be nice when playing with a new code, a constructor w/o any crazy params (w, ratio,...), we could provide a Simple*() version of the encoders with some defaults ppl can easily instantiate and see what do code does. On Sun, Nov 17, 2013 at 9:23 PM, Subutai Ahmad <[email protected]> wrote: > Hi Mark, > > Thanks for doing this and testing our new development process :-) > > It’s a good idea to improve the checking that is done in the encoders. > Also a great question: how many bits should be ON in the encoder? > > For error checks, I think there is a difference between checking for > values that are “reasonable” and values that are “required”. For example, > w>=3 might be a good check to have on all the time. No one should have a > value less than 3 and the code probably assumes that w>=3. We should throw > an exception for this case. > > A reasonable value though would be much higher. We discussed this quite a > bit at Grok over the years. We use a value of 21 in the OPF/Grok. The > reason for this is that you need a sufficiently high number of bits on to > have columns with a solid match and really be able to discriminate inputs. > If you used a value of 3 and just one field, there would be tons of columns > with overlap=3 and the winner would basically be random. A small shift in > the input will cause a large change in the SDR output. With higher w’s, the > winners will really match the input well and a small change in the input > should cause a small change in the output. > > As a philosophy, I don’t think we should throw an exception for situations > which are unreasonable but not strictly required by the code. It is often > useful to use unreasonable values when doing algorithm experimentation > (e.g. to test out the above statement). One problem with this is that > people might get themselves into bad situation without knowing about it. > Any thoughts on how to handle this? > > Thanks, > > —Subutai > > > > On Sun, Nov 17, 2013 at 11:14 AM, Matthew Taylor <[email protected]> wrote: > >> I just wanted to point out to all of you that that is exactly the way >> I think new features should be proposed, following the guidelines I've >> recently written up on the Development Process wiki[1]. Thanks, Marek, >> for the good example. >> >> [1] https://github.com/numenta/nupic/wiki/Development-Process >> --------- >> Matt Taylor >> OS Community Flag-Bearer >> Numenta >> >> >> On Sun, Nov 17, 2013 at 6:04 AM, Marek Otahal <[email protected]> >> wrote: >> > Scalar encoder is used almost everywhere (SDRCategory, Date enc,..). >> I've >> > found some (commented out) checks in SDRCategory encoder that ensure >> number >> > of ON bits is above certain level ( w>= 3). I thought this is >> reasonable and >> > should be used in ScalarEncoder as well. >> > >> > It's job of the spatial pooler to ensure that the final SDR has stable >> (~2%) >> > # of ON bits at all times. Yet it looks as a possible weakness to have >> an >> > encoder that has only 1 bit on at a time. Robustness to noice, comes to >> > mind. Do you think this is an issue? Limit of min 3 bits ON sounds >> > reasonable to me, it doesn't make things with small needs too big and >> yet is >> > somewhat robust. Also, in the brain, would some feature rarely depend on >> > just one neuron. >> > >> > I've prepared https://github.com/breznak/nupic/tree/scalar_w3 for you >> to >> > have a look. There's still one issue with a failing example, but seems >> ready >> > to me. >> > >> > Thanks, Mark >> > >> > -- >> > Marek Otahal :o) >> > >> > _______________________________________________ >> > nupic mailing list >> > [email protected] >> > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> > >> >> _______________________________________________ >> nupic mailing list >> [email protected] >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> > > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > -- Marek Otahal :o)
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
