Hi Mark, Thanks for doing this and testing our new development process :-)
It’s a good idea to improve the checking that is done in the encoders. Also a great question: how many bits should be ON in the encoder? For error checks, I think there is a difference between checking for values that are “reasonable” and values that are “required”. For example, w>=3 might be a good check to have on all the time. No one should have a value less than 3 and the code probably assumes that w>=3. We should throw an exception for this case. A reasonable value though would be much higher. We discussed this quite a bit at Grok over the years. We use a value of 21 in the OPF/Grok. The reason for this is that you need a sufficiently high number of bits on to have columns with a solid match and really be able to discriminate inputs. If you used a value of 3 and just one field, there would be tons of columns with overlap=3 and the winner would basically be random. A small shift in the input will cause a large change in the SDR output. With higher w’s, the winners will really match the input well and a small change in the input should cause a small change in the output. As a philosophy, I don’t think we should throw an exception for situations which are unreasonable but not strictly required by the code. It is often useful to use unreasonable values when doing algorithm experimentation (e.g. to test out the above statement). One problem with this is that people might get themselves into bad situation without knowing about it. Any thoughts on how to handle this? Thanks, —Subutai On Sun, Nov 17, 2013 at 11:14 AM, Matthew Taylor <[email protected]> wrote: > I just wanted to point out to all of you that that is exactly the way > I think new features should be proposed, following the guidelines I've > recently written up on the Development Process wiki[1]. Thanks, Marek, > for the good example. > > [1] https://github.com/numenta/nupic/wiki/Development-Process > --------- > Matt Taylor > OS Community Flag-Bearer > Numenta > > > On Sun, Nov 17, 2013 at 6:04 AM, Marek Otahal <[email protected]> > wrote: > > Scalar encoder is used almost everywhere (SDRCategory, Date enc,..). I've > > found some (commented out) checks in SDRCategory encoder that ensure > number > > of ON bits is above certain level ( w>= 3). I thought this is reasonable > and > > should be used in ScalarEncoder as well. > > > > It's job of the spatial pooler to ensure that the final SDR has stable > (~2%) > > # of ON bits at all times. Yet it looks as a possible weakness to have an > > encoder that has only 1 bit on at a time. Robustness to noice, comes to > > mind. Do you think this is an issue? Limit of min 3 bits ON sounds > > reasonable to me, it doesn't make things with small needs too big and > yet is > > somewhat robust. Also, in the brain, would some feature rarely depend on > > just one neuron. > > > > I've prepared https://github.com/breznak/nupic/tree/scalar_w3 for you to > > have a look. There's still one issue with a failing example, but seems > ready > > to me. > > > > Thanks, Mark > > > > -- > > Marek Otahal :o) > > > > _______________________________________________ > > nupic mailing list > > [email protected] > > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
