Patrick, A good analogy from vision would be the current CLA encoders are like looking at pictures composed on binary black or white pixels. You definitely lose some subtlety over a gray scale image, but for most uses it will work well enough.
I don't know what the equivalent would be for audition. The dynamic range of any individual neuron is fairly limited, I think it is much less than the dynamic range you are able to perceive and detect. So distributed representations are somehow making up for the limited dynamic range of neurons. I suspect that a lot is known about how the cochlea does this but I am not familiar with that research. Jeff -----Original Message----- From: nupic [mailto:[email protected]] On Behalf Of Patrick Higgins Sent: Sunday, September 01, 2013 7:16 AM To: NuPIC general mailing list. Subject: Re: [nupic-dev] Inter-layer plumbing Thank you Jeff, Add me to the list of captivated members! That makes a lot of sense. I've wondered myself why the CLA looked boolean to me and does not use FSK for scaler encoding. It doesn't seem helpful for the case you describe, a quick match, but would it be helpful for a long gaze looking for subtle detail or the richness of experience once gets after the initial recognition / risk / anomaly assessment? How does scaler encoding work say, in the ear for the loudness of a frequency or say the eye for the brightness of a blue pixel in the retina? I understand it to be the rate of firing or as I tend to think if it, a FSK encoding of magnitude of the singular (granular) input. Is the current NuPIC way of encoding scalers achieving a similar result seen in the brain, where the rate of firings are converted to an SDR by distributing the information thru many pathways? Adding robustness and converting it to a boolean binary form much like the familiar sliding set of 1s? Thanks, I really appreciate it when you go into detail, when you have the time and interest. You certainly have other people here who feel the same. Patrick On Aug 31, 2013, at 2:12 PM, Jeff Hawkins wrote: > Ian, > I have nothing against scalar activations. Here are some more details about why the CLA doesn't have scalar cell activations today. > > - I have no doubt that scalar activations exist in the cortex and that > they can help in some situations. But. > - It has been shown that some pretty significant cognitive tasks can occur in the cortex too quickly to allow for scalar activations to help. E.g. a human or monkey can recognize a visual object in just a few hundred milliseconds. Neurons in a region such as IT (four levels up in the hierarchy) indicate that the cortex has recognized the object. If you count the minimum number of neuron to neuron transitions (add up all the delay times) there isn't enough time for a second spike to have an effect. Remember a fast spiking cell might produce spikes every 20msec. To represent a scalar cell output you need at least two spikes. So a realistic cortical model cannot require scalar cell activations all the time. > - The value of scalar activations is reduced in distributed representations. This is the biggest reason for me. The cortex always uses distributed representations. In a distributed representation the contribution of any individual cell is reduced. Any individual cell could die and the system keeps working just fine. The brain represents subtle differences by changing which cells are active in an SDR. > > I don't think it would be too burdensome to add scalar activations to the CLA, but at the moment I don't see how it will help us make the CLA more useful or resolve issues. If we understood how scalar activations would solve an outstanding CLA issue I wouldn't hesitate to add them. > > The SP actually uses scalar activations of a sort. The connections to each SP bit produce a scalar depolarization of the cell. Biologically the cell which depolarizes the fastest first fires and disables others. That is pretty accepted scheme. > Jeff _______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org _______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
