Hi Nick, Great questions. Adding to the other answers that have been sent, here are my clarifications to some of your questions (out of order):
2- The wiki refers to encoder outputs as SDRs. Is that necessarily the case and if so, to what properties of encoder design is that requirement attributed to? (i.e. why do I need an SDR to be the output of the encoder as opposed to a binary vector unconstrained in density?) Not necessarily. Encoders can output any kind of binary vector, and the spatial pooler will convert them into SDRs. Most of the encoders that currently exist output a sparse binary vector because that makes it easy to distinguish between different outputs. But this is not a hard requirement. 1- Are there any specific properties that encoders need to have when designing one? What’s the rationale behind them if they exist? As Fergal mentioned, semantic similarity between inputs should translate to overlapping bits in the outputs. The rationale is simply that this will allow the CLA to generalize more easily between similar inputs. Otherwise learning would be slower. 3- Is there a biological counterpart for encoders in the general sense? Our biological sensors, like our retinas or cochleas. They have an evolutionarily-driven hard-coded way of generating electrical activations that have the above properties, and allow the lowest levels of the cortex to learn patterns quickly. - Chetan On August 2, 2014 at 5:50:47 AM, Nicholas Mitri ([email protected]) wrote: Hey all, Just a few quick questions about encoders I’d appreciate some feedback to. 1- Are there any specific properties that encoders need to have when designing one? What’s the rationale behind them if they exist? 2- The wiki refers to encoder outputs as SDRs. Is that necessarily the case and if so, to what properties of encoder design is that requirement attributed to? (i.e. why do I need an SDR to be the output of the encoder as opposed to a binary vector unconstrained in density?) 3- Is there a biological counterpart for encoders in the general sense? 4- Encoders perform quantization on the input stream by binning similar input patterns into hypercubes in feature space and assigning a single label (SDR or binary representation) to each bin. The encoder resolution determines the size of the hypercube. The SP essentially performs a very similar task by binning the outputs of the encoder in a binary feature space instead. City block distance determined by a threshold parameter controls the size of the hypercubes/bins. Why is this not viewed as a redundant operation by 2 consecutive modules of the HTM design? Is there a strong case for allowing for it? 5- Finally, is there any benefit to designing an encoding scheme that bins inputs into hyperspheres instead of hypercubes? Would the resulting combination of bins produce decision boundaries that might possibly allow for better binary classification performance for example? Thanks! Nick _______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
