Chetan, Fergal, and David, Thank you for your feedback! Your comments are very helpful. Really appreciate you taking the time to answer my questions thoroughly.
Best, Nick On Aug 4, 2014, at 9:00 PM, Chetan Surpur <[email protected]> wrote: > Hi Nick, > > Great questions. Adding to the other answers that have been sent, here are my > clarifications to some of your questions (out of order): > >> 2- The wiki refers to encoder outputs as SDRs. Is that necessarily the case >> and if so, to what properties of encoder design is that requirement >> attributed to? (i.e. why do I need an SDR to be the output of the encoder as >> opposed to a binary vector unconstrained in density?) > > > Not necessarily. Encoders can output any kind of binary vector, and the > spatial pooler will convert them into SDRs. Most of the encoders that > currently exist output a sparse binary vector because that makes it easy to > distinguish between different outputs. But this is not a hard requirement. > >> 1- Are there any specific properties that encoders need to have when >> designing one? What’s the rationale behind them if they exist? > > > As Fergal mentioned, semantic similarity between inputs should translate to > overlapping bits in the outputs. The rationale is simply that this will allow > the CLA to generalize more easily between similar inputs. Otherwise learning > would be slower. > >> 3- Is there a biological counterpart for encoders in the general sense? > > Our biological sensors, like our retinas or cochleas. They have an > evolutionarily-driven hard-coded way of generating electrical activations > that have the above properties, and allow the lowest levels of the cortex to > learn patterns quickly. > > - Chetan > > On August 2, 2014 at 5:50:47 AM, Nicholas Mitri ([email protected]) wrote: > >> Hey all, >> >> Just a few quick questions about encoders I’d appreciate some feedback to. >> >> 1- Are there any specific properties that encoders need to have when >> designing one? What’s the rationale behind them if they exist? >> 2- The wiki refers to encoder outputs as SDRs. Is that necessarily the case >> and if so, to what properties of encoder design is that requirement >> attributed to? (i.e. why do I need an SDR to be the output of the encoder as >> opposed to a binary vector unconstrained in density?) >> 3- Is there a biological counterpart for encoders in the general sense? >> 4- Encoders perform quantization on the input stream by binning similar >> input patterns into hypercubes in feature space and assigning a single label >> (SDR or binary representation) to each bin. The encoder resolution >> determines the size of the hypercube. The SP essentially performs a very >> similar task by binning the outputs of the encoder in a binary feature space >> instead. City block distance determined by a threshold parameter controls >> the size of the hypercubes/bins. Why is this not viewed as a redundant >> operation by 2 consecutive modules of the HTM design? Is there a strong case >> for allowing for it? >> 5- Finally, is there any benefit to designing an encoding scheme that bins >> inputs into hyperspheres instead of hypercubes? Would the resulting >> combination of bins produce decision boundaries that might possibly allow >> for better binary classification performance for example? >> >> Thanks! >> Nick >> >> >> >> >> _______________________________________________ >> nupic mailing list >> [email protected] >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
