Hi Marek,
Some good points there. I'll address them together rather than point by pint,
as they're intertwined.
First, the classifier is an artefact used to derive useful information from the
state of a region, it's never meant to be part of the CLA theory.
The SP and TP are only separate in the current implementation for engineering
reasons. You can use each alone if you like (useful for some applications) and
you can also subclass each separately (useful for others).
We currently lack feedback and motor connections, as well as thalamic mediated
connections; these will be added in time.
We do model inhibition in NuPIC and local inhibition is close enough to match
how the neocortex does it.
When discussing reconstruction of inputs, remember that it is the set of
predictive cells which are used to identify the associated inputs, not the
current set of active columns in the SP. There are no inhibited predictive
cells to be consulted.
We never need to reconstruct the inputs from the active columns - the input is
what causes the current activation pattern. When we have feedback from higher
regions affecting the activity of a region, we'll then have to employ
reconstruction (using feed forward permanences or a classifier) to get the
associated "imagined" inputs.
In the brain, we never need to do this, as the perception is the input as far
as our minds see it. Similarly, we don't need a classifier, as the output is
the identity when viewed from above.
—
Sent from Mailbox for iPhone
On Fri, Nov 22, 2013 at 5:41 PM, Marek Otahal <[email protected]>
wrote:
> In the following text I'll describe what (I think) Classifier does,
> why I consider it wrong and what can be done to avoid it?
> 1/ What is a Classifier:
> wiki doesn't say much, I got it's a part of Region that "translates SDRs
> back to input-space".
> Looking at the code at py/nupic/algorithms/CLAClassifier.py (and its c++
> sibling) I see there's a function compute() that basicaly pairs SDR (from
> lower layer, with input that caused the sdr), am I right?
> *) there's a KNNClassifier which would use k-nearest neighbours alg., and
> CLAClassifier..which uses what? a SP? A feed-forward NN seems to be a good
> candidate for such impl. of a classifier.
> 2/ Why I consider it "wrong"
> 2.1/ Classifier does not have a biological counterpart like other parts of
> HTM/CLA do.
> the chain :
> input --> encoder --> SP --> TP --> Classifier??!
> input can be whatever we have sensors to percieve, eg a "sound wave", it's
> of any possible data-type - input-space
> encoder is the function of the sensory organ - eg "cochlea translates the
> vibrations to the electromagnetic pulses on the cells ", it translates from
> inputspace to bit-vector (not a SDR, however)
> SP+TP: are combined together in the brain in a (micro-)region;
> they both accept and produce a SDR
> The Classifier does not have a counterpart as brain has no need to
> translate back to the input-space. We, however, do for CLAs to be useful in
> practical problems.
> 2.2/ lack of top-down compute in SP, TP breaks modularity.
> Encoder does have encode() and decode()/topDownCompute() methods. SP and TP
> dont. To make these two useful building blocks, it would be necessary to
> have the an inverse of compute() in these too.
> In nature, there are four types of connections between cells/neurons in
> brain: 2 vertical: feedforward feeding input to higher layer, and recurrent
> feeding more stable patterns to lower layers.
> And two horizontal: predictive connections (used in TP) and inhibitory
> (missing in Nupic).
> The inhibition connections are missing for performance reasons (I think)
> and we use global/local n-best inhibition instead. This fact makes it
> impossible to recostruct from a list of active columns (an SDR) the input
> that had caused it. If we had them (inh. permanences) we could use them in
> reverse meaning to boost the columns that have been silenced and from these
> "active+" columns according to permanences turn ON appropriate synapses.
> Such implementation would be slower, but interesting for bio-inspired
> research; allow SDR to be messaging format between parts of CLA (even diff
> implementations) and reduce possible space for errors happening in the
> Classifier.
> Are my thoughts proper/wrong somewhere? What do you think?
> Cheers, Mark
> --
> Marek Otahal :o)
_______________________________________________
nupic mailing list
[email protected]
http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
