Hi Steve, Inhibition appears in three places in the biological theory of CLA, at least in my head! We haven't always pointed it out because it isn't necessarily useful to think of inhibitory neurons when implementing the CLA in software. The literature on inhibitory neurons is not as rich as it is for excitatory neurons so it is harder to be precise on this.
1) There are inhibitory neurons that enforce sparsity. (used to enforce sparsity) 2) There are inhibitory neurons that help all the cells in a column be activated together (these inhibitory cells inhibit other inhibitory cells in a column). This shows up in the software by having a column of cells activated by the SP. 3) There are inhibitory neurons that form inhibitory synapses along the distal dendrites. I speculate that these regulate the dendrite activation threshold of the dendrite branches, and therefore control the sparseness of the temporal pooler. If not enough cells are pooling then the threshold would be lowered. We have never implemented or tested this idea. I imagine that when you look at a cloud and I say "do you see the dog", your cortex lowers the threshold of the dendrites to encourage the cortex to recognize anything and hopefully see the dog shaped cloud. There are six or so different types of inhibitory neurons in cortex so the situation is undoubtedly more complex. As far as I know all the cells that enter the white matter are excitatory. So the feedback projections from one region to another are excitatory. The general consensus is feedback axons form excitatory synapses on the apical dendrites of cells in layers 2,3, and 5. There still could be an inhibitory effect but it would be secondary. We have not implemented feedback in a hierarchy other than some simple experiments before we had the CLA. What I think is happening is a higher-level representation projects to lower regions and associatively links to it. In this way the higher level region can tell the lower level region what sequence of activity it should recall. This would in effect eliminate alternate possibilities in the lower region. Perhaps this addresses your concern This would be much easier to discuss in person. Jeff -----Original Message----- From: nupic [mailto:[email protected]] On Behalf Of Steven Oberlin Sent: Monday, July 22, 2013 10:15 AM To: NuPIC general mailing list. Subject: [nupic-dev] Inhibition and feedback Re-reading the CLA whitepaper, one thing that I've noticed is that the only place inhibition appears is in the enforcement of the spacial pooler's columnar constraint of winners to encourage SDR encodings of input patterns. When arranging HTM regions in a hierarchy, I assume (perhaps incorrectly) that some of the feedback from higher-level HTMs to lower-level HTMs would be inhibitory, to reduce the likelihood of activations that aren't being predicted in the larger context of the higher level sequence being played out by the higher-level HTM (if that makes any sense). However, it seems to me that it is not currently possible to provide an inhibitory input into an HTM region because of the way input data is gated and summed by the spacial pooler, i.e., there is no way to learn that active input bits (1's in the input stream) mean recognition of a pattern should be suppressed. I suppose that feedback from higher-level to lower-level HTMs in a hierarchy could be excitatory-only, i.e., "1's from above" are learned in the mix of input bits by lower-levels to help gate predicted patterns, but then it seems to me that we would need a lot of copies of each feedback bit to multiply its semantic force so it could have a significant influence on the activation sum being computed by the spacial pooler. This seems inefficient, though it makes use of existing learning mechanisms. How is hierarchical feedback intended or imagined to be accomplished? Is inhibition necessary? Maybe feedback shouldn't even be injected along with "ordinary" feed-forward input bits, should instead be a factor in individual column boost calculations, or... Perhaps this is an out-of-scope topic. Let me know if I'm off in the weeds... -Steve O. _______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org _______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
