On Thu, Nov 14, 2013 at 11:43 PM, Dennis Stark <[email protected]> wrote:
> Now it is all clear! Thank you. One last question I have is about cells. > > Given an input each column is connected to 50% of the random input bit. > Ok, so with many cols (1000), it gives you huge overlap. > That's very clear and works great for spacial pooler. When it comes down > to temporal pooler, CLA deals with cells within the columns. Do I > understand this correctly, that each cell has synapses with other cells in > other columns (this would explain the learning). > In theory (Whitepaper, our brains..yes. See Hebbian learning "who fire together, wire together" and inhibition). For speed purposes, in the algo it;s done little different. > How many connections each cell has with other cells? Does each cell in > each column connect to every other possible cell in other columns (fully > connected network). Or is there a specific % of cells each cell has to be > connected to? I'm assuming this should be limited, otherwise let's say > there are 1024 columns with 16 cells in each (16384 cells in the whole > network), this would yield in 268,435,456 synapses that you have to keep > track of for temporal pooler.. > We maintain the sparsity of the input, say 2% cols always ON. To do this, we choose n-best columns with highest activation, and inhibit the rest. (extra: thus cells compete and must adapt to new, uncovered samples. Also that's how you represent huge space of context, cell Cxy might be on for "dog" when prev word was "my", and Cxz "dog" where prev was "Your" -> pattern for dog is same, but allows predictions for "My dog Ben", "Your dog Betty") > > On Nov 14, 2013, at 1:59 PM, Marek Otahal <[email protected]> wrote: > > > > > On Thu, Nov 14, 2013 at 10:28 PM, Dennis Stark <[email protected]> wrote: > >> >> > >> > The result is that NuPIC has some visual capabilities, but they are >> very limited, in terms of what you're talking about, by the lack of >> hierarchy in NuPIC at present. >> >> You mean hierarchy in terms of creating several regions and connecting >> them in a hierarchy? Isn't this one of the key points for HTM? >> > It is a key-point of a HTM. And nupic did experiments with it in past, as > I mentioned. But now Nupic is a CLA. HTM is a hierarchy of CLA. Still you > can do great for categorization and feature detection with a simple CLA's > spatial pooler. > > > -- > Marek Otahal :o) > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > -- Marek Otahal :o)
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
