Hi Dennis, On your last question about cells and the predictive connections: a typical NuPIC region has 2048 columns, each with 32 cells, so there are 64K cells. The synapses are formed opportunistically by cells when they become active - they choose randomly from other previously active cells and form synapses with them. This mimics what happens in the neocortex (growth is promoted by coincident activity) and is also efficient in NuPIC.
The 50% input fanout is for the default non-topological configuration of NuPIC. If you have strongly topological data (such as in vision), you would instead have each column receiving inputs preferentially from the same "location" in the 2D input space. You would also switch NuPIC to use local connections and inhibition strategies, which would mean that the predictive connections would be preferentially local too. Regards, Fergal Byrne On Thu, Nov 14, 2013 at 10:45 PM, Fergal Byrne <[email protected]>wrote: > Hi Dennis, > > The HTM is Jeff's over-arching theory of how the neocortex works. The CLA > is a very detailed model (suitable for implementation in software right > now) which describes how Layer 3 of a single region recognises spatial > patterns and learns sequences of those patterns. The CLA (and NuPIC) > encompasses three of the six principles of HTM, namely Sparse Distributed > Representations, Online Learning, and Sequence Memory. Hierarchy, Attention > and Motor Function are TBD right now! > > Because of the lack of hierarchy in NuPIC, it cannot compete with other > "some neuroscience" approaches such as Deep Belief Nets for applications > such as yours. However, these other approaches lack the temporal sequence > learning which is one of the core attributes of NuPIC. > > For categorisation of static images, I would look at Geoff Hinton's work > (he has a great, and free, course on Udemy). The system he describes looks > very much like what you're doing. > > Time-based learning in NuPIC would involve providing a region with a > varying input and have it learn the invariance. This would not be done by > giving it a set of real-world, unrelated images, but perhaps by using 3D > rendering software to feed it a realistic feed of successive, semantically > consistent frames, just like what we get when we perceive the world. I > still think we'll have to wait until we have a hierarchy of CLA's before we > get general vision happening here. > > Regards, > > Fergal Byrne > > > On Thu, Nov 14, 2013 at 10:09 PM, Chetan Surpur <[email protected]>wrote: > >> I believe that if you provide the time as a field, it will be split up >> into components (day of week, is weekend, etc.) and encoded as any ordinary >> integer. So NuPIC won't treat this any differently than any other integer >> field. The learning occurs online with every record, whether 10 minutes >> have passed in the time field, or whether 1 hour has passed. Hope that >> makes sense. >> >> >> On Thu, Nov 14, 2013 at 2:06 PM, Marek Otahal <[email protected]>wrote: >> >>> >>> On Thu, Nov 14, 2013 at 10:57 PM, Chetan Surpur <[email protected]>wrote: >>> >>>> You would show many variants of the same object in a short period of >>>> time to the HTM. It will associate them together using temporal pooling, >>>> and that's what gives you an invariant representation. Basically, time acts >>>> as a supervisor to correlate the variations. >>>> >>> +1 on the time being correlation supervisor, as that's how our minds >>> perceive it. >>> >>> Btw, how do "timed streams" work in Nupic? >>> >>> Is it you provide a field {data | time} , and the OPF model takes care >>> of "when difference T - (T-1) is too big, supress connections"? >>> >>> Or in a sequential manner, eg sending a sample every 1 sec, degrading >>> connections a bit every step. So say in 10 steps a connection is unlinked >>> unless boosted by a "correlating" example on input? >>> >>> >>> -- >>> Marek Otahal :o) >>> >>> _______________________________________________ >>> nupic mailing list >>> [email protected] >>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>> >>> >> >> _______________________________________________ >> nupic mailing list >> [email protected] >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> >> > > > -- > > Fergal Byrne, Brenter IT > > <http://www.examsupport.ie>http://inbits.com - Better Living through > Thoughtful Technology > > e:[email protected] t:+353 83 4214179 > Formerly of Adnet [email protected] http://www.adnet.ie > -- Fergal Byrne, Brenter IT <http://www.examsupport.ie>http://inbits.com - Better Living through Thoughtful Technology e:[email protected] t:+353 83 4214179 Formerly of Adnet [email protected] http://www.adnet.ie
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
