Hello David and Subutai's post are convincing.
Your question > What does the term "UniformLink" mean? You can find UniformLink implementation in nupic.core repository. https://github.com/numenta/nupic.core/blob/master/src/test/unit/engine/UniformLinkPolicyTest.cpp Best regards. 2016-01-14 10:44 GMT+09:00 Subutai Ahmad <[email protected]>: > Hi all, > > David's summary is spot-on. Let me add a bit more color: > > From an API standpoint, the Network API can fully support hierarchies in the > sense that you can chain together regions that send output to higher level > regions. Each region is responsible for deciding what should be sent to the > next level. The OPF (which is built on top of the Network API) does not > really support hierarchy - the OPF currently supports a very restricted set > of models such as our streaming anomaly detection and prediction models. > You can also bypass the Network API and just chain together low-level > algorithm classes yourself. > > But that is a coding thing and has little to do with algorithms and theory. > It's unclear if simply stacking the existing algorithms will do any good (it > probably won't). Within our research group we are actively trying to > understand and implement hierarchies (and temporal pooling) the same way it > is done in cortex. You can see some of our work described here [1]. However > we tend to be pretty methodical. We are trying to do things the "right way" > and validate algorithms every step of the way. The link below is our third > (fourth? fifth?) attempt at hierarchies and we're still not happy with it. > We are not even 100% sure exactly what we want from hierarchies - we often > revisit this in our internal discussions and Jeff has discussed some his > thoughts on the mailing list. > > I think it would be great if members of the community wanted to do research > in this area. I think there is a lot of low-hanging fruit here - much can be > accomplished before the full theory is in place. The mailing list > nupic-theory would be the best place to discuss the algorithm stuff. > > --Subutai > > [1] > https://github.com/numenta/nupic.research/wiki/Overview-of-the-Temporal-Pooler > > On Wed, Jan 13, 2016 at 9:49 AM, cogmission (David Ray) > <[email protected]> wrote: >> >> Nice work Sebastian! >> >> I agree, Hierarchy as a concept is very important to neocortex emulation, >> and to my knowledge Numenta has not released and is still working on an >> implementation of this feature. The make-shift "hierarchies" that can be >> composed using repeated combinations of the current algorithms are not >> "true" hierarchies in the HTM sense (ascending regional constructs). This >> being the case, for the time being repeated layers may not be as useful >> (depending on your application of course), as they will be when the >> hierarchy is actually part of the algorithm. >> >> Cheers, >> David >> >> On Wed, Jan 13, 2016 at 11:35 AM, Sebastián Narváez <[email protected]> >> wrote: >>> >>> I had the same problem as you, and implemented a Layer class to play >>> easly with different combinations of the Nupic modules (encoders, sps and >>> tms). You can check it out here: >>> >>> https://github.com/larvasapiens/htm-teul/tree/master/Learning >>> >>> However, from my experiments, a one-level network >>> (encoder->sp->tm->classifier) works better than a multi-level hierarchical >>> approach with the current implementation of nupic. Or maybe I didn't used >>> the correct parameters for my experiments. >>> >>> On Wed, Jan 13, 2016 at 11:43 AM, cogmission (David Ray) >>> <[email protected]> wrote: >>>> >>>> Hi Wakan, >>>> >>>> I while back a worked up a very raw example of connecting the algorithms >>>> (in a very rough manner), in Python. Maybe this might be of some help? >>>> (see attached) >>>> >>>> Cheers, >>>> David >>>> >>>> On Wed, Jan 13, 2016 at 8:59 AM, 박진만 <[email protected]> wrote: >>>>> >>>>> I did not use Network API. I made the structure by combining some >>>>> example codes like hello_sp.py and hello_tm.py in >>>>> https://github.com/numenta/nupic/tree/master/examples >>>>> >>>>> I made my midi encoder by combining 3 scalar encoders and 1 category >>>>> encoder. I used FastCLAClassifier in nupic.bindings.algorithms >>>>> >>>>> I want to make an hierarchical structure of HTM, not using network API, >>>>> but using those raw simple codes. >>>>> >>>>> Thank you. >>>>> >>>>> 2016-01-13 23:35 GMT+09:00 Wakan Tanka <[email protected]>: >>>>>> >>>>>> Hello, >>>>>> May I ask which framework are you using? OPF or Network API, from what >>>>>> you've typed I guess this is Network API. I'm wondering how you made: >>>>>> raw midi data -> Encoder -> Spatial Pooler -> Temporal Pooler -> CLA > >>>>>> Classifier -> prediction >>>>>> Also what encoder did you use? Have you followed some example codes? >>>>>> >>>>>> Thank you >>>>>> >>>>>> On 01/13/2016 02:47 PM, 박진만 wrote: >>>>>>> >>>>>>> Hello, I'm newbie to NUPIC& NUPIC-mailing list. >>>>>>> >>>>>>> I'm working on training midi files(*.mid , a sort of music file) >>>>>>> using >>>>>>> low-level codes. >>>>>>> >>>>>>> Low-level codes mean just a raw sp and tp code, not network API or >>>>>>> OPF. >>>>>>> I prefer to use low-level codes because it's easier for me to modify >>>>>>> the >>>>>>> codes. >>>>>>> I used a simple structure like this: >>>>>>> raw midi data -> Encoder -> Spatial Pooler -> Temporal Pooler -> CLA >>>>>>> Classifier -> prediction >>>>>>> The result was quit awesome. the HTM successfully predicted the whole >>>>>>> sequence with no error. >>>>>>> >>>>>>> Then I wanted to change the structure to be hierarchical like this : >>>>>>> raw midi data -> Encoder -> SP1 -> TP1 -> SP2 -> TP2 -> CLA >>>>>>> Classifier >>>>>>> -> prediction >>>>>>> >>>>>>> but I cannot implement the structure because i don't know how the >>>>>>> layer >>>>>>> 1 and layer 2 is linked. I already watched >>>>>>> "hierarchy_network_demo.py", >>>>>>> but the code just tells us "UniformLink". >>>>>>> What does the term "UniformLink" mean? >>>>>>> >>>>>>> I think it's gonna be a strange architecture if TP1's output(array of >>>>>>> cells) becomes the input of SP2, because in this hierarchy, layer 2 >>>>>>> (SP2 >>>>>>> and TP2) will have bigger column dimension than those of layer 1, >>>>>>> which >>>>>>> is somehow weird. >>>>>>> >>>>>>> to sum up, my questions are : >>>>>>> >>>>>>> 1. How they are linked between layers. >>>>>>> 2. Any hierarchy structure examples in low-level (not Network API, >>>>>>> not OPF) >>>>>>> >>>>>>> Any comments would be very helpful. >>>>>>> >>>>>>> Thank you. >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> With kind regards, >>>> >>>> David Ray >>>> Java Solutions Architect >>>> >>>> Cortical.io >>>> Sponsor of: HTM.java >>>> >>>> [email protected] >>>> http://cortical.io >>> >>> >> >> >> >> -- >> With kind regards, >> >> David Ray >> Java Solutions Architect >> >> Cortical.io >> Sponsor of: HTM.java >> >> [email protected] >> http://cortical.io > > -- Kentaro Iizuka<[email protected]> Github https://github.com/iizukak/
