Hi Oreste, Yes, Phase 2 is really the key loop to optimize here. The strategy is also used in phases 1 and 3 - I think it helps there a bit too but not as much as Phase 2.
As far as implementation goes, there is a data structure called OutSynapses. It keeps track of the projections from every cell to other cells and segments. For each iteration, there are also arrays that keep track of how many inputs are converging into each segment and cell. When a new input comes in, these arrays are reset to zero. The code looks at each non-zero input and uses OutSynapses to update the two arrays. Then, you can just iterate over these arrays to actually see which cells have enough activity to be on. If you look in Cells4.cpp there is a function called computeForwardPropagation which implements the above logic. There are some additional optimizations in there (e.g. the cell activities are stored as bytes and you can do the check 8 cells at a time). There is some overhead in maintaining OutSynapses. Every time a new synapse is added or deleted it has to be updated. There is some cost to that, but in the end the benefits outweighed the costs. I am glossing over some details, but hopefully this gives you an idea of how the optimizations work. --Subutai On Thu, Sep 19, 2013 at 9:06 AM, Oreste Villa <[email protected]>wrote: > Thanks Marek and Subutai. Are we talking specifically about the nested > loops in phase 2 (lines from 42-44) or is this applicable to phase 1 and 3 > as well (of the TP obviously)? > Also, how do you keep track of the active segments without iterating them? > > > On Thu, Sep 19, 2013 at 8:49 AM, Subutai Ahmad <[email protected]>wrote: > >> >> Yes, that's the essence of it. If you implement TP the straightforward >> way, you need to loop over each segment in each cell to see which cells >> should be in predictive state. With this optimization you just visit those >> segments that have some activity. >> >> As far as the pseudo code in the whitepaper, this strategy by itself is >> just a code optimization trick. The end result should be identical to the >> pseudo code, but you will get it faster. >> >> --Subutai >> >> >> On Thu, Sep 19, 2013 at 8:40 AM, Marek Otahal <[email protected]>wrote: >> >>> As I understand it, instead of computing new values for each column/cell >>> (as the brain does, because it's totally local and parallel), we would >>> compute only those cells where some bit changed and is 1 (this requires we >>> have links: "input bit"-to what cells), since only 2% are on in a SDR, this >>> is the 50x. >>> ..or I'm completely wrong :) >>> >>> Cheers, Mark >>> >>> >>> On Thu, Sep 19, 2013 at 4:13 PM, Oreste Villa <[email protected]>wrote: >>> >>>> Hi all, >>>> >>>> I am trying to make sense to the following comment that I got a couple >>>> of weeks ago from Subutai regarding 50x speedup on the new algorithm >>>> for the TP. >>>> >>>> This is the comment: >>>> >>>> "TP is inherently slower and it was more challenging to optimize it. In >>>> the TP now rather than iterating over all the cells we iterate over all the >>>> ON input bits. The end result is identical but since we have about 2% >>>> sparsity in input bits, the latter method is about 50x faster. After >>>> optimizations, the SP and TP are now at roughly the same level in timing >>>> profiles. When the TP is more "full" of segments it becomes slower than SP >>>> again." >>>> >>>> Is there anybody available to explain how to exactly fit this into the >>>> white-paper pseudocode for the TP? >>>> >>>> In return, if I understand it enough and the topic comes up, >>>> I volunteer to help writing the "white paper 2.0" :-) >>>> >>>> Thanks, >>>> >>>> Oreste >>>> >>>> _______________________________________________ >>>> nupic mailing list >>>> [email protected] >>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>> >>>> >>> >>> >>> -- >>> Marek Otahal :o) >>> >>> _______________________________________________ >>> nupic mailing list >>> [email protected] >>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>> >>> >> >> _______________________________________________ >> nupic mailing list >> [email protected] >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> >> > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
