Hi Nick,
>> 1) Why are there no predictive states being seen for the first training pass >> (i.e. seeing the entire sequence once)? Even if activationThreshold and >> minThreshold are set sufficiently low to make segments sensitive, no lateral >> activation happens. Are cells initialized with no segments? That's right, cells are initialized with no segments. The segments are formed when a pattern is shown, and newly active cells form connections to previously active cells. >> 3) in the second training pass, we go from no predictive cells to perfectly >> predictive cells associated with the next character. I would typically >> expect the network to show scattered predictive cells before it hones in on >> the right prediction (consecutive 10 on-bits in this example). Why the >> abrupt shift in predictive behavior? That's because in this example, initialPerm == connectedPerm, so any synapses that are formed are immediately "connected". This allows the sequence to be learned in one pass. >> 4) finally, the printCells() function outputs the following. Can you please >> explain what each entry means? I'm not sure what the entries mean. However, I would recommend that if you're trying to understand the behavior of the temporal memory, take a look at the new implementation (temporal_memory.py) and tests for it (tutorial_temporal_memory_test.py and extensive_temporal_memory_test.py). They are easier to read and understand, and the implementation is closer to the pure white paper description. - Chetan On Thursday, Sep 4, 2014 at 1:37 PM, Nicholas Mitri <[email protected]>, wrote: Hey all, I’d like to dedicate this thread for discussing some TP implementation and practical questions, namely those associated with the introductory file to the TP, hello-tp.py. Below is the print out of a TP with 50 columns, 1 cell per column being trained as described in the py file for 2 iterations on the sequence A->B->C->D->E. Each pattern is fed directly into the TP as an active network state. I’ve been playing around with the configurations and have a few questions. 1) Why are there no predictive states being seen for the first training pass (i.e. seeing the entire sequence once)? Even if activationThreshold and minThreshold are set sufficiently low to make segments sensitive, no lateral activation happens. Are cells initialized with no segments? 2) if segments are created during initialization, how is their connectivity to the cells of the region configured? How are permanence values allocated? Same as proximal synapses in the TP? 3) in the second training pass, we go from no predictive cells to perfectly predictive cells associated with the next character. I would typically expect the network to show scattered predictive cells before it hones in on the right prediction (consecutive 10 on-bits in this example). Why the abrupt shift in predictive behavior? Is this related to getBestMatchingCell()? 4) finally, the printCells() function outputs the following. Can you please explain what each entry means? Column 41 Cell 0 : 1 segment(s) Seg #0 ID:41 True 0.2000000 ( 3/3 ) 0 [30,0]1.00 [31,0]1.00 [32,0]1.00 [33,0]1.00 [34,0]1.00 [35,0]1.00 [36,0]1.00 [37,0]1.00 [38,0]1.00 [39,0]1.00 Thanks, Nick ————————————— PRINT OUT———————————— ————— All the active and predicted cells: Inference Active state 1111111111 0000000000 0000000000 0000000000 0000000000 Inference Predicted state 0000000000 0000000000 0000000000 0000000000 0000000000 All the active and predicted cells: Inference Active state 0000000000 1111111111 0000000000 0000000000 0000000000 Inference Predicted state 0000000000 0000000000 0000000000 0000000000 0000000000 All the active and predicted cells: Inference Active state 0000000000 0000000000 1111111111 0000000000 0000000000 Inference Predicted state 0000000000 0000000000 0000000000 0000000000 0000000000 All the active and predicted cells: Inference Active state 0000000000 0000000000 0000000000 1111111111 0000000000 Inference Predicted state 0000000000 0000000000 0000000000 0000000000 0000000000 All the active and predicted cells: Inference Active state 0000000000 0000000000 0000000000 0000000000 1111111111 Inference Predicted state 0000000000 0000000000 0000000000 0000000000 0000000000 ############ Training Pass #1 Complete ############ All the active and predicted cells: Inference Active state 1111111111 0000000000 0000000000 0000000000 0000000000 Inference Predicted state 0000000000 1111111111 0000000000 0000000000 0000000000 All the active and predicted cells: Inference Active state 0000000000 1111111111 0000000000 0000000000 0000000000 Inference Predicted state 0000000000 0000000000 1111111111 0000000000 0000000000 All the active and predicted cells: Inference Active state 0000000000 0000000000 1111111111 0000000000 0000000000 Inference Predicted state 0000000000 0000000000 0000000000 1111111111 0000000000 All the active and predicted cells: Inference Active state 0000000000 0000000000 0000000000 1111111111 0000000000 Inference Predicted state 0000000000 0000000000 0000000000 0000000000 1111111111 All the active and predicted cells: Inference Active state 0000000000 0000000000 0000000000 0000000000 1111111111 Inference Predicted state 0000000000 0000000000 0000000000 0000000000 0000000000 ############ Training Pass #2 Complete ############ Hey all, I’d like to dedicate this thread for discussing some TP implementation and practical questions, namely those associated with the introductory file to the TP, hello-tp.py. Below is the print out of a TP with 50 columns, 1 cell per column being trained as described in the py file for 2 iterations on the sequence A->B->C->D->E. Each pattern is fed directly into the TP as an active network state. I’ve been playing around with the configurations and have a few questions. 1) Why are there no predictive states being seen for the first training pass (i.e. seeing the entire sequence once)? Even if activationThreshold and minThreshold are set sufficiently low to make segments sensitive, no lateral activation happens. Are cells initialized with no segments? 2) if segments are created during initialization, how is their connectivity to the cells of the region configured? How are permanence values allocated? Same as proximal synapses in the TP? 3) in the second training pass, we go from no predictive cells to perfectly predictive cells associated with the next character. I would typically expect the network to show scattered predictive cells before it hones in on the right prediction (consecutive 10 on-bits in this example). Why the abrupt shift in predictive behavior? Is this related to getBestMatchingCell()? 4) finally, the printCells() function outputs the following. Can you please explain what each entry means? Column 41 Cell 0 : 1 segment(s) Seg #0 ID:41 True 0.2000000 ( 3/3 ) 0 [30,0]1.00 [31,0]1.00 [32,0]1.00 [33,0]1.00 [34,0]1.00 [35,0]1.00 [36,0]1.00 [37,0]1.00 [38,0]1.00 [39,0]1.00 Thanks, Nick ————————————— PRINT OUT———————————— ————— All the active and predicted cells: Inference Active state 1111111111 0000000000 0000000000 0000000000 0000000000 Inference Predicted state 0000000000 0000000000 0000000000 0000000000 0000000000 All the active and predicted cells: Inference Active state 0000000000 1111111111 0000000000 0000000000 0000000000 Inference Predicted state 0000000000 0000000000 0000000000 0000000000 0000000000 All the active and predicted cells: Inference Active state 0000000000 0000000000 1111111111 0000000000 0000000000 Inference Predicted state 0000000000 0000000000 0000000000 0000000000 0000000000 All the active and predicted cells: Inference Active state 0000000000 0000000000 0000000000 1111111111 0000000000 Inference Predicted state 0000000000 0000000000 0000000000 0000000000 0000000000 All the active and predicted cells: Inference Active state 0000000000 0000000000 0000000000 0000000000 1111111111 Inference Predicted state 0000000000 0000000000 0000000000 0000000000 0000000000 ############ Training Pass #1 Complete ############ All the active and predicted cells: Inference Active state 1111111111 0000000000 0000000000 0000000000 0000000000 Inference Predicted state 0000000000 1111111111 0000000000 0000000000 0000000000 All the active and predicted cells: Inference Active state 0000000000 1111111111 0000000000 0000000000 0000000000 Inference Predicted state 0000000000 0000000000 1111111111 0000000000 0000000000 All the active and predicted cells: Inference Active state 0000000000 0000000000 1111111111 0000000000 0000000000 Inference Predicted state 0000000000 0000000000 0000000000 1111111111 0000000000 All the active and predicted cells: Inference Active state 0000000000 0000000000 0000000000 1111111111 0000000000 Inference Predicted state 0000000000 0000000000 0000000000 0000000000 1111111111 All the active and predicted cells: Inference Active state 0000000000 0000000000 0000000000 0000000000 1111111111 Inference Predicted state 0000000000 0000000000 0000000000 0000000000 0000000000 ############ Training Pass #2 Complete ############
