Hi, Let me reply in-line.
2013/8/23 Tim Boudreau <[email protected]> > Specifically regarding the Spatial Pooler and Temporal Pooler, I > continually wonder if these are necessary concepts - a lot of the other > concepts have direct analogues with structures in the brain. These two are > the "odd men out". It seems to be where the algorithmic complexity landed, > but it seems more like an implementation detail - thing which mutates a > bunch of data structures - than a necessary component (the logic needs to > exist; aggregating it and naming it that way seems like a design choice > for putting a bunch of procedural code somewhere). Am I missing some key > aspect here? > For SP, at least, each piece of elements seems essential to obtain a good SDR for any input pattern sequence. As "Self adjusting receptive fields" in the white paper says, a region will automatically use its columns to best represents the input patterns. Missing one element would produce unwanted behaviors. The implementation can vary. e.g. using addition instead of multiplication, using integer instead of float, scanning only connected synapses instead of all synapses , using GPU's warp instead of threads and tasks,..., etc. I suppose CLA is the set of the essential concepts (logic) that achieve the same behavior of brain. We can implement it in various ways. The pseudo code in the white paper shows one possible implementation. That's where I keep getting stuck: Is there any guidance for how to map > input-bits:synapses:columns, and how to layout distal dendrites *before you > have seen any input* - i.e. you are starting from zero, new data, new > configuration. You have to choose *some* initial layout/mapping for these > things. It's easy to randomly generate a bunch of connections and hope the > result is useful. But it seems like there ought to be some suggested > approach, not just randomly do stuff and see what comes out. > AFAIK, the only thing in public mentioning the initialization of proximal synapses is Chapter 3 "Initialization" on Page 34. I've done like the blow. 1. create an array to hold all pixel locations (x,y) in the square area of [-fanoutRadius,-fanoutRadius] - [fanoutRadius,fanoutRadius] 2. randomize the array 3. pick up the first N elements from the array, if one column has N synapses. 4. for each (x,y) in the picked-up elements of the array, initialize a potential synapses S in such a way that S.x = x + x0 S.y = y + y0 S.permanance = connectedPerm + rand(smallRange) + bias(distance) where (x0,y0) is the natural center of the column on the bitmap image, connectedPerm is the threshold for connected synapses, bias is the weight on the permanence, distance is sqrt(x^2+y^2). To eliminate the artificial boundary, I also did like x %= imageWidth if (x<0) x += imageWidth Actually, I didn't use x and y but instead use a single index by linearization because multi-dimensional array is too slow in C# and it's nice to separate out dimensions in CLA. I will upload my toy to GitHub, if anybody is interested in. Hrm, are we looking at the same thing? In my copy of the white paper (is > there more than one?), page 42 is pseudocode for Phase 2 and Phase 3. > Ops. It is on page 43. I'm sorry for my typo... Best Regards, Hideaki Suzuki.
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
