Hi, Hideaki-san, Thanks for your response! Some comments inline:
On Wed, Aug 21, 2013 at 1:55 PM, Hideaki Suzuki <[email protected]> wrote: > If you say "I have to solve it at least once myself" to mean implementing > by yourself, > I have the same feeling and have been working for three months of my part > time. > :-) > As you can see it in the mail archive (http://nupic.markmail.org/), the > current implementation of > Nupic focuses on 1D topology for the proof of concept in production level, > which means a set > of values are put to CLA, and CLA predicts for future values. Not much > "neighbors" and > "hierarchies" exist yet. > I figure that topology really should be conceptually separate - if you've got an array of stuff, you can represent that as any number of dimensions; it helps if n is an n-root of the number of elements, but if you control the number of elements, that's easy. So I'd think you'd have an abstraction for "topology" and when you want to find neighbors of a cell/column, you'd "ask the topology". Otherwise you'd end up hard-coding assumptions that might be difficult to rip out later. I find it easier to think regions in terms of 2D layers, but that's what makes sense for a human, not necessarily a computer. "proof of concept in production level" worries me a bit - I thought this thing was already in production use; did I misunderstand and most of this is untested theory? 1. In some places, the white paper implies that input bits are wired >> directly to columns; elsewhere it suggests that there is a network of >> synapses between input bits and columns in the layer that directly receives >> input. >> > > Yes. > Those synapses are the links between input bits and your columns. > Each column will have some number of synapses. Say, 256 or 1024. > Each synapse is either connected or disconnected to its input bit location > according to its permanence value. > So, I guess my question is: Does a *proximal* dendrite connect to more than one bit? > If you have 1024 columns with 1024 synapses in your region, and all > synapses are connected, you need to > check 1 million links to see how strong each column is activated in SP. > Yeah, this seems to be where the combinatorics go through the roof, and where hardware could help; probably some clever optimizations are possible to pre-determine a set that you don't need to check. > 3. It's mentioned several times that "strongly activated" columns inhibit >> nearby columns, to achieve sparseness. But it's also implied that the >> input to a cell is a single bit, which has no concept of strong or >> not-so-strong. Is "strongly activated" the count of cells in a column >> which are active due to feed-forward input, or is the input actually scalar >> and I just misunderstood that? >> > > As mentioned above, one column has many synapses. > This is the sort of statement that makes me wonder if I'm understanding things right. One *column* has many syapses, or one cell's dendrite has many synapses? Or we are summing column.cells.activeSynapseCount? > 4. I may be misunderstanding the nature of distal dendrite segments; I'm >> also trying to model the data efficiently. Is it sane to model a distal >> dendrite segment as >> - A column coordinate in a 2D topology - x,y coordinates >> - A list of path-traversal instructions - (i.e. up-left, down, down, >> right) in which no coordinate is traversed more than once >> - Permanence values mapped to instructions, to model synapses >> > > I may be misunderstanding your query..., but yes a column can be arranged > in 2D. > I was thinking of a 2D topology of columns, so you could refer to them as 0,0 ... 0,1 ... 0,2 ... 1,0 ... etc. In that case, say with 4 cells per column, the coordinates of one cell would be something like 0,1:3. > Distal dendrite segment is used for TP. > Cardinality is like... > > [Region] 1 --- 1..n [Column] > [Column] 1 --- 1 [Proximal Segment] > [Column] 1 --- 1..n [Cell] > [Cell] 1 --- 1..n [Distal Segment] > > [Proximal Segment] 1 --- 1..n [Proximal Synapse] > [Distal Segment] 1 --- 1..n [Distal Synapse] > That helps, thanks! > Potential synapses of a distal segment are setup virtually. > You will pickup any cell at random to make literal connections. > So basically, when creating a new distal segment, just randomly pick some cells "near" (in whatever topology) the cell it connects to? It seems like the choice of connections should profoundly effect the behavior of the system to the point of determining all of its future behavior. > >> 7. It's not clear how the initial state of the system (before any input) >> is established - i.e. you have to have some distal dendrite segments and >> potential synapses >> > > I believe distal segments and their potential synapses can be setup on the > fly. > Yes; the question is, what should be the algorithm to decide what cells/columns the distal segments connect to. Say that we have a grid of columns with 4-cells-per-column like ooooo ooxoo ooooo and we need to define the distal segment for x. Should we: - Pick a random path among the columns? Or some sort of gaussian distribution of neighboring columns? Or something else? - Have synapses with *every* cell in the columns intersected? Or just some? Or just one? > For edge and corner detection, I'm not so sure how well SP can handle. > At least, SP seems to work well for the roll of feature detection (V1) in > the video above. > I wasn't clear - I wasn't talking about edge and corner detection. Rather, say we have a grid of cells: xooox ooooo xooox The white paper states that columns are inhibited by other columns near them. In this grid, most cells have 8 neighbors which could inhibit or be inhibited by them. But the corner columns have only 3 neighbors and the edge cells have 5 neighbors. Which means a corner column will be less inhibited. So even if the signal activating it is less relevant, it has a higher probability of being active. This seems like it ought to be a problem; you could solve it by: - Ignoring predictions from perimeter columns - Wrapping around, so that above and to the left of 0,0 is 5,5 - Having such a large number of columns that erroneous values from (literally!) edge-cases have minimal effect Thanks, -Tim -- http://timboudreau.com
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
