Thank you for the detailed response Jeff. Through watching Rahul's video, and your explanation, it has become more clear. What was confusing me was how the data can be compressed effectively without losing meaningful information. I now understand better why the output of the Spatial Pooler must be fixed in size and sparsity. So, yes - the next step for me is to have a better understanding of how the Spatial Pooler does it's thing. This is all new territory for me, so I am moving very slowly and carefully through the material right now. If I get stuck again, it is great to know that there is such a great community ready to help!
- Jeff F. On Mon, Oct 21, 2013 at 1:28 PM, Jeff Hawkins <[email protected]> wrote: > Perhaps this wasn’t written as well as it should have been.**** > > ** ** > > The spatial pooler converts one sparse representation into another sparse > representation. The output of a spatial pooler has a fixed number of bits > (equal to the column number) and has a relatively fixed sparsity, say 2%. > The spatial pooler works just fine with a range in the number of input bits > and a range in sparsity. In some ways the goal of the SP is handle any > amount of input, convert it to a fixed size and sparsity output. The other > thing it does is learn the common spatial patterns in the input and make > sure to represent those well.**** > > ** ** > > The output sparsity of the SP needs to be relatively fixed for the > temporal pooler (sequence memory) to work. The number of output bits, > equal to the number of columns, also has to be fixed for the TP to work.** > ** > > ** ** > > Why is it important that the input can vary? In a real brain the > hierarchy of the neocortex is complicated and messy. Multiple regions > converge onto destination regions as you ascend the hierarchy. By allowing > the number of input bits to vary over a wide range, evolution could wire up > the hierarchy lots of different ways and the cortex continues to work ok. > If we took an existing brain and then added a connection between two > regions that previously were not connected the SP in the destination region > wouldn’t break. For example, in normal humans the size of primary visual > cortex varies by a factor of 3, but the size of the output of the retina is > always about 1M fibers. The SP in V1 can handle a broad range in the ratio > of the number of input bits and the number of output bits.**** > > ** ** > > The sparsity level of the input can vary due to multiple reasons. Lack of > sensory input, change in attention (which effectively turns off input > bits), and due to temporal pooling itself. So it is important that the > spatial pooler take whatever it is given and converts it into a relatively > fixed output.**** > > ** ** > > This is why the SP does what it does and why it is important. Do you need > help understanding how the SP does this?**** > > Jeff**** > > ** ** > > *From:* nupic [mailto:[email protected]] *On Behalf Of *Jeff > Fohl > *Sent:* Sunday, October 20, 2013 6:41 PM > *To:* [email protected] > *Subject:* [nupic-dev] Looking for help in understanding part of the HTM > white paper**** > > ** ** > > Hello -**** > > ** ** > > I hope this is not being posted to the wrong list. This is my first post > here. Please let me know if there is a more appropriate place for this > question.**** > > ** ** > > In preparation for learning NuPIC, I have read "On Intelligence", and I am > now reading the HTM white paper put out by Numenta.**** > > ** ** > > Making my way through the white paper, I got stuck on one passage, which I > can't really make sense of. Wondering if anyone can help me through this > part. The passage in question is on pages 11-12 of the white paper PDF - > specifically the second paragraph included below.**** > > ** ** > > *HTM regions also use sparse distributed representations. In fact, the > memory mechanisms within an HTM region are dependent on using sparse > distributed representations, and wouldn’t work otherwise. The input to an > HTM region is always a distributed representation, but it may not be > sparse, so the first thing an HTM region does is to convert its input into > a sparse distributed representation.***** > > *For example, a region might receive 20,000 input bits. The percentage of > input bits that are “1” and “0” might vary significantly over time. One > time there might be 5,000 “1” bits and another time there might be 9,000 > “1” bits. The HTM region could convert this input into an internal > representation of 10,000 bits of which 2%, or 200, are active at once, > regardless of how many of the input bits are “1”. As the input to the HTM > region varies over time, the internal representation also will change, but > there always will be about 200 bits out of 10,000 active. ***** > > So, what exactly is going on here? How does a fluctuating input flow of > 20,000 bits get converted into 200 bits? Obviously there is something > important going on here, but I don't understand what it is. Any help > illuminating this would be greatly appreciated!**** > > Many thanks,**** > > Jeff**** > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
