Hello, Is there a recommended level for input sparsity? What is the minimum and maximum sparsity it can work functionally with?
Thanks, Pedro. On Mon, Oct 21, 2013 at 6:28 PM, Jeff Hawkins <[email protected]> wrote: > Perhaps this wasn’t written as well as it should have been.**** > > ** ** > > The spatial pooler converts one sparse representation into another sparse > representation. The output of a spatial pooler has a fixed number of bits > (equal to the column number) and has a relatively fixed sparsity, say 2%. > The spatial pooler works just fine with a range in the number of input bits > and a range in sparsity. In some ways the goal of the SP is handle any > amount of input, convert it to a fixed size and sparsity output. The other > thing it does is learn the common spatial patterns in the input and make > sure to represent those well.**** > > ** ** > > The output sparsity of the SP needs to be relatively fixed for the > temporal pooler (sequence memory) to work. The number of output bits, > equal to the number of columns, also has to be fixed for the TP to work.** > ** > > ** ** > > Why is it important that the input can vary? In a real brain the > hierarchy of the neocortex is complicated and messy. Multiple regions > converge onto destination regions as you ascend the hierarchy. By allowing > the number of input bits to vary over a wide range, evolution could wire up > the hierarchy lots of different ways and the cortex continues to work ok. > If we took an existing brain and then added a connection between two > regions that previously were not connected the SP in the destination region > wouldn’t break. For example, in normal humans the size of primary visual > cortex varies by a factor of 3, but the size of the output of the retina is > always about 1M fibers. The SP in V1 can handle a broad range in the ratio > of the number of input bits and the number of output bits.**** > > ** ** > > The sparsity level of the input can vary due to multiple reasons. Lack of > sensory input, change in attention (which effectively turns off input > bits), and due to temporal pooling itself. So it is important that the > spatial pooler take whatever it is given and converts it into a relatively > fixed output.**** > > ** ** > > This is why the SP does what it does and why it is important. Do you need > help understanding how the SP does this?**** > > Jeff**** > > ** ** > > *From:* nupic [mailto:[email protected]] *On Behalf Of *Jeff > Fohl > *Sent:* Sunday, October 20, 2013 6:41 PM > *To:* [email protected] > *Subject:* [nupic-dev] Looking for help in understanding part of the HTM > white paper**** > > ** ** > > Hello -**** > > ** ** > > I hope this is not being posted to the wrong list. This is my first post > here. Please let me know if there is a more appropriate place for this > question.**** > > ** ** > > In preparation for learning NuPIC, I have read "On Intelligence", and I am > now reading the HTM white paper put out by Numenta.**** > > ** ** > > Making my way through the white paper, I got stuck on one passage, which I > can't really make sense of. Wondering if anyone can help me through this > part. The passage in question is on pages 11-12 of the white paper PDF - > specifically the second paragraph included below.**** > > ** ** > > *HTM regions also use sparse distributed representations. In fact, the > memory mechanisms within an HTM region are dependent on using sparse > distributed representations, and wouldn’t work otherwise. The input to an > HTM region is always a distributed representation, but it may not be > sparse, so the first thing an HTM region does is to convert its input into > a sparse distributed representation.***** > > *For example, a region might receive 20,000 input bits. The percentage of > input bits that are “1” and “0” might vary significantly over time. One > time there might be 5,000 “1” bits and another time there might be 9,000 > “1” bits. The HTM region could convert this input into an internal > representation of 10,000 bits of which 2%, or 200, are active at once, > regardless of how many of the input bits are “1”. As the input to the HTM > region varies over time, the internal representation also will change, but > there always will be about 200 bits out of 10,000 active. ***** > > So, what exactly is going on here? How does a fluctuating input flow of > 20,000 bits get converted into 200 bits? Obviously there is something > important going on here, but I don't understand what it is. Any help > illuminating this would be greatly appreciated!**** > > Many thanks,**** > > Jeff**** > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > -- Pedro Tabacof, Unicamp - Eng. de Computação 08.
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
