Yay! Well done. I thought it had to be in the input for some reason.
Ian
On 5 Jun 2014 14:54, "Jim Bridgewater" <[email protected]> wrote:
> It turns out that PIL's convert method uses dithering by default.
> Turning this off as seen in the code below removes the unexpected
> white pixels which can be seen in the attached document.
>
>
> def _convertToVector(self, image):
> '''
> Returns a bit vector representation (list of ints) of a PIL image.
> '''
> # Convert the image to black and white
> image = image.convert('1',dither=Image.NONE)
> # Pull out the data, turn that into a list, then a numpy array,
> # then convert from 0 255 space to binary with a threshold.
> # Finnally cast the values into a type CPP likes
> vector = (numpy.array(list(image.getdata())) < 100).astype('uint32')
>
> return vector
>
> On Mon, Jun 2, 2014 at 5:41 PM, Jim Bridgewater <[email protected]>
> wrote:
> > Hi Ian,
> >
> > That sounded like a good theory, but I tried 0, 1, 10, 150, 200, 250,
> > 255, and 256 and the "sprinkle" of white pixels is always there except
> > for the trivial cases of 0 and 256. See the attached document.
> >
> > I find it interesting that the affected inputs are always the same for
> > a given image no matter what column it is in. I assume the initial
> > permanences are different for each column. I'm in the process of
> > going through the SP python code to figure out what's going on which
> > is a useful exercise for me in any case. Plus I'm finding some
> > spelling mistakes.
> >
> > Let me know if you have any other ideas.
> >
> >
> > On Mon, Jun 2, 2014 at 8:09 AM, Ian Danforth <[email protected]>
> wrote:
> >> Jim,
> >>
> >> I think this is an artifact of the demo. The initial images are
> grayscale,
> >> to get a binary input I had to put a threshold on each pixel value. This
> >> happens in sp_viewer.py in:
> >>
> >> def _convertToVector(self, image):
> >> '''
> >> Returns a bit vector representation (list of ints) of a PIL image.
> >> '''
> >> # Convert the image to black and white
> >> image = image.convert('1')
> >> # Pull out the data, turn that into a list, then a numpy array,
> >> # then convert from 0 255 space to binary with a threshold.
> >> # Finnally cast the values into a type CPP likes
> >> vector = (numpy.array(list(image.getdata())) < 100).astype('uint32')
> >>
> >> return vector
> >>
> >>
> >> Change that 100 to something else and see what happens.
> >>
> >> Ian
> >>
> >>
> >> On Sun, Jun 1, 2014 at 4:43 PM, Jim Bridgewater <[email protected]>
> wrote:
> >>>
> >>> I tried a different set of images and interestingly the spatial pooler
> >>> groups several of the input patterns even when it is not necessary
> >>> because there are plenty of unused columns available. Increasing the
> >>> amount that permanences are decreased when a synapse does not
> >>> contribute to column activation got each column to converge on a
> >>> representation for a single input image as long as there were enough
> >>> columns available. This makes sense and is shown in the attached
> >>> document.
> >>>
> >>> However, I don't understand why, in this very simple example, the
> >>> permanences do not come to form perfect representations of the input
> >>> images.
> >>>
> >>> For these images the representations do not fill in completely, there
> >>> are synapses that are active when the column is active, but never
> >>> become connected.
> >>>
> >>> Anyone know why this happens?
> >>>
> >>> On Fri, May 30, 2014 at 1:51 AM, <[email protected]>
> wrote:
> >>> > Send nupic mailing list submissions to
> >>> > [email protected]
> >>> >
> >>> > To subscribe or unsubscribe, visit
> >>> >
> >>> > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
> >>> > or, via email, send a message with subject or body 'help' to
> >>> > [email protected]
> >>> >
> >>> > You can reach the person managing the list at
> >>> > [email protected]
> >>> >
> >>> > When replying, please edit your Subject line so it is more specific
> >>> > than "Re: Contents of nupic digest..."
> >>> >
> >>> >
> >>> > Today's Topics:
> >>> >
> >>> > 1. Re: Some observations from Ian's Fall 2013 SP demo (Ian
> Danforth)
> >>> > 2. Re: Some observations from Ian's Fall 2013 SP demo
> >>> > (Jim Bridgewater)
> >>> >
> >>> >
> >>> >
> ----------------------------------------------------------------------
> >>> >
> >>> > Message: 1
> >>> > Date: Thu, 29 May 2014 20:27:51 -0700
> >>> > From: Ian Danforth <[email protected]>
> >>> > To: "NuPIC general mailing list." <[email protected]>
> >>> > Subject: Re: [nupic-discuss] Some observations from Ian's Fall 2013
> SP
> >>> > demo
> >>> > Message-ID:
> >>> >
> >>> > <CAOajdBpF8XTt0=wkyj47q2b2b0a3hfzmv+3cjelacjc78mf...@mail.gmail.com>
> >>> > Content-Type: text/plain; charset="utf-8"
> >>> >
> >>> > Jim,
> >>> >
> >>> > Thanks for writing this up and sharing it with the list! This is
> both
> >>> > expected and good behavior! Let me see if I can shift your
> intuitions a
> >>> > bit
> >>> > so you'll agree with that :)
> >>> >
> >>> > To start out, what is the SP good for? It does two interesting
> things,
> >>> > it
> >>> > each column memorizes things it's seen and together they perform
> >>> > dimensionality reduction.
> >>> >
> >>> > Each column is a bit like a film negative, the more times it's
> exposed
> >>> > to a
> >>> > pattern the more it will resemble that pattern. If you have a double
> >>> > exposure, to multiple patterns, it might come to represent both, or a
> >>> > bit
> >>> > of each. If you show one column the same pattern again and again it
> will
> >>> > form a very strong representation of it. In my demo there are so few
> >>> > patterns each column can come to perfectly represent the input it's
> >>> > shown.
> >>> > If you force columns to become active in the presence of more than
> one
> >>> > of
> >>> > those patterns it will become a representation of a blend of the
> two. If
> >>> > you make a single column see all the patterns it will blend them all!
> >>> > Just
> >>> > like a negative exposed again and again to several scenes. Each
> column
> >>> > is a
> >>> > powerful feature learning system in it's own right, and when they are
> >>> > allowed to compete, they naturally sort themselves into groups of
> good
> >>> > representations. By forcing them to come on when they would normally
> be
> >>> > out-competed, they blend into the all-purpose representations you
> saw.
> >>> >
> >>> > But why doesn't the SP naturally take advantage of the combinatorial
> >>> > nature
> >>> > of having multiple columns on? Well that property really only comes
> into
> >>> > play when you need dimensionality reduction. You need to start with a
> >>> > set
> >>> > of inputs >> than the number of columns. Then when a column is
> exposed
> >>> > over
> >>> > and over, it never sees exactly the same input, and only the common
> >>> > areas
> >>> > to all the inputs (those bits that are most similar) will be
> retained in
> >>> > the snapshot it retains. This is the pooling of similar, but not
> exactly
> >>> > similar, features into the activity of each column. Once you have
> these
> >>> > set
> >>> > of features, you *then* use the combinatorial property of having
> >>> > multiple
> >>> > columns on so that you can assemble a complete representation from a
> set
> >>> > of
> >>> > features. If four columns represent vertical and horizontal lines,
> you
> >>> > might find the combination of all four representing a box. It is the
> >>> > ability to compose complex representations out of a much smaller set
> of
> >>> > simpler representations where the combinatorial numbers you're
> talking
> >>> > about come into play. Instead of taking 10,000 columns to learn to
> >>> > differentiate 10,000 things, you can get different representations
> from
> >>> > a
> >>> > much much smaller set of columns. The unique combination of features
> in
> >>> > each input providing the eventual unique SDR.
> >>> >
> >>> > I hope this helps in understanding the results! A good next question
> >>> > might
> >>> > be, what is the minimum number of inputs you need before having 1
> column
> >>> > /
> >>> > input becomes unreasonable and you really *need* the property of
> >>> > dimensionality reduction?
> >>> >
> >>> > Ian
> >>> >
> >>> >
> >>> > On Thu, May 29, 2014 at 4:32 PM, Jim Bridgewater <[email protected]
> >
> >>> > wrote:
> >>> >
> >>> >> Hi everyone,
> >>> >>
> >>> >> I'm working on an optical character recognition project for the
> season
> >>> >> of nupic this summer. I've been playing around with Ian Danforth's
> >>> >> Fall 2013 hackathon demo of the spatial pooler and put together the
> >>> >> attached pdf document which shows some observations of how the
> spatial
> >>> >> pooler behaves with different numbers of active columns. Maybe some
> >>> >> people on this list have thought about this before and have some
> >>> >> comments about why the SP displays this behavior.
> >>> >>
> >>> >>
> >>> >>
> >>> >> ---------- Forwarded message ----------
> >>> >> From: Scott Purdy <[email protected]>
> >>> >> Date: Thu, May 29, 2014 at 4:02 PM
> >>> >> Subject: Re: Update
> >>> >> To: Jim Bridgewater <[email protected]>
> >>> >>
> >>> >>
> >>> >> The write up is quite nice, thanks for putting that together. You
> >>> >> might send it to the discuss list in case anyone else is curious. I
> >>> >> don't think the results are particularly suprising. In particular,
> the
> >>> >> SP is known to not always perform well when the number of input bits
> >>> >> or columns is small or there are a small number of input patterns. I
> >>> >> will try to discuss with Subutai tomorrow to confirm that there is
> >>> >> nothing wrong with your experimental method and see if I can get
> more
> >>> >> details on the onion charts.
> >>> >>
> >>> >> I think the onion charts will make more sense on a more
> sophisticated
> >>> >> data set as it would likely suffer from the same issues with small
> >>> >> numbers of inputs.
> >>> >>
> >>> >>
> >>> >> On Thu, May 29, 2014 at 1:50 PM, Jim Bridgewater <
> [email protected]>
> >>> >> wrote:
> >>> >> >
> >>> >> > Hi Scott,
> >>> >> >
> >>> >> > I've been playing with Ian's demo a bit to get a feel for how the
> >>> >> > spatial pooler works. I suppose it was intended to be used with
> only
> >>> >> > a small % of active columns, but I do find it interesting how
> poorly
> >>> >> > it performs when 50% of the columns are active since it has the
> >>> >> > greatest number of output values in this case. See attached pdf.
> >>> >> >
> >>> >> > I want to work on a way of showing the results that makes it
> easier
> >>> >> > to
> >>> >> > see how the representation of each input image changes as the sp
> is
> >>> >> > trained and whether the outputs are different for each unique
> input.
> >>> >> > I was thinking about converting the sp output to a hexadecimal
> value
> >>> >> > and plotting that vs time for each input image.
> >>> >> >
> >>> >> > Maybe you guys could send me some examples of the onion graphs
> >>> >> > Subutai
> >>> >> > mentioned so I can see how they work.
> >>> >> >
> >>> >> > --
> >>> >> > Jim Bridgewater, PhD
> >>> >> > Arizona State University
> >>> >> > 480-227-9592
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>> >> --
> >>> >> Jim Bridgewater, PhD
> >>> >> Arizona State University
> >>> >> 480-227-9592
> >>> >>
> >>> >> _______________________________________________
> >>> >> nupic mailing list
> >>> >> [email protected]
> >>> >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
> >>> >>
> >>> >>
> >>> > -------------- next part --------------
> >>> > An HTML attachment was scrubbed...
> >>> > URL:
> >>> > <
> http://lists.numenta.org/pipermail/nupic_lists.numenta.org/attachments/20140529/030f10df/attachment-0001.html
> >
> >>> >
> >>> > ------------------------------
> >>> >
> >>> > Message: 2
> >>> > Date: Fri, 30 May 2014 01:51:05 -0700
> >>> > From: Jim Bridgewater <[email protected]>
> >>> > To: [email protected]
> >>> > Subject: Re: [nupic-discuss] Some observations from Ian's Fall 2013
> SP
> >>> > demo
> >>> > Message-ID:
> >>> >
> >>> > <camma+tjpw4es9ovrvy0uf1+stxxtkktpuzqt7jmtzoxxsw8...@mail.gmail.com>
> >>> > Content-Type: text/plain; charset="utf-8"
> >>> >
> >>> > I realized that I was thinking about the number of possible output
> >>> > values incorrectly. An SP that has 16 columns and is configured to
> >>> > have 8 active columns technically has over 12,000 possible output
> >>> > values, but only 2 of those output values have no active columns in
> >>> > common so an SP configured this way can only distinguish between 2
> >>> > unrelated input patterns. In the tests I ran the two patterns are
> >>> > blank and non-blank. I have updated the attached document to reflect
> >>> > this realization.
> >>> >
> >>> > On Thu, May 29, 2014 at 4:32 PM, Jim Bridgewater <[email protected]
> >
> >>> > wrote:
> >>> >> Hi everyone,
> >>> >>
> >>> >> I'm working on an optical character recognition project for the
> season
> >>> >> of nupic this summer. I've been playing around with Ian Danforth's
> >>> >> Fall 2013 hackathon demo of the spatial pooler and put together the
> >>> >> attached pdf document which shows some observations of how the
> spatial
> >>> >> pooler behaves with different numbers of active columns. Maybe some
> >>> >> people on this list have thought about this before and have some
> >>> >> comments about why the SP displays this behavior.
> >>> >>
> >>> >>
> >>> >>
> >>> >> ---------- Forwarded message ----------
> >>> >> From: Scott Purdy <[email protected]>
> >>> >> Date: Thu, May 29, 2014 at 4:02 PM
> >>> >> Subject: Re: Update
> >>> >> To: Jim Bridgewater <[email protected]>
> >>> >>
> >>> >>
> >>> >> The write up is quite nice, thanks for putting that together. You
> >>> >> might send it to the discuss list in case anyone else is curious. I
> >>> >> don't think the results are particularly suprising. In particular,
> the
> >>> >> SP is known to not always perform well when the number of input bits
> >>> >> or columns is small or there are a small number of input patterns. I
> >>> >> will try to discuss with Subutai tomorrow to confirm that there is
> >>> >> nothing wrong with your experimental method and see if I can get
> more
> >>> >> details on the onion charts.
> >>> >>
> >>> >> I think the onion charts will make more sense on a more
> sophisticated
> >>> >> data set as it would likely suffer from the same issues with small
> >>> >> numbers of inputs.
> >>> >>
> >>> >>
> >>> >> On Thu, May 29, 2014 at 1:50 PM, Jim Bridgewater <
> [email protected]>
> >>> >> wrote:
> >>> >>>
> >>> >>> Hi Scott,
> >>> >>>
> >>> >>> I've been playing with Ian's demo a bit to get a feel for how the
> >>> >>> spatial pooler works. I suppose it was intended to be used with
> only
> >>> >>> a small % of active columns, but I do find it interesting how
> poorly
> >>> >>> it performs when 50% of the columns are active since it has the
> >>> >>> greatest number of output values in this case. See attached pdf.
> >>> >>>
> >>> >>> I want to work on a way of showing the results that makes it
> easier to
> >>> >>> see how the representation of each input image changes as the sp is
> >>> >>> trained and whether the outputs are different for each unique
> input.
> >>> >>> I was thinking about converting the sp output to a hexadecimal
> value
> >>> >>> and plotting that vs time for each input image.
> >>> >>>
> >>> >>> Maybe you guys could send me some examples of the onion graphs
> Subutai
> >>> >>> mentioned so I can see how they work.
> >>> >>>
> >>> >>> --
> >>> >>> Jim Bridgewater, PhD
> >>> >>> Arizona State University
> >>> >>> 480-227-9592
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>> >> --
> >>> >> Jim Bridgewater, PhD
> >>> >> Arizona State University
> >>> >> 480-227-9592
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > Jim Bridgewater, PhD
> >>> > Arizona State University
> >>> > 480-227-9592
> >>> > -------------- next part --------------
> >>> > A non-text attachment was scrubbed...
> >>> > Name: update.pdf
> >>> > Type: application/pdf
> >>> > Size: 274713 bytes
> >>> > Desc: not available
> >>> > URL:
> >>> > <
> http://lists.numenta.org/pipermail/nupic_lists.numenta.org/attachments/20140530/84824396/attachment.pdf
> >
> >>> >
> >>> > ------------------------------
> >>> >
> >>> > Subject: Digest Footer
> >>> >
> >>> > _______________________________________________
> >>> > nupic mailing list
> >>> > [email protected]
> >>> > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
> >>> >
> >>> >
> >>> > ------------------------------
> >>> >
> >>> > End of nupic Digest, Vol 13, Issue 45
> >>> > *************************************
> >>>
> >>>
> >>>
> >>> --
> >>> Jim Bridgewater, PhD
> >>> Arizona State University
> >>> 480-227-9592
> >>>
> >>> _______________________________________________
> >>> nupic mailing list
> >>> [email protected]
> >>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
> >>>
> >>
> >>
> >> _______________________________________________
> >> nupic mailing list
> >> [email protected]
> >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
> >>
> >
> >
> >
> > --
> > Jim Bridgewater, PhD
> > Arizona State University
> > 480-227-9592
>
>
>
> --
> Jim Bridgewater, PhD
> Arizona State University
> 480-227-9592
>
> _______________________________________________
> nupic mailing list
> [email protected]
> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
>
>
_______________________________________________
nupic mailing list
[email protected]
http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
