Vladimir Nesov wrote:
On Sat, Nov 22, 2008 at 12:30 AM, Richard Loosemore <[EMAIL PROTECTED]> wrote:
They want some kind of mixture of "sparse" and "multiply redundant" and "not
distributed". The whole point of what we wrote was that there is no
consistent interpretation of what they tried to give as their conclusion.
If you think there is, bring it out and put it side by side with what we
said.
There is always a consistent interpretation that drops their
interpretation altogether and leaves the data. I don't see their
interpretation as strongly asserting anything. They are just saying
the same thing in a different language you don't like or consider
meaningless, but it's a question of definitions and style, not
essence, as long as the audience of the paper doesn't get confused.
Let me spell it out carefully.
If we try to buy their suggestion that the MTL represents concepts (such
as "Jennifer Aniston") in a "sparse" manner, then this means that a
fraction S of the neurons in MTL encode Jennifer Aniston, and the
fraction is small.
Now, if the fraction S is small, then the probability of Quiroga et al
hitting some neuron inthe set, using a random probe, is also small.
Agreed?
Clearly, as Quiroga et al point out themselves, if the probability S is
very small, we should be surprised if that random probe actually did
find a Jennifer Aniston cell.
So...
To make the argument work, they have to suggest that the number of
Jennifer Aniston cells is actually a very significant percentage of the
total number of cells. In other words, "sparse" must mean "about one in
every hundred cells", or something like that (it's late, and I am tired,
so I am not about to do the math, but if Quiroga et al do about a
hundred probes and *one* of those is a JA cell, it clearly cannot be one
in a million cells).
Agreed?
But, of that is the case, then each cell must be encoding many concepts,
because otherwise there would not be anough cells to encode more than
about a hundred concepts, would there? They admit this in the paper:
"each cell might represent more than one class of images". But there
are perhaps hundreds of thousands of different images that a given
person can recognize, so in that case, each neuron must be representing
(of the order of) thousands of images.
The points that Harley and I made were:
1) In what sense is the representation "sparse" and "not distributed" if
each neuron encodes thousands of images? Roughly one percent of the
neurons in the MTL are used for each concept, and each neuron represents
thousands of other concepts: this is just as accurate a description of
a "distributed" representation, and it is a long way from anything that
resembles a "grandmother cell" situation.
And yet, Quiroga et al give their paper the title "Invariant visual
representation by single neurons in the human brain". They say SINGLE
neurons, when what is implied is that 1% of the entire MTL (or roughly
that number) is dedicated to representing a concept like Jennifer
Aniston. They seem to want to have their cake and eat it too: they put
"single neurons" in the title, but buried in their logic is the
implication that vast numbers of neurons are redundantly coding for each
concept. That is an *incoherent* claim.
2) This entire discussion of the contrast between sparse and distributed
representations has about it the implication that "neurons" are a unit
that has some functional meaning, when talking about concepts. But
Harley and I described an example of a different (mor sophisticated) way
to encode concepts, in which it made no sense to talk about these
particular neurons as encoding particular concepts. The neurons were
just playing the role of dumb constituents in a larger structure, while
the actual concepts were (in essence) patterns of activation that were
just passing through.
This alternate conception of what might be going on leads us to the
conclusion that the distinction Quiroga et al make between "sparse" and
"distributed" is not necessarily meaningful at all. In our alternate
conception, the distinction is meaningless, and the conclusion that
Quiroga et al draw (that there is "an invariant, sparse and explicit
code") is not valid - it is only a coherent conclusion if we buy the
idea that individual neurons are doing some representing of concepts.
In other words, the conclusion was incoherent in this sense also. It
was theory laden.
The whole mess is summed up quite well by a statement that they make:
"In the ... case [of distributed representation], recognition would
require the simultaneous activation of a large number of cells and
therefore we would expect each cell to respond to many pictures with
similar basic features. This is in contrast to the sparse firing we
observe, because most MTL cells do not respond to the great majority of
images seen by the patient."
But the only way to make their 'sparse" interpretation work would be to
have (about) 1% of the MTL respond to one picture - a *huge* number of
cells, by anyone's standard!
This is a contradiction. Or, as we put it, an incoherent claim.
All of this was in the paper.
Yes, the data by itself is interesting. No, the interpretation of the
data given by the authors was meaningless.
Richard Loosemore
-------------------------------------------
agi
Archives: https://www.listbox.com/member/archive/303/=now
RSS Feed: https://www.listbox.com/member/archive/rss/303/
Modify Your Subscription:
https://www.listbox.com/member/?member_id=8660244&id_secret=120640061-aded06
Powered by Listbox: http://www.listbox.com
