I don't think Qiroga et al's statements are contradictory, just irritatingly vague...
I agree w Richard that the distributed vs sparse dichotomy is poorly framed and in large part a bogus dichotomy I feel the same way about the symbolic vs subsymbolic dichotomy... Many of the conceptual distinctions at the heart of standard cognitive science theory are very poorly defined, it's disappointing... -- ben G On Sat, Nov 22, 2008 at 12:03 AM, Richard Loosemore <[EMAIL PROTECTED]> wrote: > Vladimir Nesov wrote: >> >> On Sat, Nov 22, 2008 at 12:30 AM, Richard Loosemore <[EMAIL PROTECTED]> >> wrote: >>> >>> They want some kind of mixture of "sparse" and "multiply redundant" and >>> "not >>> distributed". The whole point of what we wrote was that there is no >>> consistent interpretation of what they tried to give as their conclusion. >>> If you think there is, bring it out and put it side by side with what we >>> said. >>> >> >> There is always a consistent interpretation that drops their >> interpretation altogether and leaves the data. I don't see their >> interpretation as strongly asserting anything. They are just saying >> the same thing in a different language you don't like or consider >> meaningless, but it's a question of definitions and style, not >> essence, as long as the audience of the paper doesn't get confused. >> > > Let me spell it out carefully. > > If we try to buy their suggestion that the MTL represents concepts (such as > "Jennifer Aniston") in a "sparse" manner, then this means that a fraction S > of the neurons in MTL encode Jennifer Aniston, and the fraction is small. > > Now, if the fraction S is small, then the probability of Quiroga et al > hitting some neuron inthe set, using a random probe, is also small. > > Agreed? > > Clearly, as Quiroga et al point out themselves, if the probability S is very > small, we should be surprised if that random probe actually did find a > Jennifer Aniston cell. > > So... > > To make the argument work, they have to suggest that the number of Jennifer > Aniston cells is actually a very significant percentage of the total number > of cells. In other words, "sparse" must mean "about one in every hundred > cells", or something like that (it's late, and I am tired, so I am not about > to do the math, but if Quiroga et al do about a hundred probes and *one* of > those is a JA cell, it clearly cannot be one in a million cells). > > Agreed? > > But, of that is the case, then each cell must be encoding many concepts, > because otherwise there would not be anough cells to encode more than about > a hundred concepts, would there? They admit this in the paper: "each cell > might represent more than one class of images". But there are perhaps > hundreds of thousands of different images that a given person can recognize, > so in that case, each neuron must be representing (of the order of) > thousands of images. > > The points that Harley and I made were: > > 1) In what sense is the representation "sparse" and "not distributed" if > each neuron encodes thousands of images? Roughly one percent of the neurons > in the MTL are used for each concept, and each neuron represents thousands > of other concepts: this is just as accurate a description of a > "distributed" representation, and it is a long way from anything that > resembles a "grandmother cell" situation. > > And yet, Quiroga et al give their paper the title "Invariant visual > representation by single neurons in the human brain". They say SINGLE > neurons, when what is implied is that 1% of the entire MTL (or roughly that > number) is dedicated to representing a concept like Jennifer Aniston. They > seem to want to have their cake and eat it too: they put "single neurons" > in the title, but buried in their logic is the implication that vast numbers > of neurons are redundantly coding for each concept. That is an *incoherent* > claim. > > 2) This entire discussion of the contrast between sparse and distributed > representations has about it the implication that "neurons" are a unit that > has some functional meaning, when talking about concepts. But Harley and I > described an example of a different (mor sophisticated) way to encode > concepts, in which it made no sense to talk about these particular neurons > as encoding particular concepts. The neurons were just playing the role of > dumb constituents in a larger structure, while the actual concepts were (in > essence) patterns of activation that were just passing through. > > This alternate conception of what might be going on leads us to the > conclusion that the distinction Quiroga et al make between "sparse" and > "distributed" is not necessarily meaningful at all. In our alternate > conception, the distinction is meaningless, and the conclusion that Quiroga > et al draw (that there is "an invariant, sparse and explicit code") is not > valid - it is only a coherent conclusion if we buy the idea that individual > neurons are doing some representing of concepts. > > In other words, the conclusion was incoherent in this sense also. It was > theory laden. > > > > The whole mess is summed up quite well by a statement that they make: > > > "In the ... case [of distributed representation], recognition would require > the simultaneous activation of a large number of cells and therefore we > would expect each cell to respond to many pictures with similar basic > features. This is in contrast to the sparse firing we observe, because most > MTL cells do not respond to the great majority of images seen by the > patient." > > > But the only way to make their 'sparse" interpretation work would be to have > (about) 1% of the MTL respond to one picture - a *huge* number of cells, by > anyone's standard! > > This is a contradiction. Or, as we put it, an incoherent claim. > > > > All of this was in the paper. > > Yes, the data by itself is interesting. No, the interpretation of the data > given by the authors was meaningless. > > > > > > > > Richard Loosemore > > > > > ------------------------------------------- > agi > Archives: https://www.listbox.com/member/archive/303/=now > RSS Feed: https://www.listbox.com/member/archive/rss/303/ > Modify Your Subscription: > https://www.listbox.com/member/?&; > Powered by Listbox: http://www.listbox.com > -- Ben Goertzel, PhD CEO, Novamente LLC and Biomind LLC Director of Research, SIAI [EMAIL PROTECTED] "The empires of the future are the empires of the mind." -- Sir Winston Churchill ------------------------------------------- agi Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: https://www.listbox.com/member/?member_id=8660244&id_secret=120640061-aded06 Powered by Listbox: http://www.listbox.com
