Richard, Let's just bury the hatchet. I am too busy right now to spend any more time on this.
Edward W. Porter Porter & Associates 24 String Bridge S12 Exeter, NH 03833 (617) 494-1722 Fax (617) 494-1822 [EMAIL PROTECTED] -----Original Message----- From: Richard Loosemore [mailto:[EMAIL PROTECTED] Sent: Tuesday, October 23, 2007 9:48 PM To: [email protected] Subject: Re: Bogus Neuroscience [WAS Re: [agi] Human memory and number of synapses] Edward W. Porter wrote: > Richard, > > I am aware of the type-token distinction, and I think the distinction > between the class of Diet Coke cans and the particular physical object > can_1 I discussed in my prior email is, in fact, an example of just such > a distinction. > > If not, please to explain to me why it is not. I will explain, as you request. The "type-token" distinction you refer to is one aspect of how types and tokens manifest themselves, but it is not the sense that is relevant to the discussion we were having ... and it is because you did not perceive the way it was relevant, that I dismissed your last post so abruptly and suggested further reading. The issue in the case of Granger's model of cognition is that there are some types of system in which the arrival of a percept (e.g. image of a coke can) causes the [coke can] node to become activated. So far so good, but what if two coke cans are seen? In these primitive types of system, two coke cans simply cause the ONE [coke can] to fire some more. How does the system represent the appearance of two things that belong to the same category? Simple answer: in Granger's system and in other simple systems of this sort, there is NO provision for representing two coke cans, because all that can happen is that the one [coke can] node fires when there is evidence for any number of coke cans in the visual field. Ideally, we would like the system to have spare nodes available (this is the Good Old fashioned AI way of doing things), so the system can assign [node_32718] to the first coke can, and [node_32719] to the second coke can, and then make links back to the generic [coke can] node (with the links probably labelled with the IS-A marker). Trouble with that way of doing things is this: well, in fact, there isn't anything particularly wrong with this way of doing things, EXCEPT that Granger's naive idea of a neural representation scheme makes absolutely no provision for temporary nodes that can be used in this way to represent instances. His system cannot do this. The neural machinery to do this, if it existed in his model of cognition, would be utterly crucial to the operation of the system, but it is not there. He leaves it out. He seems unaware that some such machinery is needed. One of the main reasons to ask about this issue of how temporary nodes are deployed to represent instances is that IF such temporary nodes exist, THEN more than likely they are not simply fixed clusters of neurons. If they were just fixed clusters, how would they be recruited, how would they be transiently connected to the generic [coke can] nodes, how could they later develop and become more permanent, or how could they be recycled for use as other temporary concepts? A whole rat's nest of issues is raised by the choice about how to represent instances. And the worst part of this rat's nest of issues is that whichever way you choose to resolve the problem, the simple idea of {one concept equals one fixed cluster of neurons} will probably have to be abandoned. Does Granger deal with this? Does he show any sign that he understands this? No. Instead, he naively assumes that clusters of neurons represent concepts like [coke can], and he talks as if the only important thing is to see that the [coke can] cluster has its activation increased when a coke can is seen (or thought about). So, to summarize. If Granger means us to take his model as it stands, then there are single, fixed clusters of neurons encoding concepts, and the system cannot represent multiple instances 9and as a model of cognition, therefore, it is Dead On Arrival). If, on the other hand, someone modifies or extends Granger's model so that it can deal with multiple instances, the modifications would have to be so extensive that there is a very high probability that everything he said in his paper would have to be thrown out, and he would have to start all over again. That is the "type/token" problem I was talking about. I was not making any reference to any of the other type/token issues that you were alluding to. If you look back at Ben Goertzel's responses to myself and Mark Waser, you will see that he concedes that the cognitive psychology component of Granger's paper is "largely BS". It is issues such as this one that, I believe, he had in mind when he said that. Ben actually went further and suggested that some of Susan Greenfield's proposals might be used to resolve the problem ... but he then adds that, of course, this was not what Granger was proposing. ***** I have gone back and re-read your previous post in which you explain in detail your thoughts about the differences between coke cans seen in different contexts. The problem (and I hope you can see this now) is that none of it makes any difference because that was not the issue I raised: I talked only about the ability of Granger's model to encode any of the distinctions you talked about. My beef with Granger is that he implicitly uses a model that cannot handle the representation of multiple instances of the same type of object. ***** Now, please believe me, but when I first wrote about how bad I thought Granger's paper was, I did not have the slightest intention of criticising you: I was not rude to you, made no personal attack on you, nor implied you were stupid. I simply launched an attack on the content of his paper. I was shocked when you came back and seemed to take it personally that I had attacked his paper, because I did not put any blame on you at all. Since then I have tried to keep emotion out of the discussion: if you read our sequence of exhanges you will see that I have been brief and to the point, and I have disagreed, but the only way you can see rudeness is if you choose to find rudeness in what was actually an attempt to deal with complex issues in a brief span of words. I am certainly annoyed at the general problem caused by papers such as Granger's, but that annoyance should not be mistaken for a personal attack on you. The only point at which I became testy was in that last message when you really did start to make personal remarks. I mean, come on: all those sarcastic references to the word "DEEP"? Those implications that I was simply being closed minded? Did I engage in such sarcasm? (When I used the emphasized word "DEEP", by the way, I was meaning "This is a heck of a big issue to get into, and I dread the thought of having to explain the whole thing in posts to this list, because then we will be here all week." That was all.) Please review the sequence if you are not convinced of this: I intended no personal rudeness, and other than a little impatience with the amount of time I was spending on the matter, I do not believe I showed any. And then, I did indeed suggest that you should do some more reading. Do you not think, in the light of the explanation I have just given above, that it would be fair to say that the real issue I was addressing (Grangers' model's inability to handle particular types of encodings) was not actually in your mind when you replied? Is it not true that you did indeed miss the point completely? And if a person misses the point in such a significant way, several posts in a row, AND starts to become angry and tell me that I should stop being so closed-minded, would it not be fair, under those circumstances, that I run out of energy to write an entire textbook to explain, and instead just say that you really should do some more reading? My suggestion about the reading was a last resort. Finally, do not drag my interactions with other people into this discussion, please. Some people here have known me for a long time, and some have axes that they like to get out and grind. As I have said before, when people mount unprovoked attacks, I tend to respond. Simple as that. If you want to know more about the reason why, email me in private. Regards, (... and nevertheless, with respect,) Richard Loosemore > With regard you P.S., I agree very much with its general thrust. I > have > normally attempted to avoid attacking people themselves. In fact, > except for issue were I feel it is important to fight hard for a > paradigm shift, such as in fighting the small machine mindset, I > normally try to be relatively tempered even in my critiques. > > BUT RICHARD HAVE YOU ANY IDEA HOW INSULTING AND DISMISSIVE THE TONE OF > MANY OF YOUR OWN POSTS HAVE BEEN -- NOT ONLY TO ME -- BUT TO OTHERS? I > would guess at least 1/3 of your many posts in this thread have either > explicitly or implicitly been more insulting than my language you > complained of below. > > For example, in the below post you implied I am too dumb to know what > the type-token distinction is and need to do some reading to understand > it, with the implication you know much more on the subject than I. This > is when my response, if you actually took the time to read it, indicates > I was not only aware of the distinction, but directly addressing it > (although, in the interest of space and time, perhaps not every possible > ramification of it). > > In sum, such an implication is quite an insult. > > I emphasized the word DEEP with implied reverb in the post before > not > because it was incorrect to label the issue as deep, in fact, I thought > it was deep, but because of the implication that it was too DEEP for > me, but not you, with our current respective levels of knowledge, to > understand. > > Are you capable of understanding how that might be considered > insulting? > > But that is nothing compared to some of the dismissive language I have > read in some of your responses to others. > > So, Richard, I was only trying to give you a hint -- so a note like > this > would not be necessary -- to treat others a little more the way you > apparently want them to treat you. > > If we could all do that, perhaps we could have a little more light and > a > little less darkness on this list. > > Ed Porter > > > -----Original Message----- > From: Richard Loosemore [_mailto:[EMAIL PROTECTED] > Sent: Monday, October 22, 2007 8:21 PM > To: [email protected] > Subject: Re: Bogus Neuroscience [WAS Re: [agi] Human memory and number > of synapses] > > > Edward W. Porter wrote: >> Richard, >> >> I will only respond to the below copied one of the questions in your >> last message because of lack of time. I pick this example because it >> was so DEEP (to be heard in your mind with max reverb). I hoped >> that if I could give a halfway reasonable answer to it and if, just >> maybe, you could open your mind (and that is one of the main issue >> in this thread), you might actually also try to think how your other >> questions could be answered. >> >> In response to this DEEP question, I ask "How do you, Richard >> Loosemore, normally distinguish different instances of a given type." > > Okay, I have to stop you right there. > > I pointed out the question of type-token distinctions because it has > been a serious issue for a long time (decades) and anyone who wants to > understand AI systems or models of cognition at all has to know what > it is and what its ramifications are. > > By saying that it is a "DEEP" issue I was inviting you to do some > reading, not to make a reverb happen inside your head. > > You can find a good summary of it in many places, but one is in the > second volume of the Parallel Distributed Processing set (McClelland > and Rumelhart, 1986), chapter 26 (a chapter by Donald Norman). > > Granger's proposal makes no mention of how to handle multiple > instances, and IMPLICITLY refers to a type of system that is known to > be incapable of handling multiple instances. > > > Richard Loosemore > > P.S. Why is it necessary to personalize this issue by comments such > as "... if, just maybe, you could open your mind (and that is one of > the main issue in this thread)..."? > > > > > > >> By distinguishing characteristics? (This would include things like >> little dings on your car or the junk in its back seat that >> distinquish it from a similar make and model of the same years and >> color. ) >> >> If so, that is handled by Grangers system in the manner described >> in my response to the question copied below. >> >> Now when you are dealing with objects that have an identical >> appearance, such as Diet Coke cans (the example I normally use when >> I think of this problem), often the only thing you can distinguish >> them by is again their distinguishing characteristics. But in >> this case the distinguishing characteristics would be things like >> their location, orientation, or perhaps relationship to other >> objects. It would also include implications that can properly be >> drawn from or about such characteristics for the type of thing >> involved. >> >> For example, if you leave a Diet Coke can (can_1) downstairs in your >> kitchen and go up to you bedroom and see an identical looking coke >> can next to your bed, you would normally assume the can next to your >> bed was not can_1, unless you had some explanation for how can_1 was moved next >> to your bed. (For purposes of dealing with the hardest part of the >> problem we will assume all coke cans have been opened and have the >> same amount of coke with roughly the same level of carbonation.) If >> you go back down stairs and see a Diet Coke can exactly where you >> left can_1, you will assume it is can_1, itself, barring some reason >> to believe the can might have been replaced with another, such as if >> you know someone was in your kitchen during your absence. >> >> All these types of inferences are based on generalities, often >> important broad generalities like the persistence of objects, that >> take the learning of even more basic or more primiative generalities >> (such as those needed for object recognition, understanding the >> concept of physical objects, the ability to see similarities and >> dissimilarities between objects, and spatial and temporal models), >> all of which take millions of trillions of machine opps and weeks or >> months of experience to learn. So I hope you will forgive me and >> Granger if we dont explain them in detail. (Goertzel in "Hidden >> Pattern", I think it is, actually gives an example of how an AGI >> could learn object persistence.) >> >> However, the whole notion of AGI is built on the premise that such >> things can be learned by a machine architecture having certain >> generalized capabilities and having something like the physical world >> to interact in and with. Those of us who are bullish on AGI think >> we already have a pretty good ideas how to make system that can have >> the required capabilities to learn such broad generalities, or at >> least get us much closer to such a system, so we can get a much >> better understanding of what more is needed, and then try to add it. >> >> With such ideas of how to make an AGI, it become much easier to map >> the various aspects of it into known, or hypothesized, operations in >> the brain. The features described in Grangers paper, when combined >> with other previous ideas on how the brain could function as an AGI, >> would seem to describe a system having roughly the general >> capability to learn and properly inference from all of the basic >> generalizations of the type I described above, such as the >> persistence of objects, and what types of objects move on their own, >> and with what probabilities under what circumstances. For example, >> Granger's article explains how to learn patterns, generalizations of >> pattersn, patterns of generalizations of patterns, and with >> something like a hippocampus it could learn episodes, and then >> patterns from episodes, and generalizations from patterns from >> episodes, and patterns of generalazations from episodes, etc. >> >> Yes, the Granger article, itself, does not describe all of the >> features necessary for the brain to act as a general AGI, but when >> interpreted in the context of enlightened AGI models, such as >> Novamente, and the current knowledge and leading hypotheses in brain >> science, it is easy to imagine how what he describes could play a >> very important role in solving even mental problems as DEEP (again >> with reverb) as that of determining whether the Diet Coke can on the >> table is the one you have been drinking from, or someone elses. >> >> Has there been a little hand waving in the above explanation? Yes, >> but if you have a good understanding of AGI and its brain >> equivalent, you will understand the amount of hand waving is >> actually rather limited. >> >> Ed Porter >> >> >> ============= from prior post ==================== >> >> > RICHARD>> How does it cope with the instance/generic >> > distinction? >> > >> > I assume after the most general cluster, or the cluster >> > having the most activation from the current feature set, >> > spreads its activation through the matrix loop, then the >> > cluster most activated by the remaining features spreads >> > activation through the matrix loop. This sequence can >> > continue to presumably any desired level of detail supported >> > by the current set of observed, remembered, or imagined >> > features to be communicated in the brain. The added detail >> > from such a sequence of descriptions would distinguish an >> > instance from a generic description reprsented by just one >> > such description.. >> >> A misunnderstanding: the question is how it can represent multiple >> copies of a concept that occur in a situation without getting >> confused about which is which. If the appearance of one chair in a >> scene causes the [chair] neuron (or neurons, if they are a cluster) >> to fire, then what happens when you walk into a chair factory? What >> happens when you try to understand a sentence in which there are >> several >> nouns: does the [noun] node fire more than before, and if it does, >> how does this help you parse the sentence? >> >> This is a DEEP issue: you cannot just say that this will be handled >> by other neural machinery on top of the basic (neural-cluster = >> representation of generic thing) idea, because that "other machinery >> is nontrivial, and potentially it will require the original >> (neural-cluster = representation of generic thing) idea to be >> abandoned completely. >> >> -----Original Message----- >> From: Richard Loosemore [_mailto:[EMAIL PROTECTED] >> Sent: Monday, October 22, 2007 2:55 PM >> To: [email protected] >> Subject: Re: Bogus Neuroscience [WAS Re: [agi] Human memory and >> number of synapses] >> >> >> Edward W. Porter wrote: >> > Dear Readers of the RE: Bogus Neuroscience Thread, >> > >> > Because I am the one responsible for bringing to the attention of >> > this list the Granger article (Engines of the brain: The >> > computational instruction set of human cognition, by Richard >> > Granger) that has caused the recent kerfuffle, this morning I >> > took the time to do a reasonably careful re-read of it. >> > >> > [snip] >> > >> > In his Sun 10/21/2007 2:12 PM post Richard Loosemore cited failure >> > to answer the following questions as indications of the papers >> worthlessness. >> > >> > RICHARD>> How does it cope with the instance/generic >> > distinction? >> > >> > I assume after the most general cluster, or the cluster >> > having the most activation from the current feature set, >> > spreads its activation through the matrix loop, then the >> > cluster most activated by the remaining features spreads >> > activation through the matrix loop. This sequence can >> > continue to presumably any desired level of detail supported >> > by the current set of observed, remembered, or imagined >> > features to be communicated in the brain. The added detail >> > from such a sequence of descriptions would distinguish an >> > instance from a generic description reprsented by just one >> > such description.. >> >> A misunnderstanding: the question is how it can represent multiple >> copies of a concept that occur in a situation without getting >> confused about which is which. If the appearance of one chair in a >> scene causes the [chair] neuron (or neurons, if they are a cluster) >> to fire, then what happens when you walk into a chair factory? What >> happens when you try to understand a sentence in which there are >> several >> nouns: does the [noun] node fire more than before, and if it does, >> how does this help you parse the sentence? >> >> This is a DEEP issue: you cannot just say that this will be handled >> by other neural machinery on top of the basic (neural-cluster = >> representation of generic thing) idea, because that "other machinery >> is nontrivial, and potentially it will require the original >> (neural-cluster = representation of generic thing) idea to be >> abandoned completely. >> >> > >> > RICHARD>> How does it allow top-down processes to operate in the >> > recognition process? >> > >> > I dont think there was anything said about this, but the >> > need for, and presence in the brain of, both top-down and >> > bottom-up processes is so well know as to have properly been >> > assumed. >> >> Granted, but in a system in which the final state is determined by >> expectations as well as by incoming input, the dynamics of the system >> are potentially completely different, and all of Granger's assertions >> about the roles played by various neural structures may have to be >> completely abandoned in order to make allowance for that new dynamic. >> >> >> > RICHARD>> How are relationships between instances encoded? >> > >> > I assume the readers will understand how it handles temporal >> > relationships (if you add the time dilation and compression >> > mentioned above). Spatial relationships would come from the >> > topology of V1 (but sensed spatial relationships can also be >> > build via a kohonen net SOM with temporal difference of >> > activiation time as the SOMs similarity metric). >> > Similarly, other higher order relationships can be built >> > from patterns in the space of hierarchical gen/comp pats >> > networks derived from inputs in these two basic dimensions >> > of space and time plus in the dimensions defined by other >> > sensory, emotional, and motor inputs. [I consider motor >> > outputs as a type of input]. >> >> Again, no: relationships are extremely dynamic: any two concepts >> can be linked by a relationship at any moment, so the specific >> question is, if "things" are represented as clusters of neurons, how >> does the system set up a temporary connection between those >> clusters, given that there is not, in general, a direct link between >> any two neurons > in the brain? >> You cannot simply "strengthen" the link between your "artichoke" >> neuron and your "basilisk" neuron in order to form the relationship >> caused by my mention of both of them in the same sentence, because, >> in general, there may not be any axons going from one to the other. >> >> >> > RICHARD>> How are relationships abstracted? >> > >> > By shared features. He addresses how clusters tend to form >> > automatically. These clusters are abstractions. >> >> These are only clusters of "things". He has to address this issue >> separately for "relationships" which are connections or links between >> things. The question is about "types" of links, and about how there >> are potentially an infinite number of different types of such links: >> how are those different types represented and built and used? Again, >> a simple neural connection is not good enough, because there would >> only be one possible type of relationship in your thoughts. >> >> >> > RICHARD>> How does position-independent recognition occur? >> > >> > He deals with this. His nodes are nodes in a hierarchical >> > memory that provides degrees of position and shape >> > invariance, or the type mentioned by Hawkins and the Serre >> > paper I have cited so many times. Grangers figures 6 and 7 >> > indicates exactly this type of invariance. >> >> I have not looked in detail at this, but how does his position >> invariance scale up? For example, if I learn the new concept of >> "floo powder", do I now have to build an entire set of neural >> machinery for the all the possible positions on my retina where I >> might see "floo powder"? If the answer is yes, the mechanism is >> bankrupt, as I am sure you realise: we do not have that much neural >> machinery to dedicate to it. >> >> >> > RICHARD>> What about the main issue that usually devastates any >> > behaviorist-type proposal: patterns to be associated with other >> > patterns are first extracted from the input by some (invisible, >> > unacknowledged) preprocessor, but when the nature of this >> > preprocessor is examined carefully, it turns out that its job is >> > far, far more intelligent than the supposed association engine to >> > which it delivers its goods? >> > >> > What he feeds to his system are things like the output of >> > Gabor filters. I dont think a Gabor filter is something >> > that is far, far, more intelligent than the supposed >> > association engine to which it delivers its goods. >> >> He has to show that the system is capable, by itself, of picking up >> objects like the "letter A" in a scene without the programmer of the >> simulation giving it some hint. The fact that he uses Gabor filters >> does not bear on the issue, as far as I can see. >> >> This issue is more subtle than the others. Too much for me to go >> into in great detail, due to time constraints. Suffice it to say >> that you do not really address the issue I had in mind. >> >> >> > This is just an example of how a serious attempt to understand >> > what is good in Grangers paper, and to expand on those good >> > features, overcomes a significant number of the objections raised >> > by those whose major motivation seems to be to dismiss it. >> >> I think I have shown that none of my objections were overcome, alas. >> >> >> > Wikipedia, that font of undisputed truth, defines Cognitive >> > science as >> > >> > Cognitive science is most simply defined as the scientific >> > study either of mind or of intelligence (e.g. Luger 1994). >> > It is an interdisciplinary study drawing from relevant >> > fields including psychology, philosophy, neuroscience, >> > linguistics, anthropology, computer science, biology, and >> > physics >> > >> > Based on this definition I would say the cognitive science aspect >> > of >> > Grangers paper, although speculative and far from fully fleshed out, >> > is actually quite good. >> >> Cognitive science is more than just saying a few things that seem to >> come from a selction of these fields. >> >> I would welcome further discussion of these issues, but it might be >> better for me to point to some references in which they are discussed >> properly, rather than for me to try to do the whole job here. ----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?&; ----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=57537833-231d2f
