Re: [agi] Understanding Natural Language
Once you have these sentences in predicate form, it becomes much easier to do some statistical matching on them, and group and classify them together to generate a set of more logical statements, and to disambiguate the simple english term you use first, into a single Term entity in the knowledge base. IE on Christmas people give presents on Thanksgiving people eat turkey on a pizza there is pepperoni on a pizza there is olives These will start to form patterns for the different uses, that we can use to formalize general rules or logics (Not necessarily FOL) like on(Holiday, people do things) on(food, are other foods) James Philip Goetz [EMAIL PROTECTED] wrote: Oops - looking back at my earlier post, I said that English sentences translate neatly into predicate logic statements. I should have left out logic. I like using predicates to organize sentences. I made that post because Josh was pointing out some of the problems with logic, but then making the false conclusion that predications are a bad representation. I wanted to say that you can use a predicate representation, but use something other than FOPL to process it. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303 ___ James Ratcliff - http://falazar.com New Torrent Site, Has TV and Movie Downloads! http://www.falazar.com/projects/Torrents/tvtorrents_show.php - Everyone is raving about the all-new Yahoo! Mail beta. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Understanding Natural Language
On Tuesday 28 November 2006 17:50, Philip Goetz wrote: I see that a raster is a vector. I see that you can have rasters at different resolutions. I don't see what you mean by map the regions that represent the same face between higher and lower-dimensional spaces, or what you are taking the limit of as resolution goes to infinity, or why you don't just stick with one particular resolution. Take rasters representing the faces of Tom and Dick. Just for concreteness let's assume they're 16K numbers long. Each one represents a point in a 16K-dimensional space. If we think of all the points in the space that are pictures of Tom, they form a (probably connected) region in the space. All the pictures that represent Dick form a similar shape, offset from the Tom region. There's a larger region that contains both of them that is the union of all men's faces, and a larger one yet that's all human faces, and so forth. Now imagine the picture of Tom being shrunk until it's only 1/4 of the original raster. It's still Tom's face so it's still part of the Tom region of the 16K-D space, but 12K of those dimensions can change however they like and not affect the Tom-ness of the picture. A 4K-D slice across this subspace would look like the original space would if it only had 4K dimensions in the first place -- and it will also resemble a diagonal slice across the big part of the Tom region. There will be many occurances of the smaller subregions, corresponding to all different sizes and positions of Tom's face in the raster. In other words, the Tom's face region is fractal. So, of course, is the Dick's face region, but note that at they lower limits of resolution they begin to overlap; after a while you're just able to recognize a human face but not say whose. So even if you started out saying only 16K-D space, fixed resolution, you wind up having to work with the other dimensionalities anyway. Cheers, --Josh - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Understanding Natural Language
On 11/28/06, Matt Mahoney [EMAIL PROTECTED] wrote: First order logic (FOL) is good for expressing simple facts like all birds have wings or no bird has hair, but not for statements like most birds can fly. To do that you have to at least extend it with fuzzy logic (probability and confidence). Quantification is a logic problem. I am not talking about logic, but using predications for representation. I can represent most birds can fly as something like [S [NP (mod most) (head birds)] [VP (mod can) (head fly)]] No quantification involved. A second problem is, how do you ground the terms? If you have for all X, bird(X) = has(X, wings), where does bird, wings, has get their meanings? The terms do not map 1-1 to English words, even though we may use the same notation. They DO map 1-1 to English words, because I simply use the English words in my predicates. I don't care if wing has multiple meanings. It is the business of whatever process works with those predicates to sort that out. I've said this twice already: I am not talking about using logic, in which you assign semantics to the terms, and assume that every instance of a particular predicate has the same semantics. I am just talking about using predicates to organize the English terms in a sentence. Predicates are a nice representation, even if you are not going to use FOPL. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Understanding Natural Language
Oops - looking back at my earlier post, I said that English sentences translate neatly into predicate logic statements. I should have left out logic. I like using predicates to organize sentences. I made that post because Josh was pointing out some of the problems with logic, but then making the false conclusion that predications are a bad representation. I wanted to say that you can use a predicate representation, but use something other than FOPL to process it. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Understanding Natural Language
On Wednesday 29 November 2006 13:56, Matt Mahoney wrote: How is a raster scan (16K vector) of an image useful? The difference between two images of faces is the RMS of the differences of the images obtained by subtracting pixels. Given an image of Tom, how do you compute the set of all images that look like Tom? Humans perceive images by reducing them to a small set of complex features, which can be compared in a space with much fewer dimensions. Certainly. The n-space representation at that early stage is primarily an abstraction for thinking about the process mathematically. In humans, the retina starts with more like a 16M raster but reduces it precipitously before even sending it down the optic nerve. On the other hand, visual cortex has not one but many 2-d maps where various functions of images are manipulated in a very straightforward way -- the raster form is still used quite a bit before the higher abstractions are generated. One way of computing the set of images that look like Tom would be to follow from your one image along a trajectory that you knew maintained invariants in pose, orientation, lighting, etc, until you hit the hypersurface that was the images of Dick, and then shift the whole Dick region the opposite offset. I doubt it actually happens at this level in vision, but I bet the basic mechanism, at higher levels, is *extremely* common in carrying expectations from an experience to a new situation. --Josh - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Understanding Natural Language
On Wednesday 29 November 2006 16:04, Philip Goetz wrote: On 11/29/06, J. Storrs Hall, PhD. [EMAIL PROTECTED] wrote: There will be many occurances of the smaller subregions, corresponding to all different sizes and positions of Tom's face in the raster. In other words, the Tom's face region is fractal. Are you saying that a hierarchy of categories is just a linear chain of resolutions? A *linear* chain of resolutions would be just one root-to-leaf path in an abstraction tree (root=lo-res, leaves = all the hi-res pix that would map into that lo-res one). The whole tree would be a hierarchy of categories. I don't see why you need to work with multiple dimensionalities - at least, when identifying Tom's face, you need only deal with one dimensionality, although you might use fewer dimensions when looking for any old human face. At the raster level, you can brighten or dim any one pixel without substantially changing whose face it is. At higher levels of abstraction you can move the vector along dimensions of lighting, orientation, and size without changing whose face it is. These invariants can be captured by transformations or projections in the space -- they're the kind of regularity that I'm trying to capture implicitly by using n-spaces, rather than having to represent explicitly in pointer-and-tag record structures. --Josh - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Understanding Natural Language
On 11/29/06, J. Storrs Hall, PhD. [EMAIL PROTECTED] wrote: On Wednesday 29 November 2006 16:04, Philip Goetz wrote: On 11/29/06, J. Storrs Hall, PhD. [EMAIL PROTECTED] wrote: There will be many occurances of the smaller subregions, corresponding to all different sizes and positions of Tom's face in the raster. In other words, the Tom's face region is fractal. Are you saying that a hierarchy of categories is just a linear chain of resolutions? A *linear* chain of resolutions would be just one root-to-leaf path in an abstraction tree (root=lo-res, leaves = all the hi-res pix that would map into that lo-res one). The whole tree would be a hierarchy of categories. I meant that a linear chain of resolutions would create a tree, because at finer resolutions, you would have more categories. At the raster level, you can brighten or dim any one pixel without substantially changing whose face it is. At higher levels of abstraction you can move the vector along dimensions of lighting, orientation, and size without changing whose face it is. These invariants can be captured by transformations or projections in the space -- they're the kind of regularity that I'm trying to capture implicitly by using n-spaces, rather than having to represent explicitly in pointer-and-tag record structures. What is a pointer-and-tag record structure, and what's it got to do with n-dim vectors? I still don't know why you talk about using different numbers of dimensions simultaneously. Seems to me that you can capture these invariants in whatever dimensionality you choose, so no need to talk about fractal representations. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Understanding Natural Language
On 11/28/06, Philip Goetz [EMAIL PROTECTED] wrote: I see evidence of dimensionality reduction by humans in the fact that adopting a viewpoint has such a strong effect on the kind of information a person is able to absorb. In conversations about politics or religion, I often find ideas that to me seem simple, that I cannot communicate to someone of a different viewpoint. We both start with the same input - some English sentences, say - but I think we compress them in different, yet internally consistent, ways. Their viewpoint is based on a compression scheme that simply compresses out what I am trying to communicate. Be careful drawing conclusions about what the brain _can_ do from what it _does_ do when talking about politics and religion - in those domains you hit that's a feature not a bug issues like self-deception, perceived social allegiance, the bozo bit etc, adaptations that explicitly disable a lot of our normal cognitive abilities. To draw conclusions about our full cognitive potential you really need to look at performance in complex but politically neutral domains. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Understanding Natural Language
On Monday 27 November 2006 10:35, Ben Goertzel wrote: Amusingly, one of my projects at the moment is to show that Novamente's economic attention allocation module can display Hopfield net type content-addressable-memory behavior on simple examples. As a preliminary step to integrating it with other aspects of Novamente cognition (reasoning, evolutionary learning, etc.) I assume everyone here is familiar with the agorics papers of Drexler and Miller: http://www.agorics.com/Library/agoricpapers.html and this one of mine: http://autogeny.org/chsmith.html which combines agoric and genetic algortihms (in a system named Charles Smith :-) Josh - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: Re: [agi] Understanding Natural Language
On 11/28/06, J. Storrs Hall, PhD. [EMAIL PROTECTED] wrote: On Monday 27 November 2006 10:35, Ben Goertzel wrote: Amusingly, one of my projects at the moment is to show that Novamente's economic attention allocation module can display Hopfield net type content-addressable-memory behavior on simple examples. As a preliminary step to integrating it with other aspects of Novamente cognition (reasoning, evolutionary learning, etc.) I assume everyone here is familiar with the agorics papers of Drexler and Miller: http://www.agorics.com/Library/agoricpapers.html and this one of mine: http://autogeny.org/chsmith.html which combines agoric and genetic algortihms (in a system named Charles Smith :-) Josh Also related is Eric Baum's work described at http://www.whatisthought.com/eric.html See Manifesto for an Evolutionary Economics of Intelligence for theory and Evolution of Cooperative Problem-Solving in an Artificial Economy (or related discussion in What Is Thought?) for a simple narrow-AI application based on the theory. What I am doing with artificial currencies within Novamente is not much like either Drexler Miller's or Baum's ideas, but was philosophically inspired by both... -- Ben - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Understanding Natural Language
On Monday 27 November 2006 10:35, Ben Goertzel wrote: ... An issue with Hopfield content-addressable memories is that their memory capability gets worse and worse as the networks get sparser and sparser. I did some experiments on this in 1997, though I never bothered to publish the results ... [General observations, not aimed at Ben in particular:] One of the reasons I'm not looking at actual Hopfield (or any other kind of NN) is that I think that a huge amount of what goes on in AI today is premature optimization. I.e. the vast majority of the technical work has more to do with taking operations that don't have intelligence and making them run fast, than with finding operations that do exhibit intelligence. My approach, admittedly unusual, is to assume I have all the processing power and memory I need, up to a generous estimate of what the brain provides (a petawords and 100 petaMACs), and then see if I can come up with operations that do what it does. If not it, would be silly to try and do the same task with a machine one to 100 thousand times smaller. There are plenty of cases where it's just a royal pain to get a Hopfield net or any other NN to do something that's blindingly simple for an ordinary program or vector equation. Ignore the implementation, think in the data representation as long as you can. When you've got that nailed, you can try for that factor of a thousand optimization... --Josh - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: Re: [agi] Understanding Natural Language
My approach, admittedly unusual, is to assume I have all the processing power and memory I need, up to a generous estimate of what the brain provides (a petawords and 100 petaMACs), and then see if I can come up with operations that do what it does. If not it, would be silly to try and do the same task with a machine one to 100 thousand times smaller. Yes, this is what we've done with Novamente as well. In fact, I am quite confident the NM architecture when fully implemented and tuned can yield powerful AGI The biggest open question in my mind is exactly how much computational resources will be required, to achieve what levels of intelligence. We have tried to make estimates of this but they're all pretty fudgy It is clear that the resource requirements won't be insane on the level of AIXI or AIXItl, but in practical terms, a factor of 100 in computational resource requirements makes a big difference... -- Ben - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Understanding Natural Language
On 11/24/06, J. Storrs Hall, PhD. [EMAIL PROTECTED] wrote: On Friday 24 November 2006 06:03, YKY (Yan King Yin) wrote: You talked mainly about how sentences require vast amounts of external knowledge to interpret, but it does not imply that those sentences cannot be represented in (predicate) logical form. Substitute bit string for predicate logic and you'll have a sentence that is just as true and not a lot less useful. I think there should be a working memory in which sentences under attention would bring up other sentences by association. For example if a person is being kicked is in working memory, that fact would bring up other facts such as being kicked causes a person to feel pain and possibly to get angry, etc. All this is orthogonal to *how* the facts are represented. Oh, I think the representation is quite important. In particular, logic lets you in for gazillions of inferences that are totally inapropos and no good way to say which is better. Logic also has the enormous disadvantage that you tend to have frozen the terms and levels of abstraction. Actual word meanings are a lot more plastic, and I'd bet internal representations are damn near fluid. The use of predicates for representation, and the use of logic for reasoning, are separate issues. I think it's pretty clear that English sentences translate neatly into predicate logic statements, and that such a transformation is likely a useful first step for any sentence-understanding process. Whether those predicates are then used to draw conclusions according to a standard logic system, or are used as inputs to a completely different process, is a different matter. The open questions are representation -- I'm leaning towards CSG in Hilbert spaces at the moment, but that may be too computationally demanding -- and how to form abstractions. Does CSG = context-sensitive grammar in this case? How would you use Hilbert spaces? - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: Re: [agi] Understanding Natural Language
On 11/27/06, Ben Goertzel [EMAIL PROTECTED] wrote: An issue with Hopfield content-addressable memories is that their memory capability gets worse and worse as the networks get sparser and sparser. I did some experiments on this in 1997, though I never bothered to publish the results ... some of them are at: http://www.goertzel.org/papers/ANNPaper.html I found just the opposite - Hopfield network memory capability gets much better as the networks get sparser, down to very low levels of sparseness. However, I was measuring performance as a function of storage space and computation. A fully-connected Hopfield network of 100 neurons has about 10,000 connections. A Hopfield network of 100 neurons that has only 10 connections per neuron performs has one-tenth as many connections, and can recall more than one-tenth as many patterns. Furthermore, if you selectively eliminate the weak connections and save the strong connections, you can make Hopfield networks very sparse that perform almost as well as the fully-connected ones. BTW, the canonical results about Hopfield network capacity in the McEliece 1987 paper are wrong - I can't find the flaw, so I don't know why they're wrong, but I know that the paper a) makes the mistake of comparing recall errors of a fixed number of bits between networks of different sizes, which means that it counts a 1-bit error in recalling a 1000-node pattern as equivalent to a 1-bit error in recalling a 10-node pattern, and b) the paper claims that recall of n-bit patterns, starting from a presented pattern that differs in n/2 bits, is quite good. This is impossible, since differing in n/2 bits means the input pattern is a RANDOM pattern wrt the target, and half of all the targets should be closer to the input pattern. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Understanding Natural Language
On 11/26/06, Pei Wang [EMAIL PROTECTED] wrote: Therefore, the problem of using an n-space representation for AGI is not its theoretical possibility (it is possible), but its practical feasibility. I have no doubt that for many limited application, n-space representation is the most natural and efficient choice. However, for a general purpose system, the situation is very different. I'm afraid for AGI we may have to need millions (if not more) dimensions, and it won't be easy to decide in advance what dimensions are necessary. I see evidence of dimensionality reduction by humans in the fact that adopting a viewpoint has such a strong effect on the kind of information a person is able to absorb. In conversations about politics or religion, I often find ideas that to me seem simple, that I cannot communicate to someone of a different viewpoint. We both start with the same input - some English sentences, say - but I think we compress them in different, yet internally consistent, ways. Their viewpoint is based on a compression scheme that simply compresses out what I am trying to communicate. It may be that psychological repression is the result of compressing out dimensions, or data, that had low utility. Someone who is repeatedly exposed to a trauma which they are unable to do anything about may calculate, subconsciously, that the awareness of that trauma is simply useless information. Trying to suggest a PCA-like dimensionality reduction of concepts by humans has the difficulty that a human should then remember, or be aware of, those implications of a sentence which have had the most variance, or the most impact, in their experiences. In fact, we often find people make the greatest compression along dimensions that have the highest importance to them, compressing a whole set of important distinctions into the binary good-evil dimension. It may be that our motivational system can handle only a small number of dimensions - say, five - and that good-evil is one of the principle components whose impact is so large we are actually aware of it. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Understanding Natural Language
Philip Goetz [EMAIL PROTECTED] wrote: The use of predicates for representation, and the use of logic for reasoning, are separate issues. I think it's pretty clear that English sentences translate neatly into predicate logic statements, and that such a transformation is likely a useful first step for any sentence-understanding process. I don't think it is clear at all. Try translating some poetry. Even for sentences that do have a clear representation in first order logic, the translation from English is not straightforward at all. It is an unsolved problem. I also dispute that it is even useful for sentence understanding. Google understands simple questions, and its model is just a bag of words. Attempts to apply parsing or reasoning to information retrieval have generally been a failure. It would help to define what sentence-understanding means. I say a computer understands English if it can correctly assign probabilities to long strings, where correct means ranked in the same order as judged by humans. So a program that recognizes the error in the string the cat caught a moose could be said to understand English. Thus, the grammar checker in Microsoft Word would have more understanding of a text document than a simple spell checker, but less understanding than most humans. Maybe you have a different definition. A reasonable definition for AI should be close to the conventional meaning and also be testable without making any assumption about the internals of the machine. Now it seems to me that you need to understand sentences before you can translate them into FOL, not the other way around. Before you can translate to FOL you have to parse the sentence, and before you can parse it you have to understand it, e.g. I ate pizza with pepperoni. I ate pizza with a fork. Using my definition of understanding, you have to recognize that ate with a fork and pizza with pepperoni rank higher than ate with pepperoni and pizza with a fork. A parser needs to know millions of rules like this. -- Matt Mahoney, [EMAIL PROTECTED] - Original Message From: Philip Goetz [EMAIL PROTECTED] To: agi@v2.listbox.com Sent: Tuesday, November 28, 2006 2:47:41 PM Subject: Re: [agi] Understanding Natural Language On 11/24/06, J. Storrs Hall, PhD. [EMAIL PROTECTED] wrote: On Friday 24 November 2006 06:03, YKY (Yan King Yin) wrote: You talked mainly about how sentences require vast amounts of external knowledge to interpret, but it does not imply that those sentences cannot be represented in (predicate) logical form. Substitute bit string for predicate logic and you'll have a sentence that is just as true and not a lot less useful. I think there should be a working memory in which sentences under attention would bring up other sentences by association. For example if a person is being kicked is in working memory, that fact would bring up other facts such as being kicked causes a person to feel pain and possibly to get angry, etc. All this is orthogonal to *how* the facts are represented. Oh, I think the representation is quite important. In particular, logic lets you in for gazillions of inferences that are totally inapropos and no good way to say which is better. Logic also has the enormous disadvantage that you tend to have frozen the terms and levels of abstraction. Actual word meanings are a lot more plastic, and I'd bet internal representations are damn near fluid. The use of predicates for representation, and the use of logic for reasoning, are separate issues. I think it's pretty clear that English sentences translate neatly into predicate logic statements, and that such a transformation is likely a useful first step for any sentence-understanding process. Whether those predicates are then used to draw conclusions according to a standard logic system, or are used as inputs to a completely different process, is a different matter. The open questions are representation -- I'm leaning towards CSG in Hilbert spaces at the moment, but that may be too computationally demanding -- and how to form abstractions. Does CSG = context-sensitive grammar in this case? How would you use Hilbert spaces? - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303 - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Understanding Natural Language
On Tuesday 28 November 2006 14:47, Philip Goetz wrote: The use of predicates for representation, and the use of logic for reasoning, are separate issues. I think it's pretty clear that English sentences translate neatly into predicate logic statements, and that such a transformation is likely a useful first step for any sentence-understanding process. Whether those predicates are then used to draw conclusions according to a standard logic system, or are used as inputs to a completely different process, is a different matter. I would beg to differ. While it is clearly straightforward to translate a sencence into a predicate expression in a syntactic way, the resulting structure has no coherent semantics. Consider how much harder it is to translate a sentence of English into a sentence of Chinese. Even then you won't have uncovered the meat of the semantics, since in both languages you can rely on a lot of knowledge the hearer already knows. But when you put the sentence into predicate form, you've moved into a formalism where there is no such semantics behind the representation. In order to provide them, you have to do the equivalent of writing a Prolog program that could make the same predictions, explanations, or replies that a human speaker could to the original English sentence. Consider the following sentences. Could you translate them all using the single predicate on(A,B)? If not, the translation gets messier: On the table is an apple. On Lake Ontario is Toronto. On Hadamard's theory transubstantiation is ineffable. On Comet, on Cupid, on Prancer and Vixen. On Christmas we open presents. On time is better than late. On budget expenditures are dwarfed by Social Security. On and on the list goes... The open questions are representation -- I'm leaning towards CSG in Hilbert spaces at the moment, but that may be too computationally demanding -- and how to form abstractions. Does CSG = context-sensitive grammar in this case? How would you use Hilbert spaces? Sorry -- should have been clearer. Constructive Solid Geometry. Manipulating shapes in high- (possibly infinite-) dimensional spaces. Suppose I want to represent a face as a point in a space. First, represent it as a raster. That is in turn a series of numbers that can be a vector in the space. Same face, higher resolution: more numbers, higher dimensionality space, but you can map the regions that represent the same face between higher and lower-dimensional spaces. Do it again, again, etc: take the limit as the resolution and dimensionality go to infinity. You can no more represent this explicitly than you can a real number, but you can use it as an abstraction, as a theory to tell you how well your approximations are working. --Josh - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Understanding Natural Language
I think that Matt and Josh are both misunderstanding what I said in the same way. Really, you're both attacking the use of logic on the predicates, not the predicates themselves as a representation, and so ignoring the distinction I was trying to create. I am not saying that rewriting English as predicates magically provides semantics. On 11/28/06, J. Storrs Hall, PhD. [EMAIL PROTECTED] wrote: On Tuesday 28 November 2006 14:47, Philip Goetz wrote: The use of predicates for representation, and the use of logic for reasoning, are separate issues. I think it's pretty clear that English sentences translate neatly into predicate logic statements, and that such a transformation is likely a useful first step for any sentence-understanding process. Whether those predicates are then used to draw conclusions according to a standard logic system, or are used as inputs to a completely different process, is a different matter. I would beg to differ. While it is clearly straightforward to translate a sencence into a predicate expression in a syntactic way, the resulting structure has no coherent semantics. Translating into a predicate expression doesn't give you any semantics. But it doesn't take any away, either. It just gives you the sentence in a neater form, with the hierarchies and dependencies spelled out. Consider the following sentences. Could you translate them all using the single predicate on(A,B)? If not, the translation gets messier: On the table is an apple. On Lake Ontario is Toronto. On Hadamard's theory transubstantiation is ineffable. On Comet, on Cupid, on Prancer and Vixen. On Christmas we open presents. On time is better than late. On budget expenditures are dwarfed by Social Security. On and on the list goes... You used the same word on in English for each of them. I thus get to use the same word on in a predicate representation for each of them. I don't claim that each instance of the predicate on means the same thing! The application of a logic rule that matched any instance of on(A,B) would be making such a claim, but, as I tried to explicitly point out, that is a problem with logic, not with predicates as a representation. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Understanding Natural Language
Oops, Matt actually is making a different objection than Josh. Now it seems to me that you need to understand sentences before you can translate them into FOL, not the other way around. Before you can translate to FOL you have to parse the sentence, and before you can parse it you have to understand it, e.g. I ate pizza with pepperoni. I ate pizza with a fork. Using my definition of understanding, you have to recognize that ate with a fork and pizza with pepperoni rank higher than ate with pepperoni and pizza with a fork. A parser needs to know millions of rules like this. Yes, this is true. When I said neatly, I didn't mean easily. I mean that the correct representation in predicate logic is very similar to the English, and doesn't lose much meaning. It was misleading of me to say that it's a good starting point, though, since you do have to do a lot to get those predicates. A predicate representation can be very useful. This doesn't mean that you have to represent all of the predications that could be extracted from a sentence. The NLP system I'm working on does not, in fact, use a parse tree, for essentially the reasons Matt just gave. It doesn't want to make commitments about grammatical structure, so instead it just groups things into phrases, without deciding what the dependencies are between those phrases, and then has a bunch of different demons that scan those phrases looking for particular predications. As you find predications in the text, you can eliminate certain choices of lexical or semantic category for words, and eliminate arguments so that they can't be re-used in other predications. You never actually find the correct parse in our system, but you could if you wanted to. It's just that, we've already extracted the meaning that we're interested in by the time we have enough information to get the right parse, so the parse tree isn't of much use. We get the predicates that we're interested in, for the purposes at hand. We might never have to figure out whether pepperoni is a part or an instrument, because we don't care. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Understanding Natural Language
On 11/28/06, J. Storrs Hall, PhD. [EMAIL PROTECTED] wrote: Sorry -- should have been clearer. Constructive Solid Geometry. Manipulating shapes in high- (possibly infinite-) dimensional spaces. Suppose I want to represent a face as a point in a space. First, represent it as a raster. That is in turn a series of numbers that can be a vector in the space. Same face, higher resolution: more numbers, higher dimensionality space, but you can map the regions that represent the same face between higher and lower-dimensional spaces. Do it again, again, etc: take the limit as the resolution and dimensionality go to infinity. You can no more represent this explicitly than you can a real number, but you can use it as an abstraction, as a theory to tell you how well your approximations are working. I see that a raster is a vector. I see that you can have rasters at different resolutions. I don't see what you mean by map the regions that represent the same face between higher and lower-dimensional spaces, or what you are taking the limit of as resolution goes to infinity, or why you don't just stick with one particular resolution. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Understanding Natural Language
First order logic (FOL) is good for expressing simple facts like all birds have wings or no bird has hair, but not for statements like most birds can fly. To do that you have to at least extend it with fuzzy logic (probability and confidence). A second problem is, how do you ground the terms? If you have for all X, bird(X) = has(X, wings), where does bird, wings, has get their meanings? The terms do not map 1-1 to English words, even though we may use the same notation. For example, you can talk about the wings of a building, or the idiom wing it. Most words in the dictionary list several definitions that depend on context. Also, words gradually change their meaning over time. I think FOL represents complex ideas poorly. Try translating what you just wrote into FOL and you will see what I mean. -- Matt Mahoney, [EMAIL PROTECTED] - Original Message From: Philip Goetz [EMAIL PROTECTED] To: agi@v2.listbox.com Sent: Tuesday, November 28, 2006 5:45:51 PM Subject: Re: [agi] Understanding Natural Language Oops, Matt actually is making a different objection than Josh. Now it seems to me that you need to understand sentences before you can translate them into FOL, not the other way around. Before you can translate to FOL you have to parse the sentence, and before you can parse it you have to understand it, e.g. I ate pizza with pepperoni. I ate pizza with a fork. Using my definition of understanding, you have to recognize that ate with a fork and pizza with pepperoni rank higher than ate with pepperoni and pizza with a fork. A parser needs to know millions of rules like this. Yes, this is true. When I said neatly, I didn't mean easily. I mean that the correct representation in predicate logic is very similar to the English, and doesn't lose much meaning. It was misleading of me to say that it's a good starting point, though, since you do have to do a lot to get those predicates. A predicate representation can be very useful. This doesn't mean that you have to represent all of the predications that could be extracted from a sentence. The NLP system I'm working on does not, in fact, use a parse tree, for essentially the reasons Matt just gave. It doesn't want to make commitments about grammatical structure, so instead it just groups things into phrases, without deciding what the dependencies are between those phrases, and then has a bunch of different demons that scan those phrases looking for particular predications. As you find predications in the text, you can eliminate certain choices of lexical or semantic category for words, and eliminate arguments so that they can't be re-used in other predications. You never actually find the correct parse in our system, but you could if you wanted to. It's just that, we've already extracted the meaning that we're interested in by the time we have enough information to get the right parse, so the parse tree isn't of much use. We get the predicates that we're interested in, for the purposes at hand. We might never have to figure out whether pepperoni is a part or an instrument, because we don't care. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303 - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Understanding Natural Language
On Sunday 26 November 2006 18:02, Mike Dougherty wrote: I was thinking about the N-space representation of an idea... Then I thought about the tilting table analogy Richard posted elsewhere (sorry, I'm terrible at citing sources) Then I starting wondering what would happen if the N-space geometric object were not an idea, but the computing machine - responding to the surface upon which it found itself. So if the 'computer' (brain, etc.) were a simple sphere like a marble affected by gravity on a wobbly tabletop, the phase space would be straightforward. It's difficult to conceive of an N dimensional object in an N+m dimensional tabletop being acted upon by some number of gravity analogues. This is essentially what a Hopfield net does. The setting of all the weights produces an energy surface in the n-dimensional space generated by the signal strengths of the n units. The state of the system follows the surface, seeking lowest energy; the surface gets tilted by virtue of different inputs on some of the wires, and some of the dimensions get used as continuously varying outputs on other wires. I saw Hopfield demo a net with just ten units (10 op-amps, 100 potentiometers for the synaptic weights) that was connected to a microphone and could recognize the ten digits spoken into it. He claimed that it would work at radio frequencies, if anybody could talk that fast :-) The only trouble with Hopfield nets is that nobody but Hopfield can program them. Hugo wants to build special-purpose hardware just to evolve weight-settings, and I wish him luck. Is this at least in the right direction of what you are proposing? Have you projected the dimensionality of the human brain? That would at least give a baseline upon which to speculate - especially considering that we have enough difficulty understanding perspective dimension on a 2D painting, let alone conceive of (and articulate) dimensions higher than our own. (assuming the incompleteness theorem isn't expressly prohibiting it) I'm proposing to use the better-understood (by me) hardware content-sddressable memory (or rather simulate it on an ordinary computer) to do a poor man's version of that, but in a way that I do know how to program, and most importantly, that mostly programs itself by watching what's going on. Chances are that someone really smart could rig a way to do that with a real Hopfield net, since he invented them as associative memories in the first place... (J. J. Hopfield, Neural networks and physical systems with emergent collective computational abilities, Proceedings of the National Academy of Sciences of the USA, vol. 79 no. 8 pp. 2554-2558, April 1982. http://www.pnas.org/cgi/content/abstract/79/8/2554) ... but I'm not that smart :-) --Josh - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: Re: [agi] Understanding Natural Language
Amusingly, one of my projects at the moment is to show that Novamente's economic attention allocation module can display Hopfield net type content-addressable-memory behavior on simple examples. As a preliminary step to integrating it with other aspects of Novamente cognition (reasoning, evolutionary learning, etc.) Those interested in Hopfield nets may want to look up Daniel Amit's old book modeling brain function http://www.amazon.com/Modelling-Brain-Function-Attractor-Networks/dp/0521421241/sr=1-3/qid=1164641397/ref=sr_1_3/002-6495259-3104828?ie=UTF8s=books which goes way beyond the fixed-point attractors John Hopfield focused on, and discusses at length strange attractors in neural nets with asymmetric weights. This work was inspirational for Novamente, which is intended to show similar attractor-formation effects through the flow of artificial currency (allocated among knowledge items and relationships via probability theory) rather than the flow of simulated neural net activation. An issue with Hopfield content-addressable memories is that their memory capability gets worse and worse as the networks get sparser and sparser. I did some experiments on this in 1997, though I never bothered to publish the results ... some of them are at: http://www.goertzel.org/papers/ANNPaper.html The probability/economics approach used in Novamente enables the same sort of attractor formation but with better behavior under realistic network sparsity... Novamente however does not rely on attractors as the sole method of memory storage. Rather, it uses logical knowledge representation, but then also uses attractors of logical atoms (under economic-attention-allocation dynamics) to represent an upper layer of more fluid knowledge. -- Ben G On 11/27/06, J. Storrs Hall, PhD. [EMAIL PROTECTED] wrote: On Sunday 26 November 2006 18:02, Mike Dougherty wrote: I was thinking about the N-space representation of an idea... Then I thought about the tilting table analogy Richard posted elsewhere (sorry, I'm terrible at citing sources) Then I starting wondering what would happen if the N-space geometric object were not an idea, but the computing machine - responding to the surface upon which it found itself. So if the 'computer' (brain, etc.) were a simple sphere like a marble affected by gravity on a wobbly tabletop, the phase space would be straightforward. It's difficult to conceive of an N dimensional object in an N+m dimensional tabletop being acted upon by some number of gravity analogues. This is essentially what a Hopfield net does. The setting of all the weights produces an energy surface in the n-dimensional space generated by the signal strengths of the n units. The state of the system follows the surface, seeking lowest energy; the surface gets tilted by virtue of different inputs on some of the wires, and some of the dimensions get used as continuously varying outputs on other wires. I saw Hopfield demo a net with just ten units (10 op-amps, 100 potentiometers for the synaptic weights) that was connected to a microphone and could recognize the ten digits spoken into it. He claimed that it would work at radio frequencies, if anybody could talk that fast :-) The only trouble with Hopfield nets is that nobody but Hopfield can program them. Hugo wants to build special-purpose hardware just to evolve weight-settings, and I wish him luck. Is this at least in the right direction of what you are proposing? Have you projected the dimensionality of the human brain? That would at least give a baseline upon which to speculate - especially considering that we have enough difficulty understanding perspective dimension on a 2D painting, let alone conceive of (and articulate) dimensions higher than our own. (assuming the incompleteness theorem isn't expressly prohibiting it) I'm proposing to use the better-understood (by me) hardware content-sddressable memory (or rather simulate it on an ordinary computer) to do a poor man's version of that, but in a way that I do know how to program, and most importantly, that mostly programs itself by watching what's going on. Chances are that someone really smart could rig a way to do that with a real Hopfield net, since he invented them as associative memories in the first place... (J. J. Hopfield, Neural networks and physical systems with emergent collective computational abilities, Proceedings of the National Academy of Sciences of the USA, vol. 79 no. 8 pp. 2554-2558, April 1982. http://www.pnas.org/cgi/content/abstract/79/8/2554) ... but I'm not that smart :-) --Josh - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303 - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: Re: [agi] Understanding Natural Language
I'm not saying that the n-space approach wouldn't work, but I have used that approach before and faced a problem. It was because of that problem that I switched to a logic-based approach. Maybe you can solve it. To illustrate it with an example, let's say the AGI can recognize apples, bananas, tables, chairs, the face of Einstein, etc, in the n-dimensional feature space. So, Einstein's face is defined by a hypersurface where each point is an instance of Einstein's face; and you can get a caricature of Einstein by going near the fringes of this hypervolume. So far so good. Now suppose you want to say: the apple is *on* the table, the banana is *on* the chair, etc. In logical form it would be on(table,apple), etc. There can be infinitely many such statements. The problem is that this thing, on, is not definable in n-space via operations like AND, OR, NOT, etc. It seems that on is not definable by *any* hypersurface, so it cannot be learned by classifiers like feedforward neural networks or SVMs. You can define apple on table in n-space, which is the set of all configurations of apples on tables; but there is no way to define X is on Y as a hypervolume, and thus to make it learnable. This problem extends to other predicates besides on(x,y). YY - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: Re: [agi] Understanding Natural Language
On 11/27/06, YKY (Yan King Yin) [EMAIL PROTECTED] wrote: The problem is that this thing, on, is not definable in n-space via operations like AND, OR, NOT, etc. It seems that on is not definable by *any* hypersurface, so it cannot be learned by classifiers like feedforward neural networks or SVMs. You can define apple on table in n-space, which is the set of all configurations of apples on tables; but there is no way to define X is on Y as a hypervolume, and thus to make it learnable. perhaps my view of a hypersurface is wrong, but wouldn't a subset of the dimensions associated with an object be the physical dimensions? (ok, virtual physical dimensions) Is On determined by a point of contact between two objects? (A is on B and B is on A) Or is there a dependancy on the direction of gravity? (A is on B, but B is on the floor) You say that on could not be learned - why not? In this case it would seem that the meaning would effectively be cultural and the meaning would depend on the semantic usage/intent of the tutors.. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: Re: [agi] Understanding Natural Language
On 11/28/06, Mike Dougherty [EMAIL PROTECTED] wrote: perhaps my view of a hypersurface is wrong, but wouldn't a subset of the dimensions associated with an object be the physical dimensions? (ok, virtual physical dimensions) Is On determined by a point of contact between two objects? (A is on B and B is on A) Or is there a dependancy on the direction of gravity? (A is on B, but B is on the floor) You say that on could not be learned - why not? In this case it would seem that the meaning would effectively be cultural and the meaning would depend on the semantic usage/intent of the tutors.. I'm talking about a system whose objects are points in a hyper feature-space, the features are sensor values, for example pixels of the camera. This is not the same as physical space which is (3+1)D. Maybe we arn't talking about the same thing. Anyway, by using predicate logic I have made some bold simplifications and the resulting system is still not easy to build. Anything fancier than that could only be harder. I think we need to keep it simple. YY - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Understanding Natural Language
On Monday 27 November 2006 11:49, YKY (Yan King Yin) wrote: To illustrate it with an example, let's say the AGI can recognize apples, bananas, tables, chairs, the face of Einstein, etc, in the n-dimensional feature space. So, Einstein's face is defined by a hypersurface where each point is an instance of Einstein's face; and you can get a caricature of Einstein by going near the fringes of this hypervolume. So far so good. In my scheme at least, there's not just one space. There's one space per abstractable phenomenon in the ontology of the AI. The features that most of them are defined in terms of are projections of other ones, in many cases simply a signal saying how strongly the lower-level unit understands whatever it's seeing. Now suppose you want to say: the apple is *on* the table, the banana is *on* the chair, etc. In logical form it would be on(table,apple), etc. There can be infinitely many such statements. But they all mean subtly different things. The notion that a predicate at the level of abstraction of a natural language preposition captures anything coherent in reality is very likely an illusion. I can put a ship on the ocean and a drop of water on a table, but not a ship on a table (it would crush it to splinters) or a drop of water on the ocean. I can put an apple on the table but not on the ocean (it's in the ocean even though it floats just like the ship). If I look at the underside of the tabletop and see a serial number stencilled there, is the ink on the table? If I glued an apple to the same spot, would it be on the table? I think that this is part of what Minsky was trying to capture with his Society of More, but I don't think that most people reading it get the whole point -- I certainly didn't at first. The idea is that the things we think of as unitary, simple semantic relations are in reality broad and *poorly defined* similarity classes in the many, many micromodels that make up the patchwork quilt of our overall world model. The problem is that this thing, on, is not definable in n-space via operations like AND, OR, NOT, etc. It seems that on is not definable by *any* hypersurface, so it cannot be learned by classifiers like feedforward neural networks or SVMs. You can define apple on table in n-space, which is the set of all configurations of apples on tables; but there is no way to define X is on Y as a hypervolume, and thus to make it learnable. On is certainly not defineable in the space of the features that could distinguish apples from oranges, for example. But I think most of the listners here have at least stipulated that n-spaces are good for representing physics, and by extension I trust no one will have a problem if I claim that it's not too hard to do simulations of simple household rigid-body mechanics. Take the space where you're doing that and project into one where you only have the trajectories of the centers of gravity of the small objects. Now look at the space in the vicinity of the table (after having done a lot of random experiments with objects). There will be two distinct classes of points: those where the object is falling, and those where it is at rest. Hah! a regularity. Split off two concepts, call one above the table, the other on the table. We can't put a ship on the table but we can put it on the ocean. If we do a mapping between the ship micromodel and the table one, there are some features of the dynamics that match up pretty well. In normal experience there is no problem disambiguating these two meanings of on, so we use the same word for them and don't even realize they're different. Until we try to translate languages, that is -- prepositions are notoriously hard to translate. --Josh - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Understanding Natural Language
On Saturday 25 November 2006 13:52, Ben Goertzel wrote: About Teddy Meese: a well-designed Teddy Moose is almost surely going to have the big antlers characterizing a male moose, rather than the head-profile of a female moose; and it would be disappointing if a Teddy Moose had the head and upper body of a bear and the udders and hooves of a moose; etc. So obviously a simple blend like this is not just **any** interpolation, it's an interpolation where the most salient features of each item being blended are favored, wherever this is possible without conflict. But I agree that this should be doable within an n-vector framework without requiring any breakthroughs... A little more about this: The salient features of a bear or moose are those that would go into a caricature. (There is also a significant anthropomorphization, a blending in of human characteristics.) It's long been shown that *with the proper mapping*, caricatures can be generated by n-space geometry. You find a point that represents an average of individuals in the class you're interested in, take the individual you're trying to caricature and project further along the line of difference. A classic example is Susan Brennan's caricature generator: Brennan, S. Caricature Generation: The Dynamic Exaggeration of Faces by Computer. Leonardo 18, No. 3 (1985), 170-178. (an example is shown in http://cogprints.org/172/00/faces1.ps) Another more recent result using an n-space representation (they call it a Vector Space Model) is Turney, Peter D. and Littman, Michael L. (2005) Corpus-based Learning of Analogies and Semantic Relations. Machine Learning 60(1-3):pp. 251-278. (http://cogprints.org/4518/01/NRC-48273.pdf) A follow-on paper (http://arxiv.org/pdf/cs.CL/0412024) is the work that recently got in the news by equalling the performance of college-bound students on verbal-analogy SAT test questions. You can get some help finding the average animal and seeing how much human character is mixed in by backtracking from teddy bears. Another approach, just as congenial to my tentative architecture, is to use a memory of a caricature moose, e.g. Bullwinkle. --Josh - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: Re: [agi] Understanding Natural Language
HI, Therefore, the problem of using an n-space representation for AGI is not its theoretical possibility (it is possible), but its practical feasibility. I have no doubt that for many limited application, n-space representation is the most natural and efficient choice. However, for a general purpose system, the situation is very different. I'm afraid for AGI we may have to need millions (if not more) dimensions, and it won't be easy to decide in advance what dimensions are necessary. I see no problem with using a sparse representation of dimensions in an n-vector K-rep approach, actually... ben - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: Re: [agi] Understanding Natural Language
On 11/26/06, Ben Goertzel [EMAIL PROTECTED] wrote: HI, Therefore, the problem of using an n-space representation for AGI is not its theoretical possibility (it is possible), but its practical feasibility. I have no doubt that for many limited application, n-space representation is the most natural and efficient choice. However, for a general purpose system, the situation is very different. I'm afraid for AGI we may have to need millions (if not more) dimensions, and it won't be easy to decide in advance what dimensions are necessary. I see no problem with using a sparse representation of dimensions in an n-vector K-rep approach, actually... It is not about the time-space cost, but how to choose the dimensions (if they are determined in advance and remain constant), or how to maintain consistency (if they are dynamically added and deleted). Pei - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Understanding Natural Language
My best ideas at the moment don't have one big space where everything sits, but something more like a Society of Mind where each agent has its own space. New agents are being tried all the time by some heuristic search process, and will come with new dimensions if that does them any good. Equally important is to collapse dimensions in higher-level agents, forming abstractions. Let's say I'm playing tennis. I want to hit a backhand. I have an agent for each joint in my arm that reads proprioceptive info and sends motor signals. Each one of these knows a lot about the amount of effort necessary to get what angle or acceleration at that joint as a function of the existing position, tiredness of the muscle, etc, etc. This info is essentially an interpolation of memories. I have a higher-level agent that knows how to do a backhand drive using these lower-level ones, and it has a much more abbreviated notion of what's going on at each joint, but it does know a lot about sequencing, timing, and how far the ball will go -- also based on memory. I also have a forehand agent using the same lower-level ones, and so forth. It probably has a space very similar to the backhand one, but the warp and woof of the remembered trajectories in the space will be all different. At higher levels I have within-the-point strategy agents that decide which strokes to use and where to hit to in the opposite court. The spaces for these agents may have subspaces that map recognizeably to a 2-d tennis court, perhaps. Higher up I have an agent that knows how the game scoring works, in which most the dimensions are binary -- I win the point or my opponent does. Such a space boils down to a finite state machine. Chances are that in real life, I've been in a tennis game at every possible score, but I didn't have to -- I didn't build the state space for that agent purely from memory, indicating a more sophisticated form of interpolation. So the basic idea is like Minsky's or Brooks' or Albus' modular architectures but with interpolating n-space trajectory memories as each agent or module. I don't understand Hugo's architecture of Hopfield nets well enough to say whether it's equivalent or not; it could certainly match the performance but I couldn't say whether it could match the learning. --Josh On Sunday 26 November 2006 10:15, Pei Wang wrote: On 11/26/06, Ben Goertzel [EMAIL PROTECTED] wrote: HI, Therefore, the problem of using an n-space representation for AGI is not its theoretical possibility (it is possible), but its practical feasibility. I have no doubt that for many limited application, n-space representation is the most natural and efficient choice. However, for a general purpose system, the situation is very different. I'm afraid for AGI we may have to need millions (if not more) dimensions, and it won't be easy to decide in advance what dimensions are necessary. I see no problem with using a sparse representation of dimensions in an n-vector K-rep approach, actually... It is not about the time-space cost, but how to choose the dimensions (if they are determined in advance and remain constant), or how to maintain consistency (if they are dynamically added and deleted). Pei - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303 - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Understanding Natural Language
J. Storrs Hall, PhD. wrote: My best ideas at the moment don't have one big space where everything sits, but something more like a Society of Mind where each agent has its own space. New agents are being tried all the time by some heuristic search process, and will come with new dimensions if that does them any good. Equally important is to collapse dimensions in higher-level agents, forming abstractions. Let's say I'm playing tennis. I want to hit a backhand. I have an agent for each joint in my arm that reads proprioceptive info and sends motor signals. Each one of these knows a lot about the amount of effort necessary to get what angle or acceleration at that joint as a function of the existing position, tiredness of the muscle, etc, etc. This info is essentially an interpolation of memories. I have a higher-level agent that knows how to do a backhand drive using these lower-level ones, and it has a much more abbreviated notion of what's going on at each joint, but it does know a lot about sequencing, timing, and how far the ball will go -- also based on memory. I also have a forehand agent using the same lower-level ones, and so forth. It probably has a space very similar to the backhand one, but the warp and woof of the remembered trajectories in the space will be all different. At higher levels I have within-the-point strategy agents that decide which strokes to use and where to hit to in the opposite court. The spaces for these agents may have subspaces that map recognizeably to a 2-d tennis court, perhaps. Higher up I have an agent that knows how the game scoring works, in which most the dimensions are binary -- I win the point or my opponent does. Such a space boils down to a finite state machine. Chances are that in real life, I've been in a tennis game at every possible score, but I didn't have to -- I didn't build the state space for that agent purely from memory, indicating a more sophisticated form of interpolation. So the basic idea is like Minsky's or Brooks' or Albus' modular architectures but with interpolating n-space trajectory memories as each agent or module. I don't understand Hugo's architecture of Hopfield nets well enough to say whether it's equivalent or not; it could certainly match the performance but I couldn't say whether it could match the learning. The problem with this, as I see it, is that the reason a physicist cares about vector spaces is for their metrical properties: there is a distance measure, and since that is the way the real world is, it buys the physicist a *lot* of traction. But if you want to uses spaces for this reason, I have to ask: why? What does the metricality of a space buy you? And what makes you think that, in practice, you will actually get what you wanted to get from it when you sit down and implement the model? If, on the other hand, you don't really care about the metric properties (if they don't correspond to anything in your model) then your description reduces to a framework that only has hierarchicality in it, and nothing more (hierarchies of agents). Now, there are a million such frameworks (mine looks identitical to yours, in that respect), so you would not have made much progress. I know what Minsky meant by physics envy. Hull, the psychologist, had the same affliction (in spades: IIRC his stuff was a monumental vector space version of the behaviorist paradigm). I can sympathize, being an emigre physicist myself, but I must say that that I think it buys nothing, because at the end of the day you have no reason to suppose that such a framework heads in the direction of a system that is intelligent. You could build an entire system using the framework, and then do some experiments, and then I'd be convinced. But short of that I don't see any reason to be optimistic. Richard Loosemore - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Understanding Natural Language
On Sunday 26 November 2006 14:14, Pei Wang wrote: In this design, the tough job is to make the agents working together to cover all kinds of tasks, and for this part, I'm afraid that the multi-dimensional space representation won't help much. Also, we haven't seen much work on high-level cognition in that framework. Pei and Richard in a similar vein: The problem with this, as I see it, is that the reason a physicist cares about vector spaces is for their metrical properties: there is a distance measure, and since that is the way the real world is, it buys the physicist a *lot* of traction. But if you want to uses spaces for this reason, I have to ask: why? What does the metricality of a space buy you? And what makes you think that, in practice, you will actually get what you wanted to get from it when you sit down and implement the model? Your mutual point is well taken, and I don't have a better reason than to note that a mouse and a giraffe have essentially the same skeleton -- evolution tends to take whatever works in one place and warp the hell out of it in another one, rather than come up with something new (and more optimal). It's something like me with my hammer and everything looking like a nail. My hammer in this case is that I know how to do a very straightforward memory-based learning within such a module, and I intend to have that, together with the interpolation and extrapolation capabilities I get for free, at *every* point in the overall architecture, since it needs all the help it can get. But I really think that the metric properties of the spaces continue to help even at the very highest levels of abstraction. I'm willing to spend some time giving it a shot, anyway. So we'll see! Josh - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: Re: [agi] Understanding Natural Language
that do can't easily add code. I was thinking of making an AI library, but realized you'd have to make one for every language. Then I though about how we could unify every language. Then I realized that there was non-standard convention everywhere. It was mentioned to me that Lojban existed I found it learned it, am now making parser. Allow for standard compliance to be easier as it is intuitive and natural to comply to standards when you are simply extending the language that you speak by making new sentances from words you already know. Sentances which are no different than programs. A general AI would emerge out of many hundreds of people using the development framework and extending it to understand them. Eventually it will understand it's users and be able to do what they ask of it. As what has been explained once doesn't have to be again -- this is of course assuming the small functions connected to distributed network to redistribute functions to other computers that will look for them without any need of user interferance (mmorpg will probably run on same network). -- Matt Mahoney, [EMAIL PROTECTED] - Original Message From: Andrii (lOkadin) Zvorygin [EMAIL PROTECTED] To: agi@v2.listbox.com Sent: Saturday, November 25, 2006 5:01:04 AM Subject: Re: Re: [agi] Understanding Natural Language On 11/24/06, Matt Mahoney [EMAIL PROTECTED] wrote: Andrii (lOkadin) Zvorygin [EMAIL PROTECTED] wrote: I personally don't understand why everyone seems to insist on using ambiguous illogical languages to express things when there are viable alternative available. I think because an AGI needs to communicate in languages that people already know. I don't understand how artificial languages like Lojban contribute to this goal. We should focus our efforts instead on learning and modeling existing languages. I understand that artificial languages like Lojban and Esperanto and Attempto have simple grammars. I don't believe they would stay that way if they were widely used for person to person communication (as opposed to machine interfaces). Lojban grammar is easily extensible and forwards compatible. You can add features to the language through CMAvo and GISmu. Lojban already exceeds many natural languages in it's ability to express. There are very crucial parts of communication that English lacks such as logical connectives and attitudinals. Languages evolve over time, both in individuals, and more slowly in social groups. Are you implying languages evolve faster in individuals? A language model is not a simple set of rules. A natural language model is not. An artificial language is constructed with rules that were also created by individual -- as opposed to groups of -- humans. Lojban was especially designed to be logical, unlike Esperanto. Therefore making them recreatable by individual humans, and depending on your definition: simple. It is a probability distribution described by a large set of patterns such as words, word associations, grammatical structures and sentences. The approach of a world of blind to seeing is to feel at things. Sometimes they wonder if there is not another way. Each time you read or hear a message, the probabilities for the observed patterns are increased a little and new patterns are added. In a social setting, these probabilities tend to converge by consensus as this knowledge is shared. I agree this is a wonderful solution to predicting what the vocabulary of a language group is. Formal definitions of artificial languages do not capture this type of knowledge, the thousands or millions of new words, idioms, shared knowledge and habits of usage. sa'u(simply speaking) Artificial languages lack a historic/cultural user base. Do I even need to reply to that?zo'o.ui.u'i(last statement humourously while happy in an amused kind of way) Even if we were able to constrain the grammar, you still have the problem that people will still make ungrammatical statements, misspell words, omit words, and so on. Amazing you should mention such valid points against natural languages. * ungrammatical statements: If they were ungrammatical they wouldn't parse in the universal Lojban parser(All Lojban parsers can be Universal Lojban parsers as long as they follow the few simple grammar rules). * misspell words: In Lojban words have a very strict formation, mu'a(for example): GISmu are either in (ccvcv or cvccv formation) all others are also syntactically unambiguous. Additionally words in Lojban are specifically designed not to sound similar to each other, so chances are it still looks/sounds just like the original word even when misspelled. If a parse error occurs(rare for Lojban users, usually typos) the user can always be notified. *omit words: (I gave an example of some GISmu before, basically they have predefined places, so you can always ask a specific question about ommitted information by simply putting a ma for the SUMti(argument) which you wish
Re: [agi] Understanding Natural Language
On 11/26/06, J. Storrs Hall, PhD. [EMAIL PROTECTED] wrote: But I really think that the metric properties of the spaces continue to help even at the very highest levels of abstraction. I'm willing to spend some time giving it a shot, anyway. So we'll see! I was thinking about the N-space representation of an idea... Then I thought about the tilting table analogy Richard posted elsewhere (sorry, I'm terrible at citing sources) Then I starting wondering what would happen if the N-space geometric object were not an idea, but the computing machine - responding to the surface upon which it found itself. So if the 'computer' (brain, etc.) were a simple sphere like a marble affected by gravity on a wobbly tabletop, the phase space would be straightforward. It's difficult to conceive of an N dimensional object in an N+m dimensional tabletop being acted upon by some number of gravity analogues. Is this at least in the right direction of what you are proposing? Have you projected the dimensionality of the human brain? That would at least give a baseline upon which to speculate - especially considering that we have enough difficulty understanding perspective dimension on a 2D painting, let alone conceive of (and articulate) dimensions higher than our own. (assuming the incompleteness theorem isn't expressly prohibiting it) - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: Re: [agi] Understanding Natural Language
My point about artificial languages is I don't believe that they are of much use in helping to understand or solve the natural language modeling problem, which is a central problem to AGI. Ben mentioned one use, which is to use Lojban++ in combination with English to train an AGI in English. In this case, Lojban++ serves to help ground the language, just as using a 3-D modeling language could also be used to describe the environment. In this case, any language which is expressive enough to do this and is familiar to the developer will do. It is a different case where we require users to learn an artificial language because we don't know how to model natural language. I don't see how this can lead to any significant insights. There are already many examples of unabiguous and easy to parse programming languages (including superficially English-like languages such as COBOL and SQL) and formal knowledge representation languages (Cycl, prolog, etc). An AGI has to deal with ambiguity and errors in language. Consider the following sentence which I used earlier: I could even invent a new branch of mathematics, introduce appropriate notation, and express ideas in it. What does it refer to? The solution in an artificial language would be either to forbid pronouns (as in most programming languages) or explicitly label it to make the meaning explicit. But people don't want or need to do this. They can figure it out by context. If your AGI can't use context to solve such problems then you haven't solved the natural language modeling problem, and a vast body of knowledge will be inaccessible. I think you will find that writing a Lojban parser will be trivial compared to writing an English to Lojban translator. Andrii (lOkadin) Zvorygin [EMAIL PROTECTED] wrote: My initial reasoning was that right now many programs don't use AI, because programmers don't know, and the ones that do can't easily add code. It is because language modeling is unsolved. Computers would be much easier to use if we could talk to them in English. But they do not understand. We don't know how to make them understand. But we are making progress. Google will answer simple, natural language questions (although they don't advertise it). The fact that others haven't done it suggests the problem requires vast computational resources and training data. -- Matt Mahoney, [EMAIL PROTECTED] - Original Message From: Andrii (lOkadin) Zvorygin [EMAIL PROTECTED] To: agi@v2.listbox.com Sent: Sunday, November 26, 2006 4:37:02 PM Subject: Re: Re: [agi] Understanding Natural Language On 11/25/06, Matt Mahoney [EMAIL PROTECTED] wrote: Andrii (lOkadin) Zvorygin [EMAIL PROTECTED] wrote: Even if we were able to constrain the grammar, you still have the problem that people will still make ungrammatical statements, misspell words, omit words, and so on. Amazing you should mention such valid points against natural languages. This misses the point. Where are you going to get 1 GB of Lojban text to train your language model? Well A: I could just get IRC logs and mailing lists of the current Lojban community. B: point is to translate English into Lojban C: I'm not training a language model. I'm creating a parser, then a translator, then other things. The translator will have some elements of an AI probably Bayesian probability will be involved, it's too early to say however. I may be on the wrong list discussing this. If you require that all text pass through a syntax checker for errors, you will greatly increase the cost of generating your training data. Well A: There are rarely any errors -- unlike in a natural language like say English. B: Addressed above. This is not a trivial problem. Which one? Maybe as a whole it's not trivial, but when you break it down the little pieces are all individually trivial. It is a big part of why programmers can only write 10 lines of code per day on projects 1/1000 the size of a language model. Monolithic programming is the paradigm of the past, is one of the reasons I'm creating this new development model. Then when you have built the model, you will still have a system that is intolerant of errors and hard to use. Because of the nature of the development model -- designed after functional programming languages, going to be able to add functions anywhere in the process without interupting the rest of the functions, as it wont be changing the input other functions recieve(unless that is the intent). Hard to use? Well we'll see when I have a basic implementation, the whole point is so that it will be easy to use, maybe it wont work out though -- can't see how. .iacu'i(skepticism) Your language model needs to have a better way to deal with inconsistency than to report errors and make more work for the user. It can easily just check what the previous response of this user, or someone else that has made a similar error was when correcting. Trivial once we get
Re: Re: [agi] Understanding Natural Language
I constructed a while ago (mathematically) a detailed mapping from Novamente Atoms (nodes/links) into n-dimensional vectors. You can certainly view the state of a Novamente system at a given point in time as a collection of n-vectors, and the various cognition methods in Novamente as mappings from R^k - R^m ... So, based on this, I can see that an n-dimensional representation of knowledge and mind-states is definitely feasible... However, I don't see that this, in itself, really gets you much of anything... I'm afraid the analogies between vector space operations and cognitive operations don't really take you very far. For instance, you map conceptual blending into quantitative interpolation -- but as you surely know, it's not just **any** interpolation, it's a highly special kind of interpolation, and to formalize or teach an AI system this specialness is nontrivial whether your underlying k-rep is n-vectors or probabilistic logic formulas or whatever... -- Ben On 11/25/06, J. Storrs Hall, PhD. [EMAIL PROTECTED] wrote: On Friday 24 November 2006 10:26, William Pearson wrote: On 24/11/06, J. Storrs Hall, PhD. [EMAIL PROTECTED] wrote: The open questions are representation -- I'm leaning towards CSG Constructive solid geometry? You could probably go quite far towards a real world navigator with this, but I'm not usre how you plan to get it to represent the internal state of other systems, so it can try to predict what actions people may take due to their emotions etc. I'm in favour of multiple and changeable representations myself. Quite how to integrate them all into a somewhat coherent world view is an interesting problem though. Imagine a space in which a frame is a single point. This is standard practice in physical science, where the space is sometimes referred to as a phase space. As the frame evolves in time, it will describe a trajectory in the space. Similar, slightly variant frames will occur near the original one in the space. Combining all possible trajectories under a given constraint produces a subspace that is a hypersurface in the original one, like the potential energy surfaces seen in physics and chemistry. (The spaces typically have numbers of dimensions ranging from thousands to Avagadro's number...) A frame can be thought of a sentence, a constraint on the set of possible worlds. The surface can be thought of as a space of possible worlds, and regions in the space represent sentences. Doing CSG on regions in the space is semantically equivalent to using propositional connectives on sentences -- AND = intersection, OR = union, etc. The reason for going to all this trouble is that not only propositional logic, but many other useful conceptual operations have fairly simple geometric cognates in this representation. Conceptual blending is just interpolation. Various kinds of prediction are extrapolation. Some fairly standard forms of metaphor reduce to geometric quadrature: if A is to B as C is to D, A = B + C - D. Finding abstractions reduces to projective geometry, or rather doing abstraction does -- finding useful ones is really the holy grail! The really cool thing about n-spaces as a representation scheme is that they do ground out in simple geometric representations of the physical world at the concrete end, but extend more or less seamlessly into the abstract. Some intermediates: from the 3-d representation of a robot's surroundings, to the n-DOF configuration space of its possible positions, to one with it and all the other objects around, to a Lagrangian space for dynamics where least-energy action planning turns into path-finding. Now take 10 million or so associative memories (think cortical columns), each of which records trajectories of a different frame-type. Most of these are abstractions that are defined in terms of transformations and combinations of other frames, of course. Each one can be used as a CBR-style predictor, planner, and modeller. Hook them all together and you have a fair dinkum engine of reason and seat of the soul. Josh - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303 - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Understanding Natural Language
On Saturday 25 November 2006 12:42, Ben Goertzel wrote: I'm afraid the analogies between vector space operations and cognitive operations don't really take you very far. For instance, you map conceptual blending into quantitative interpolation -- but as you surely know, it's not just **any** interpolation, it's a highly special kind of interpolation, and to formalize or teach an AI system this specialness is nontrivial whether your underlying k-rep is n-vectors or probabilistic logic formulas or whatever... Let's suppose we're trying to blend a steamship, which is a vehicle that carries people, and a bird, which flies, to get a notion of how to build an airplane, which flies and carries people. Historically, lots of combinations of the salient aspects were tried, including Maxim's 9-ton steam-powered aircraft of the early 1890's. What turned out to work was adding the screw and rudder from the boat to the wings and tail of the bird, in a body that was of intermediate size. Poopdecks, masts, and the captain's gig were dispensed with, as were clawfeet and feathers. So the creation of a new, blended concept including some features and discarding others from each blendee, as well as averaging or extrapolating others. The Wrights used wing-warping, as in birds, for attitude control, where Curtis duplicated the rudder once again on the wings as ailerons. Such complex blends require a complex mapping to be discovered from the blendees to the new concept; they do retain the character of a blend by virtue of inheriting some of the predictive ability of the blendees. (For example, the plane yaws the same way the ship does when the vertical tail rudder is turned the same way.) On the other hand, somewhat simpler blends can be done by simple interpolation or mappings like the analogical quadrature I mentioned. For example, you will instantly understand teddy moose to be that which is to a moose as a teddy bear is to a bear, i.e. a stuffed-animal toy caricature. I'm fairly sure I could define a continuous space in which such a thing would fall out of the simple geometric formula. The key is always always always always always to get the mapping into the space right, which is to say getting the projections that form the abstractions from the lower-level concepts right, which is why I described it as the holy grail. --Josh - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: Re: Re: [agi] Understanding Natural Language
The point of using Lojban for proto-AGI's is to enable productive, interactive conversations with AGI's at a fairly early stage in their development ... Of course, mining masses of online English text is a better way for the system to gain general knowledge about science, politics, human psychology and the like... But, perhaps the deeper understanding an AGI needs to meaningfully interpret all that data in the masses of online English text, can best be imparted to the AGI via interacting with it in a simulation world and conversing with it about its experiences using a language that it can interpret more readily (Lojban). Lojban does not solve any of the hard problems of AGI, but it may decrease the amount of pragmatic effort required to turn a theoretical solution to the hard problems of AGI into a practical solution. -- BenG On 11/25/06, Matt Mahoney [EMAIL PROTECTED] wrote: Andrii (lOkadin) Zvorygin [EMAIL PROTECTED] wrote: Even if we were able to constrain the grammar, you still have the problem that people will still make ungrammatical statements, misspell words, omit words, and so on. Amazing you should mention such valid points against natural languages. This misses the point. Where are you going to get 1 GB of Lojban text to train your language model? If you require that all text pass through a syntax checker for errors, you will greatly increase the cost of generating your training data. This is not a trivial problem. It is a big part of why programmers can only write 10 lines of code per day on projects 1/1000 the size of a language model. Then when you have built the model, you will still have a system that is intolerant of errors and hard to use. Your language model needs to have a better way to deal with inconsistency than to report errors and make more work for the user. Lojban already exceeds many natural languages in it's ability to express. How so? In English I can use mathematical notation understood by others to express complex ideas. I could even invent a new branch of mathematics, introduce appropriate notation, and express ideas in it. There are cognitive limits to what natural language can express, such as the inability to describe a person's face (as well as a picture would), or to describe a novel odor, or to convey learned physical skills such as swimming or riding a bicycle. One could conceivably introduce notation to describe such things in any natural or artificial language, but that does not solve the problem. Your neural circuitry has limits; it allows you to connect a face to a name but not to a description. Any such notation might be usable by machines but not by humans. -- Matt Mahoney, [EMAIL PROTECTED] - Original Message From: Andrii (lOkadin) Zvorygin [EMAIL PROTECTED] To: agi@v2.listbox.com Sent: Saturday, November 25, 2006 5:01:04 AM Subject: Re: Re: [agi] Understanding Natural Language On 11/24/06, Matt Mahoney [EMAIL PROTECTED] wrote: Andrii (lOkadin) Zvorygin [EMAIL PROTECTED] wrote: I personally don't understand why everyone seems to insist on using ambiguous illogical languages to express things when there are viable alternative available. I think because an AGI needs to communicate in languages that people already know. I don't understand how artificial languages like Lojban contribute to this goal. We should focus our efforts instead on learning and modeling existing languages. I understand that artificial languages like Lojban and Esperanto and Attempto have simple grammars. I don't believe they would stay that way if they were widely used for person to person communication (as opposed to machine interfaces). Lojban grammar is easily extensible and forwards compatible. You can add features to the language through CMAvo and GISmu. Lojban already exceeds many natural languages in it's ability to express. There are very crucial parts of communication that English lacks such as logical connectives and attitudinals. Languages evolve over time, both in individuals, and more slowly in social groups. Are you implying languages evolve faster in individuals? A language model is not a simple set of rules. A natural language model is not. An artificial language is constructed with rules that were also created by individual -- as opposed to groups of -- humans. Lojban was especially designed to be logical, unlike Esperanto. Therefore making them recreatable by individual humans, and depending on your definition: simple. It is a probability distribution described by a large set of patterns such as words, word associations, grammatical structures and sentences. The approach of a world of blind to seeing is to feel at things. Sometimes they wonder if there is not another way. Each time you read or hear a message, the probabilities for the observed patterns are increased a little and new patterns are added. In a social setting, these probabilities tend to converge by consensus as this knowledge is shared. I
Re: [agi] Understanding Natural Language
On Friday 24 November 2006 06:03, YKY (Yan King Yin) wrote: You talked mainly about how sentences require vast amounts of external knowledge to interpret, but it does not imply that those sentences cannot be represented in (predicate) logical form. Substitute bit string for predicate logic and you'll have a sentence that is just as true and not a lot less useful. I think there should be a working memory in which sentences under attention would bring up other sentences by association. For example if a person is being kicked is in working memory, that fact would bring up other facts such as being kicked causes a person to feel pain and possibly to get angry, etc. All this is orthogonal to *how* the facts are represented. Oh, I think the representation is quite important. In particular, logic lets you in for gazillions of inferences that are totally inapropos and no good way to say which is better. Logic also has the enormous disadvantage that you tend to have frozen the terms and levels of abstraction. Actual word meanings are a lot more plastic, and I'd bet internal representations are damn near fluid. What you have described is how facts in working memory invoke other facts, to form a complex scenario. This is what classical AI calls frames, I call it working memory. As Ben pointed out, one of the major challenges in AGI is how to control vast amounts of facts that follow from or associate with the current facts, What Minsky said was the more important part of his notion than frames, were what he called frame-arrays in the early papers (I think he adopted some other name like frame-systems later). A frame-array is like a movie with frames for the, ah, frames. It can represent what you see as you turn in a room, or what happens as you watch a fight. If you look up and down in the room, the array may be 2-D; given other actions it may be n-D. What Minsky doesn't understand, for my money, is that the brain has enough oomph to have the equivalent of a fairly substantial processor for every frame-array in memory, so they can all be comparing themselves to the item of attention all the time. Given that, you can produce a damn good predictive model with (a) a representation that allows you to interpolate in some appropriate space between frames, and (b) enough experience to have remembered arrays in the vicinity of the actual experience you're trying to extrapolate. Then take the weighted average of the arrays in the neighborhood of the given experience that best approximates it, which gives you a model for how it will continue. The open questions are representation -- I'm leaning towards CSG in Hilbert spaces at the moment, but that may be too computationally demanding -- and how to form abstractions. As I noted in the original essay, a key need is to be able to do interpolation not only between situations at the same levels, but between levels as well. --Josh - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: Re: [agi] Understanding Natural Language
Oh, I think the representation is quite important. In particular, logic lets you in for gazillions of inferences that are totally inapropos and no good way to say which is better. Logic also has the enormous disadvantage that you tend to have frozen the terms and levels of abstraction. Actual word meanings are a lot more plastic, and I'd bet internal representations are damn near fluid. Logic is a highly generic term ... I agree with your statement re crisp predicate logic as typically utilized, but uncertain term logic does provide guidance regarding which inferences are apropos It also however gets rid of the elegance and compactness that YKY likes: an uncertain logic representation of a simple sentence may involve tens of thousands of contextal, uncertain relationships, possibly including the obvious ones involved in the standard crisp predicate logic representation... -- Ben G - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: Re: [agi] Understanding Natural Language
It was a true solar-plexus blow, and completely knocked out, Perkins
staggered back against the instrument-board. His outflung arm pushed the
power-lever out to its last notch, throwing full current through the
bar, which was pointed straight up as it had been when they made their
landing.
LOJban: zo'e seDARxi foloMIDjubeloCUTne gi'e CIRko leVACri leFEPri
Prolog:
gihe(['fa'],darxi(_,_,_,lo('MIDju',[be(lo('CUTne'))])),cirko(_,le('VACri'),le('FEPri'))).
English: unknown is hit at locus that which really is the middle of
the chest and unknown is a loser of air at/near lungs.
LOJban: .i la.PERkinz. FALdzu seka'aleTUTcizeiTANbo
Prolog: gihe(['fa'],faldzu(la('PERkinz')), klama(_,le(zei('TUTci','TANbo'
English: Perkins (fell kind of walked, I don't know what stagger
really means, dictionary didn't help much, walking with uncertainty?
Probably not the intended meaning here.) (with destination) tool kind
of (board/plank).
DARxi dax da'i hit
x1 hits/strikes/[beats] x2 with instrument [or
body-part] x3 at locus x4
CIRko cri lose
x1 loses person/thing x2 at/near x3; x1 loses
property/feature x2 in conditions/situation x3
I hope Ben Goertzel has introduced you to Lojban already -- I haven't
checked the logs but saw his lojbanplusplus proposal.
I personally don't understand why everyone seems to insist on using
ambiguous illogical languages to express things when there are viable
alternative available. The mass can get it's English translation
rendered out of a Logical language very easily. It's harder to make a
translation function from English to Lojban, than one from Lojban to
English. Though it is possible and I'm sure it will be done as that
would mean we could have a Prolog database of facts representing any
english text. Such as say the book you were refering to.
I'm currently working on a Lojban parser in Prolog. I've just recently
started learning Prolog though the program is going rather well that
considered. I currently have support for CMAvo(read grammar word(s))
and GISmu(read root word(s)), in the example I gave I also used a
LUJvo(read compound word(s)) and CMEne(read name(s)) all of wthich
I have support for in my Haskell(with Parsec) Lojban Parser(there is
one more Lojban word class fu'ivla for foreign words, I'll get support
for that as well). I started coding the Prolog parser maybe a week
ago, so it should be able to parse all Lojban text before the new
year.
Well after that there will be a pretty trivial design phase of the
Lojban-to-Prolog translator, which will support direct conversion.
Then a rewrite of the Lojban-to-Prolog parser and translator in
Lojban. We'll have the first ever programming language that is used
for human-to-human interaction (I use it everyday) http://lojban.org.
mibaziKLAma(I short time interval future am goer)
.imu'o(over to you)mi'e.LOkadin.(my name is Lokadin.)
On 11/24/06, Ben Goertzel [EMAIL PROTECTED] wrote:
Oh, I think the representation is quite important. In particular, logic lets
you in for gazillions of inferences that are totally inapropos and no good
way to say which is better. Logic also has the enormous disadvantage that you
tend to have frozen the terms and levels of abstraction. Actual word meanings
are a lot more plastic, and I'd bet internal representations are damn near
fluid.
Logic is a highly generic term ...
I agree with your statement re crisp predicate logic as typically
utilized, but uncertain term logic does provide guidance regarding
which inferences are apropos It also however gets rid of the
elegance and compactness that YKY likes: an uncertain logic
representation of a simple sentence may involve tens of thousands of
contextal, uncertain relationships, possibly including the obvious
ones involved in the standard crisp predicate logic representation...
-- Ben G
-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303
--
ta'o(by the way) We With You Network at: http://lokiworld.org .i(and)
more on Lojban: http://lojban.org
mu'oimi'e lOkadin (Over, my name is lOkadin)
-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303
Re: [agi] Understanding Natural Language
Excellent. Summarizing: the idea of understanding something (in this case a fragment of (written) natural language) involves many representations being constructed on many levels simultaneously (from word recognition through syntactic parsing to story-archetype recognition). There is nothing in this picture to indicate that understanding is a simple process, or that it can be defined in terms of a single optimization function. When I see attempts to capture the meaning of understanding in an expansive way, like the one you have laid out here (involving a catalog of many different processes that work cooperatively to give the whole), I feel like we are making progress. On the other hand, when I hear people trying to narrow down a definition to some canonical core like an agent that can find the most optimal actions to manipulate the world in the service of its goals (or some other nonsense of that sort), I feel like I am listening to content-free gibberish. Definitions like that use words in such a way as to say nothing. Josh's elaboration of some of the stuff that goes on in the understanding of a paragraph reminds me of a book that I have mentioned here before (Levelt's Speaking) in which the author takes apart a single conversational exchange consisting of a couple of short sentences. Richard Loosemore J. Storrs Hall, PhD. wrote: It was a true solar-plexus blow, and completely knocked out, Perkins staggered back against the instrument-board. His outflung arm pushed the power-lever out to its last notch, throwing full current through the bar, which was pointed straight up as it had been when they made their landing. This is an essay about natural language understanding. The above quote is from the middle of a book, and what I intend to do is to unravel the process by which someone reading the book could be said to understand it. Largely the concern is about what kind of mental structures are being built and what structures must have been built by reading the previous half of the book for the passage to do what it does in the mind of the reader. Without further ado, let us jump into the quote, which starts at the beginning of a paragraph: It was a true solar-plexus blow, There are two sources for the comprehension of this clause. First is the preceding paragraph, where a fight is described. A scene and script have been built up, like a movie in the mind. In particular, one man is holding a girl (who is struggling to escape) and another is trying to tie her feet. She kicks the second man, and that is the blow that's being referred to. Unlike a cinematic movie, however, much that would be evident on the screen has been left out. The specific positions of the bodies, the clothing in some cases, and many aspects of the background have been left to the imagination. In other words, the movie is a sequence of *abstractions.* It is in no sense simply a pile of predicates, however. When I read this, I come away with a semi-visual motion script, such as could be used to orchestrate a re-enactment by action-figure dolls, even though the text doesn't come close to specifying the actual positions or motions I imagine. The second source is the reader's memories of pertinent experiences, either of watching fights or having been in them. In the multi-level abstraction structure that's being built, by and large, at least in the hands of a skillful writer, the things that get mentioned are the things that you'd pay attention to if watching the scene. It's well established, for example in studies of eyewitness acounts in criminology, that people confabulate what happens between such points in their memory of actual events, much less from verbal stories. So to that extent, the structure of a story reflects that of memory. If you've ever taken a hard blow to the solar plexus, you'll have a much deeper understanding of this passage than someone who hasn't. I have, and the sensation is unique; nothing else in my experience feels the same or has the same effects. If you have, note that among the few descriptions of clothing that were provided was that the girl was wearing riding boots... At a higher level, the scene is part of an attempted abduction of the girl by the men. On this level, the reader is on tenterhooks to discover whether the abduction will succeed, given the girl's spirited and at least partially efficacious resistance. and completely knocked out, Syntactically, this is a bit of a garden path; we expect it to be a conjunct of the previous predicate until we see the comma. This phrase appears to be there for those readers who have not experienced solar plexus blows. It describes the effect well enough to follow the action sensibly, but doesn't really capture the experience. This points out that there can be different amounts of actual understanding going on in different readers each of whom would claim to have understood the passage: there can be ties to
