Re: [agi] Natural versus formal AI interface languages
On 11/9/06, Eric Baum [EMAIL PROTECTED] wrote: It is true that much modern encryption is based on simple algorithms. However, some crypto-experts would advise more primitive approaches. RSA is not known to be hard, even if P!=NP, someone may find a number-theoretic trick tomorrow that factors. (Or maybe they already have it, and choose not to publish). If you use a mess machine like a modern version of enigma, that is much less likely to get broken, even though you may not have the theoretical results. DES is essentially a big messy bit-scrambler; like Enigma, but with bits instead of letters. The relative security of the two approaches is debated by cryptologists. On one hand, RSA could be broken by a computational trick (or a quantum computer). On the other hand, DES is so messy that it's very hard to be sure there isn't a foothold for an attack, or even a deliberate backdoor, in it. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
Ben Goertzel wrote: It's just that problem X is NP-hard means roughly Any problem Y in NP is polynomial-time reducible to problem X, and your example did not seem to exemplify this... All your example seemed to exemplify was a problem that was solvable in polynomial time (class P, not class NP-hard) However, this is irrelevant to your main conceptual point, which as I understood it was that theorems regarding the scaling behavior of the worst-case complexity of problems as problem size n goes to infinity are pragmatically irrelevant... [I'm not sure I fully agree with your conceptual point, but that's another issue. I used to agree but when I encountered Immerman's descriptive complexity theory, I started wavering. Immerman showed e.g. that -- P, the class of problems solvable in polynomial time, corresponds to languages recognizable by first-order logic plus a recursion operator -- NP, the class of problems whose solutions are checkable in polynomial time, corresponds to languages recognized by existential second order logic (second order logic with second-order existential but not universal quantification) This is interesting and suggests that these complexity classes could possibly have some fundamental cognitive meaning, even though such a meaning is not obvious from their standard definitions...] Your point is well taken: I did fudge the issue by giving an example that was a specific, polynomial instance and made the mistake of calling it NP-Hard. My goal (as you correctly point out) was to try to make it clear that NP-Hardness statements are not about how hard a given language mechanism is to build, but about the scaling behavior. I don't really want to get too sidetracked, but even if Immerman's analysis were correct, would this make a difference to the way that Eric was using NP-Hard, though? In other words, this would still not undermine my point that a statement like the building of a language /learning mechanism by evolution is NP-Hard does not actually tell us anything about how difficult the particular process that led to human language really was? It sounds like Immerman is putting the significance of complexity classes on firmer ground, but not changing the nature of what they are saying. Richard Loosemore -- Ben On 11/24/06, Richard Loosemore [EMAIL PROTECTED] wrote: Ben Goertzel wrote: Richard, I know it's peripheral to your main argument, but in this example ... Suppose that the computational effort that evolution needs to build different sized language understanding mechanisms scales as: 2.5 * (N/7 + 1)^^6 planet-years ... where different sized is captured by the value N, which is the number of conceptual primitives used in the language understanding mechanism, and a planet-year is one planet worth of human DNA randomly working on the problem for one year. (I am plucking this out of the air, of course, but that doesn't matter.) Here are the resource requirements for this polynomial resource function: N R 1 2.23E+000 7 6.40E+001 10 2.05E+002 50 2.92E+005 100 1.28E+007 300 7.12E+009 (N = Number of conceptual primitives) (R = resource requirement in planet-years) I am assuming that the appropriate measure of size of problem is number of conceptual primitives that are involved in the language understanding mechanism (a measure picked at random, and as far as I can see, as likely a measure as any, but if you think something else should be the N, be my guest). If there were 300 conceptual primitives in the human LUM, resource requirement would be 7 billion planet-years. That would be bad. But if there are only 7 conceptual primitives, it would take 64 years. Pathetically small and of no consequence. The function is polynomial, so in a sense you could say this is an NP-hard problem. I don't think you're using the term NP-hard correctly. http://en.wikipedia.org/wiki/Complexity_classes_P_and_NP The class P consists of all those decision problems that can be solved on a deterministic sequential machine in an amount of time that is polynomial in the size of the input; the class NP consists of all those decision problems whose positive solutions can be **verified** in polynomial time given the right information. [This page also reviews, and agrees with, many of your complaints regarding the intuitive interpretation of P as easy and NP as hard] http://en.wikipedia.org/wiki/NP-hard In computational complexity theory, NP-hard (Non-deterministic Polynomial-time hard) refers to the class of decision problems H such that for every decision problem L in NP there exists a polynomial-time many-one reduction to H, written . If H itself is in NP, then H is called NP-complete. I'd certainly welcome clarification, and I may have gotten this wrong... but I'm not quite sure where you are
Re: [agi] Natural versus formal AI interface languages
Eric Baum wrote: The argument, in very brief, is the following. Evolution found a very compact program that does the right thing. (This is my hypothesis, not claimed proved but lots of reasons to believe it given in WIT?.) Finding such programs is NP-hard. Richard Hold it right there. As far as I can see, you just asserted Richard the result that is under dispute, right there at the Richard beginning of your argument! First, above I was discussing finding an understanding system, not necessarily a language understanding system-- say a monkey, not a person. Then I went on to talk about additional problems coming when you want language. Richard Finding a language-understanding mechanism is NP-hard? Richard That prompts two questions: Richard 1) Making statements about NP-hardness requires a problem to Richard be formalized in such a way as to do the math. But in order Richard to do that formalization you have to make assumptions, and Richard the only assumptions I have ever seen reported in this Richard context are close relatives of the ones that are under Richard dispute (that grammar induction is context free, Richard essentially), and if you have made those assumptions, you Richard have assumed what you were trying to demonstrate! At this point, I suggest again you read What is Thought? In emails, I am skipping a lot of corners and not giving caveats and whatnot, to give 3 paragraph summaries. But if you don't want to take the time, I'll cut to the chase. I am not claiming to have proved that building a human is NP-hard. Eric, I am having serious difficulty here: I made a very specific point, originally, and you made a reply to that - but your replies are wandering off into other topics. For example: I neither thought nor implied that your claim was to have proved that building a human is NP-hard, so I am puzzled why you should say this. I am suggesting a theory of thought, for which I think there is a lot of evidence and basis. Computational learning theory has more or less established that generalization (never mind thought) follows from finding constrained-enough hypothesis. And in every case that has been studied of sufficient richness to be really interesting, it turns out that the problems you have to solve are NP-hard. So naturally, in my extrapolation to thought, I expect that the problems you will have to solve here are NP-hard as well. This isn't exactly rigorous, but its the way to bet. Your protests are mostly wishful thinking. Its also true that proving something is NP-hard doesn't prove its insoluble, or even hard in the average case, or hard in any particular case, or any of that. Hell, it might even be true that P=NP. But there are a lot of strong reasons to think that all of this is also wishful thinking, and that NP-hard problems are really hard. I'll say again, if you don't believe that, you shouldn't be using cryptography, because cryptography as practiced not only relies on P!=NP, but much much stronger assumptions, like its very hard to factor *random* products of primes, and factoring isn't even NP-hard. Its clear from what you are writing here that you are not familiar with computational learning theory, or computational complexity theory. I strongly suggest you read WIT?. I think you will learn a lot. [Please don't be tempted to make comments about my general level of expertise in this or that field. You are mistaken in this, but I am not going to argue about it]. My understanding of these areas is reasonably deep, though not complete, and I have made specific points that (if you will read Pei Wang's comment) are being reinforced by others more expert than myself. As for COLT, you have said something that I completely disagree with: Computational learning theory has more or less established that generalization (never mind thought) follows from finding constrained-enough hypothesis. That would be true if you accepted the narrow characterization of generalization that is used in COLT, but it is most emphatically not true if (as I do) you consider this narrow reading to be only a trivial version of the mechanism to be found in the human cognitive system. This is an absolutely crucial point. The COLT people have decided to use the word generalization to describe a formal process that THEY define, and the reason they define it their way is that their narrow definition makes it amenable to mathematical proofs (e.g. proofs that THEIR version is an NP-Hard problem). Frankly, I don't care if they can prove that their version is NP-Hard, because nothing follows from it. Other people do not take that narrow view, but instead consider generalization to be a much more complicated process (probably a cluster of several processes) defined on a system that is not nearly as simple in structure as the systems that COLT people use . and for all these reasons, there is no way to get a handle on this larger version of
Re: Re: [agi] Natural versus formal AI interface languages
Hi Richard, I don't really want to get too sidetracked, but even if Immerman's analysis were correct, would this make a difference to the way that Eric was using NP-Hard, though? No, Immerman's perspective on complexity classes doesn't really affect your objections... Firstly, the descriptive logic depiction of complexity classes is **still** about what happens as n gets large. So it doesn't affect one of the key objections that both you and Pei have to using concepts from computational complexity theory to analyze AGI: which is that AGI systems don't have to deal with general classes of problems of problem size tending to infinity, they have to deal with **particular** problems of bounded size. For instance, if an AGI is good at learning **human** language, it may not matter how its language learning capability scales when dealing with languages falling into the same grammar category as human language whose grammars have sizes tending to infinity. If an AGI is good at solving path-finding problems in real life, it may not matter how its worst-case path-finding capability scales when dealing with paths between n cities as n tends to infinity Etc. In fact there are decent qualitative arguments that most of the algorithms used by human cognition (insofar as it makes sense to say that human cognition uses algorithms, which is another issue, as Pei has noted) are **exponential time** in terms of their scaling as problem size approaches infinity ... but the point is that they are tuned to give tractable performance for the problem-instances that humans generally encounter in real life... Secondly, Immerman's analysis doesn't affect the fact that the formalization of language learning referred to by Eric Baum is only tenuously related to the actual cognitive phenomenon of human language learning. On the other hand, Immerman's analysis does **suggest** (not demonstrate) that there could be some cognitive meaningfulness to the classes P and NP. For instance, if someone were to show that the learning of languages in the same general category as human natural languages (natural-like languages)... -- can be naturally represented using existential second-order logic but -- cannot be naturally represented using first-order logic with recursion this would be interesting, and would match up naturally with the observation that natural-like language learning is NP but not P. On the other hand, this kind of analysis would only be really cognitively meaningful in the context of an explanation of how this formalization of language learning is related to actual cognitive language learning I happen to think that such an explanation **could* be formulated; but no one has really done so, so far. That is, no one has given a formalization encompassing the embodied, social semantics and pragmatics of language learning (as discussed e.g. in Tomassello's excellent recent book Constructing a Language); and in the absence of such a formalization, formal discussions of grammar learning are not convincingly connected to real cognitive language learning. -- Ben - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
Richard Eric Baum wrote: I don't think the proofs depend on any special assumptions about the nature of learning. I beg to differ. IIRC the sense of learning they require is induction over example sentences. They exclude the use of real world knowledge, in spite of the fact that such knowledge (or at least primitives involved in the development of real world knowledge ) are posited to play a significant role in the learning of grammar in humans. As such, these proofs say nothing whatsoever about the learning of NL grammars. I fully agree the proofs don't take into account such stuff. And I believe such stuff is critical. Thus I've never claimed language learning was proved hard, I've just suggested evolution could have encrypted it. The point I began with was, if there are lots of different locally optimal codings for thought, it may be hard to figure out which one is programamed into the mind, and thus language learning could be a hard additional problem to producing an AGI. The AGI has to understand what the word foobar means, and thus it has to have (or build) a code module meaning ``foobar that it can invoke with this word. If it has a different set of modules, it might be sunk in communication. My sense about grammars for natural language, is that there are lots of different equally valid grammars that could be used to communicate. For example, there are the grammars of English and of Swahili. One isn't better than the other. And there is a wide variety of other kinds of grammars that might be just as good, that aren't even used in natural language, because evolution chose one convention at random. Figuring out what that convention is, is hard, at least Linguists have tried hard to do it and failed. And this grammar stuff is pretty much on top of, the meanings of the words. It serves to disambiguate, for example for error correction in understanding. But you could communicate pretty well in pidgin, without it, so long as you understand the meanings of the words. The grammar learning results (as well as the experience of linguists, who've tried very hard to build a model for natural grammar) I think, are indicative that this problem is hard, and it seems that this problem is superimposed above the real world knowledge aspect. Richard Eric, Richard Thankyou, I think you have focussed down on the exact nature Richard of the claim. Richard My reply could start from a couple of different places in Richard your above text (all equivalent), but the one that brings out Richard the point best is this: And there is a wide variety of other kinds of grammars that might be just as good, that aren't even used in natural language, because evolution chose one convention at random. Richard Richard ^^ Richard This is precisely where I think the flase assumption is Richard buried. When I say that grammar learning can be dependent on Richard real world knowledge, I mean specifically that there are Richard certain conceptual primitives involved in the basic design of Richard a concept-learning system. We all share these primitives, Richard and [my claim is that] our language learning mechanisms start Richard from those things. Because both I and a native Swahili Richard speaker use languages whose grammars are founded on common Richard conceptual primitives, our grammars are more alike than we Richard imagine. Richard Not only that, but if myself and the Swahili speaker suddenly Richard met and tried to discover each other's languages, we would be Richard able to do so, eventually, because our conceptual primitives Richard are the same and our learning mechanisms are so similar. Richard Finally, I would argue that most cognitive systems, if they Richard are to be successful in negotiating this same 3-D universe, Richard would do best to have much the same conceptual primitives Richard that we do. This is much harder to argue, but it could be Richard done. Richard As a result of this, evolution would not by any means have Richard been making random choices of languages to implement. It Richard remains to be seen just how constrained the choices are, but Richard there is at least a prima facie case to be made (the one I Richard just sketched) that evolution was extremely constrained in Richard her choices. Richard In the face of these ideas, your argument that evolution Richard essentially made a random choice from a quasi-infinite space Richard of possibilities needs a great deal more to back it up. The Richard grammar-from-conceptual-primitives idea is so plausible that Richard the burden is on you to give a powerful reason for rejecting Richard it. Richard Correct me if I am wrong, but I see no argument from you on Richard this specific point (maybe there is one in your book but Richard in that case, why say without qualification, as if it was Richard obvious, that evolution made a random selection?). Richard Unless you can destroy the
Re: Re: [agi] Natural versus formal AI interface languages
Richard, I know it's peripheral to your main argument, but in this example ... Suppose that the computational effort that evolution needs to build different sized language understanding mechanisms scales as: 2.5 * (N/7 + 1)^^6 planet-years ... where different sized is captured by the value N, which is the number of conceptual primitives used in the language understanding mechanism, and a planet-year is one planet worth of human DNA randomly working on the problem for one year. (I am plucking this out of the air, of course, but that doesn't matter.) Here are the resource requirements for this polynomial resource function: N R 1 2.23E+000 7 6.40E+001 10 2.05E+002 50 2.92E+005 100 1.28E+007 300 7.12E+009 (N = Number of conceptual primitives) (R = resource requirement in planet-years) I am assuming that the appropriate measure of size of problem is number of conceptual primitives that are involved in the language understanding mechanism (a measure picked at random, and as far as I can see, as likely a measure as any, but if you think something else should be the N, be my guest). If there were 300 conceptual primitives in the human LUM, resource requirement would be 7 billion planet-years. That would be bad. But if there are only 7 conceptual primitives, it would take 64 years. Pathetically small and of no consequence. The function is polynomial, so in a sense you could say this is an NP-hard problem. I don't think you're using the term NP-hard correctly. http://en.wikipedia.org/wiki/Complexity_classes_P_and_NP The class P consists of all those decision problems that can be solved on a deterministic sequential machine in an amount of time that is polynomial in the size of the input; the class NP consists of all those decision problems whose positive solutions can be **verified** in polynomial time given the right information. [This page also reviews, and agrees with, many of your complaints regarding the intuitive interpretation of P as easy and NP as hard] http://en.wikipedia.org/wiki/NP-hard In computational complexity theory, NP-hard (Non-deterministic Polynomial-time hard) refers to the class of decision problems H such that for every decision problem L in NP there exists a polynomial-time many-one reduction to H, written . If H itself is in NP, then H is called NP-complete. -- Ben G - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
Ben Goertzel wrote: Richard, I know it's peripheral to your main argument, but in this example ... Suppose that the computational effort that evolution needs to build different sized language understanding mechanisms scales as: 2.5 * (N/7 + 1)^^6 planet-years ... where different sized is captured by the value N, which is the number of conceptual primitives used in the language understanding mechanism, and a planet-year is one planet worth of human DNA randomly working on the problem for one year. (I am plucking this out of the air, of course, but that doesn't matter.) Here are the resource requirements for this polynomial resource function: N R 1 2.23E+000 7 6.40E+001 10 2.05E+002 50 2.92E+005 100 1.28E+007 300 7.12E+009 (N = Number of conceptual primitives) (R = resource requirement in planet-years) I am assuming that the appropriate measure of size of problem is number of conceptual primitives that are involved in the language understanding mechanism (a measure picked at random, and as far as I can see, as likely a measure as any, but if you think something else should be the N, be my guest). If there were 300 conceptual primitives in the human LUM, resource requirement would be 7 billion planet-years. That would be bad. But if there are only 7 conceptual primitives, it would take 64 years. Pathetically small and of no consequence. The function is polynomial, so in a sense you could say this is an NP-hard problem. I don't think you're using the term NP-hard correctly. http://en.wikipedia.org/wiki/Complexity_classes_P_and_NP The class P consists of all those decision problems that can be solved on a deterministic sequential machine in an amount of time that is polynomial in the size of the input; the class NP consists of all those decision problems whose positive solutions can be **verified** in polynomial time given the right information. [This page also reviews, and agrees with, many of your complaints regarding the intuitive interpretation of P as easy and NP as hard] http://en.wikipedia.org/wiki/NP-hard In computational complexity theory, NP-hard (Non-deterministic Polynomial-time hard) refers to the class of decision problems H such that for every decision problem L in NP there exists a polynomial-time many-one reduction to H, written . If H itself is in NP, then H is called NP-complete. I'd certainly welcome clarification, and I may have gotten this wrong... but I'm not quite sure where you are directing my attention here. Are you targeting the fact that NP-Hard is defined with respect to decision problems, or to the reduction aspect? My understanding of NP-hard is that it does strictly only apply to decision problems ... but what I was doing was trying to interpret the loose sense in which Eric himself was using NP-Hard, so if I have stretched the definition a little, I woudl claim I was inheriting something that was already stretched. But maybe that was not what you meant. I stand ready to be corrected, if it turns out I have goofed. Richard Loosemore. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
The primitive terms arent random, just some of the structure of it. English standard does Sub VB Ob, while others do VB Subj Ob or another manner, as long as they are known and roughly consistently used, the actual choice coudl well be random there and not matter, but a 'concept' of a dog in any language is roughly the same, based on what we share when we see, hear, smell, and interact with the concept. Everything is based on these primitives of experiencing the world, so I am using english, but modeling my knowledge base terms on these experiences. James Richard Loosemore [EMAIL PROTECTED] wrote: Eric Baum wrote: I don't think the proofs depend on any special assumptions about the nature of learning. I beg to differ. IIRC the sense of learning they require is induction over example sentences. They exclude the use of real world knowledge, in spite of the fact that such knowledge (or at least knowledge) are posited to play a significant role in the learning of grammar in humans. As such, these proofs say nothing whatsoever about the learning of NL grammars. I fully agree the proofs don't take into account such stuff. And I believe such stuff is critical. Thus I've never claimed language learning was proved hard, I've just suggested evolution could have encrypted it. The point I began with was, if there are lots of different locally optimal codings for thought, it may be hard to figure out which one is programamed into the mind, and thus language learning could be a hard additional problem to producing an AGI. The AGI has to understand what the word foobar means, and thus it has to have (or build) a code module meaning ``foobar that it can invoke with this word. If it has a different set of modules, it might be sunk in communication. My sense about grammars for natural language, is that there are lots of different equally valid grammars that could be used to communicate. For example, there are the grammars of English and of Swahili. One isn't better than the other. And there is a wide variety of other kinds of grammars that might be just as good, that aren't even used in natural language, because evolution chose one convention at random. Figuring out what that convention is, is hard, at least Linguists have tried hard to do it and failed. And this grammar stuff is pretty much on top of, the meanings of the words. It serves to disambiguate, for example for error correction in understanding. But you could communicate pretty well in pidgin, without it, so long as you understand the meanings of the words. The grammar learning results (as well as the experience of linguists, who've tried very hard to build a model for natural grammar) I think, are indicative that this problem is hard, and it seems that this problem is superimposed above the real world knowledge aspect. Eric, Thankyou, I think you have focussed down on the exact nature of the claim. My reply could start from a couple of different places in your above text (all equivalent), but the one that brings out the point best is this: And there is a wide variety of other kinds of grammars that might be just as good, that aren't even used in natural language, because evolution chose one convention at random. ^^ This is precisely where I think the flase assumption is buried. When I say that grammar learning can be dependent on real world knowledge, I mean specifically that there are certain conceptual primitives involved in the basic design of a concept-learning system. We all share these primitives, and [my claim is that] our language learning mechanisms start from those things. Because both I and a native Swahili speaker use languages whose grammars are founded on common conceptual primitives, our grammars are more alike than we imagine. Not only that, but if myself and the Swahili speaker suddenly met and tried to discover each other's languages, we would be able to do so, eventually, because our conceptual primitives are the same and our learning mechanisms are so similar. Finally, I would argue that most cognitive systems, if they are to be successful in negotiating this same 3-D universe, would do best to have much the same conceptual primitives that we do. This is much harder to argue, but it could be done. As a result of this, evolution would not by any means have been making random choices of languages to implement. It remains to be seen just how constrained the choices are, but there is at least a prima facie case to be made (the one I just sketched) that evolution was extremely constrained in her choices. In the face of these ideas, your argument that evolution essentially made a random choice from a quasi-infinite space of possibilities needs a great deal more to back it up. The grammar-from-conceptual-primitives idea is so plausible that
Re: Re: [agi] Natural versus formal AI interface languages
Sorry for my delay in responding... too busy to keep up with most of this, just got some downtime and scanning various messages: I don't know what you mean by incrementally updateable, but if you look up the literature on language learning, you will find that learning various sorts of relatively simple grammars from examples, or even if memory serves examples and queries, is NP-hard. Try looking for Dana Angluin's papers back in the 80's. No, a thousand times no. (Oh, why do we have to fight the same battles over and over again?) These proofs depend on assumptions about what learning is, and those assumptions involve a type of learning that is stupider than stupid. Ben I don't think the proofs depend on any special assumptions about Ben the nature of learning. Ben Rather, the points to be noted are: Ben 1) these are theorems about the learning of general grammars in a Ben certain class, as n (some measure of grammar size) goes to Ben infinity Ben 2) NP-hard is about worst-case time complexity of learning Ben grammars in that class, of size n These comments are of course true of any NP-hardness result. They are reasons why the NP-hardness result does not *prove* (even if P!=NP) that the problem is insuperable. However, the way to bet is generally that the problem is actually hard. Ch. 11 of WIT? gives some arguments why. If you don't believe that, you shouldn't rely on encryption. Encryption has all the above weaknesses in spades, and plus, its not even proved secure given P!=NP, that requires additional assumptions. Also, in addition to the hardness results, there has been considerable effort in modelling natural grammars by linguists, which has failed, thus also providing evidence the problem is hard. Ben So the reason these results are not cognitively interesting is: Ben 1) real language learning is about learning specific grammars of Ben finite size, not parametrized classes of grammars as n goes to Ben infinity Ben 2) even if you want to talk about learning over parametrized Ben classes, real learning is about average-case rather than Ben worst-case complexity, anyway (where the average is over some Ben appropriate probability distribution) Ben -- Ben G Any learning mechanism that had the ability to do modest analogy building across domains, and which had the benefit of primitives involving concepts like on, in, through, manipulate, during, before (etc etc) would probably be able to do the grammer learning, and in any case, the proofs are completely incapable of representing the capabilities of such learning mechanisms. Such ideas have been (to coin a phrase) debunked every which way from sunday. ;-) Richard Loosemore Ben - This list is sponsored by AGIRI: http://www.agiri.org/email Ben To unsubscribe or change your options, please go to: Ben http://v2.listbox.com/member/?list_id=303 - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
Eric Baum wrote: Sorry for my delay in responding... too busy to keep up with most of this, just got some downtime and scanning various messages: I don't know what you mean by incrementally updateable, but if you look up the literature on language learning, you will find that learning various sorts of relatively simple grammars from examples, or even if memory serves examples and queries, is NP-hard. Try looking for Dana Angluin's papers back in the 80's. No, a thousand times no. (Oh, why do we have to fight the same battles over and over again?) These proofs depend on assumptions about what learning is, and those assumptions involve a type of learning that is stupider than stupid. Ben I don't think the proofs depend on any special assumptions about Ben the nature of learning. Ben Rather, the points to be noted are: Ben 1) these are theorems about the learning of general grammars in a Ben certain class, as n (some measure of grammar size) goes to Ben infinity Ben 2) NP-hard is about worst-case time complexity of learning Ben grammars in that class, of size n These comments are of course true of any NP-hardness result. They are reasons why the NP-hardness result does not *prove* (even if P!=NP) that the problem is insuperable. However, the way to bet is generally that the problem is actually hard. Ch. 11 of WIT? gives some arguments why. If you don't believe that, you shouldn't rely on encryption. Encryption has all the above weaknesses in spades, and plus, its not even proved secure given P!=NP, that requires additional assumptions. Also, in addition to the hardness results, there has been considerable effort in modelling natural grammars by linguists, which has failed, thus also providing evidence the problem is hard. Eric, You quoted Ben above and addressed part 2 of his response, without noticing that he later retracted part 1 (I don't think the proofs depend on any special assumptions about the nature of learning.) and therefore, because of that retraction, made the part 2 points irrelevant to the argument we were discussing. The result of all that is that your own comments, above, are also stranded out on that irrelevant subbranch, because I have already pointed out that all the efforts of the linguists and others who talk about grammar learning are *indeed* making special assumptions about the nature of language learning that are extremely unlikely to be valid. The result: you cannot make any sensible conclusions about the hardness of the grammar learning task. Here is my previous response to Ben's points that you quote above, together with his reply: Ben Goertzel wrote: I don't think the proofs depend on any special assumptions about the nature of learning. Richard Loosemore wrote: I beg to differ. IIRC the sense of learning they require is induction over example sentences. They exclude the use of real world knowledge, in spite of the fact that such knowledge (or at least primitives involved in the development of real world knowledge) are posited to play a significant role in the learning of grammar in humans. As such, these proofs say nothing whatsoever about the learning of NL grammars. I agree they do have other limitations, of the sort you suggest below. Ben Goertzel wrote: Ah, I see Yes, it is true that these theorems are about grammar learning in isolation, not taking into account interactions btw semantics, pragmatics and grammar, for example... ben As I said before, since your arguments are based on these same assumptions, your claims about the learnability of grammars are completely spurious. If you can show an analysis that includes the impact of real world knowledge on the learning mechanism, and prove that the grammar learning problem is still hard, you might be able to come to the conclusions you do, but I have never seen anyone show the remotest signs of being able to do that. Richard Loosemore - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
Eric, can you give an example of a one way function (such as a cryptographic hash or cipher) produced by evolution or by a genetic algorithm? A one-way function f has the property that y = f(x) is easy to compute, but it is hard to find x given f and y. Other examples might be modular exponentiation in large finite groups, or multiplication of prime numbers with thousands of digits. By incrementally updatable, I mean that you can make a small change to a system and the result will be a small change in behavior. For example, most DNA mutations have a small effect. We try to design software systems with this property so we can modify them without breaking them. However, as the system gets bigger, there is more interaction between components, until it reaches the point where every change introduces more bugs than it fixes and the code becomes unmaintainable. This is what happens when the system crosses the boundary from stability to chaotic. My argument for Kauffman's observation that complex systems sit on this boundary is that stable systems are less useful, but chaotic systems can't be developed as a long sequence of small steps. We are able to produce cryptosystems only because they are relatively simple, and even then it is hard. I don't dispute that learning some simple grammars is NP-hard. However, I don't believe that natural language is one of these grammars. It certainly is not simple. The human brain is less powerful than a Turing machine, so it has no special ability to solve NP-hard problems. The fact that humans can learn natural language is proof enough that it can be done. -- Matt Mahoney, [EMAIL PROTECTED] - Original Message From: Eric Baum [EMAIL PROTECTED] To: agi@v2.listbox.com Sent: Sunday, November 12, 2006 9:29:13 AM Subject: Re: [agi] Natural versus formal AI interface languages Matt wrote: Anyway, my point is that decoding the human genome or natural language is n= ot as hard as breaking encryption. It cannot be because these systems are = incrementally updatable, unlike ciphers. This allows you to use search str= ategies that run in polynomial time. A key search requires exponential tim= e, or else the cipher is broken. Modeling language or the genome in O(n) t= ime or even O(n^2) time with n =3D 10^9 is much faster than brute force cry= ptanalysis in O(2^n) time with n =3D 128. I don't know what you mean by incrementally updateable, but if you look up the literature on language learning, you will find that learning various sorts of relatively simple grammars from examples, or even if memory serves examples and queries, is NP-hard. Try looking for Dana Angluin's papers back in the 80's. If your claim is that evolution can not produce a 1-way function, that's crazy. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303 - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
Eric Baum wrote: Matt wrote: Anyway, my point is that decoding the human genome or natural language is n= ot as hard as breaking encryption. It cannot be because these systems are = incrementally updatable, unlike ciphers. This allows you to use search str= ategies that run in polynomial time. A key search requires exponential tim= e, or else the cipher is broken. Modeling language or the genome in O(n) t= ime or even O(n^2) time with n =3D 10^9 is much faster than brute force cry= ptanalysis in O(2^n) time with n =3D 128. I don't know what you mean by incrementally updateable, but if you look up the literature on language learning, you will find that learning various sorts of relatively simple grammars from examples, or even if memory serves examples and queries, is NP-hard. Try looking for Dana Angluin's papers back in the 80's. No, a thousand times no. (Oh, why do we have to fight the same battles over and over again?) These proofs depend on assumptions about what learning is, and those assumptions involve a type of learning that is stupider than stupid. Any learning mechanism that had the ability to do modest analogy building across domains, and which had the benefit of primitives involving concepts like on, in, through, manipulate, during, before (etc etc) would probably be able to do the grammer learning, and in any case, the proofs are completely incapable of representing the capabilities of such learning mechanisms. Such ideas have been (to coin a phrase) debunked every which way from sunday. ;-) Richard Loosemore - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: Re: [agi] Natural versus formal AI interface languages
I don't know what you mean by incrementally updateable, but if you look up the literature on language learning, you will find that learning various sorts of relatively simple grammars from examples, or even if memory serves examples and queries, is NP-hard. Try looking for Dana Angluin's papers back in the 80's. No, a thousand times no. (Oh, why do we have to fight the same battles over and over again?) These proofs depend on assumptions about what learning is, and those assumptions involve a type of learning that is stupider than stupid. I don't think the proofs depend on any special assumptions about the nature of learning. Rather, the points to be noted are: 1) these are theorems about the learning of general grammars in a certain class, as n (some measure of grammar size) goes to infinity 2) NP-hard is about worst-case time complexity of learning grammars in that class, of size n So the reason these results are not cognitively interesting is: 1) real language learning is about learning specific grammars of finite size, not parametrized classes of grammars as n goes to infinity 2) even if you want to talk about learning over parametrized classes, real learning is about average-case rather than worst-case complexity, anyway (where the average is over some appropriate probability distribution) -- Ben G Any learning mechanism that had the ability to do modest analogy building across domains, and which had the benefit of primitives involving concepts like on, in, through, manipulate, during, before (etc etc) would probably be able to do the grammer learning, and in any case, the proofs are completely incapable of representing the capabilities of such learning mechanisms. Such ideas have been (to coin a phrase) debunked every which way from sunday. ;-) Richard Loosemore - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
Ben Goertzel wrote: I don't know what you mean by incrementally updateable, but if you look up the literature on language learning, you will find that learning various sorts of relatively simple grammars from examples, or even if memory serves examples and queries, is NP-hard. Try looking for Dana Angluin's papers back in the 80's. No, a thousand times no. (Oh, why do we have to fight the same battles over and over again?) These proofs depend on assumptions about what learning is, and those assumptions involve a type of learning that is stupider than stupid. I don't think the proofs depend on any special assumptions about the nature of learning. I beg to differ. IIRC the sense of learning they require is induction over example sentences. They exclude the use of real world knowledge, in spite of the fact that such knowledge (or at least primitives involved in the development of real world knowledge) are posited to play a significant role in the learning of grammar in humans. As such, these proofs say nothing whatsoever about the learning of NL grammars. I agree they do have other limitations, of the sort you suggest below. Richard Loosemore. Rather, the points to be noted are: 1) these are theorems about the learning of general grammars in a certain class, as n (some measure of grammar size) goes to infinity 2) NP-hard is about worst-case time complexity of learning grammars in that class, of size n So the reason these results are not cognitively interesting is: 1) real language learning is about learning specific grammars of finite size, not parametrized classes of grammars as n goes to infinity 2) even if you want to talk about learning over parametrized classes, real learning is about average-case rather than worst-case complexity, anyway (where the average is over some appropriate probability distribution) -- Ben G Any learning mechanism that had the ability to do modest analogy building across domains, and which had the benefit of primitives involving concepts like on, in, through, manipulate, during, before (etc etc) would probably be able to do the grammer learning, and in any case, the proofs are completely incapable of representing the capabilities of such learning mechanisms. Such ideas have been (to coin a phrase) debunked every which way from sunday. ;-) Richard Loosemore - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303 - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: Re: [agi] Natural versus formal AI interface languages
I don't think the proofs depend on any special assumptions about the nature of learning. I beg to differ. IIRC the sense of learning they require is induction over example sentences. They exclude the use of real world knowledge, in spite of the fact that such knowledge (or at least primitives involved in the development of real world knowledge) are posited to play a significant role in the learning of grammar in humans. As such, these proofs say nothing whatsoever about the learning of NL grammars. I agree they do have other limitations, of the sort you suggest below. Ah, I see Yes, it is true that these theorems are about grammar learning in isolation, not taking into account interactions btw semantics, pragmatics and grammar, for example... ben - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
The security of Enigma depended on the secrecy of the algorithm in addition to the key. This violated Kirchoff's principle, the requirement that a system be secure against an adversary who has everything except the key. This mistake has been repeated many times by amateur cryptographers who thought that keeping the algorithm secret improved security. Such systems are invariably broken. Secure systems are built by publishing the algorithm so that people can try to break them before they are used for anything important. It has to be done this way because there is no provably secure system (regardless of whether P = NP), except the one time pad, which is impractical because it lacks message integrity, and the key has to be as large as the plaintext and can't be reused. Anyway, my point is that decoding the human genome or natural language is not as hard as breaking encryption. It cannot be because these systems are incrementally updatable, unlike ciphers. This allows you to use search strategies that run in polynomial time. A key search requires exponential time, or else the cipher is broken. Modeling language or the genome in O(n) time or even O(n^2) time with n = 10^9 is much faster than brute force cryptanalysis in O(2^n) time with n = 128. -- Matt Mahoney, [EMAIL PROTECTED] - Original Message From: Eric Baum [EMAIL PROTECTED] To: agi@v2.listbox.com Sent: Thursday, November 9, 2006 12:18:34 PM Subject: Re: [agi] Natural versus formal AI interface languages Eric Baum [EMAIL PROTECTED] wrote: Matt wrote: Changing one bit of the key or plaintext affects every bit of the cipherte= xt. That is simply not true of most encryptions. For example, Enigma.=20 Matt: Enigma is laughably weak compared to modern encryption, such as AES, RSA, S= HA-256, ECC, etc. Enigma was broken with primitive mechanical computers an= d pencil and paper. Enigma was broken without modern computers, *given access to the machine.* I chose Enigma as an example, because to break language it may be necessary to pay attention to the machine-- namely examining the genomics. But that is more work than you envisage ;^) It is true that much modern encryption is based on simple algorithms. However, some crypto-experts would advise more primitive approaches. RSA is not known to be hard, even if P!=NP, someone may find a number-theoretic trick tomorrow that factors. (Or maybe they already have it, and choose not to publish). If you use a mess machine like a modern version of enigma, that is much less likely to get broken, even though you may not have the theoretical results. Your response admits that for stream ciphers changing a bit of the plaintext doesn't affect many bits of the ciphertext, which was what I was mainly responding to. You may prefer other kinds of cipher, but your arguments about chaos are clearly not germane to concluding language is easy to decode. Incidentally, while no encryption scheme is provably hard to break (even assuming P!=NP) more is known about grammars: they are provably hard to decode given P!=NP. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303 - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
John Fully decoding the human genome is almost impossible. Not only John is there the problem of protein folding, which I think even John supercomputers can't fully solve, but the purpose for the John structure of each protein depends on interaction with the John incredibly complex molecular structures inside cells. Yes, but you have all kinds of advantages in decoding the genome that you don't have, for example, in decoding the human mind (although you might have in an AGI): such as the ability to perform ingenious knockout experiments, comparative genomics, etc. Also, the John genetic code for a human being is basically made of the same John elements that the genetic code for the lowliest single-celled John creature is made of, and yet it somehow describes the initial John structure of a system of neural cells that then developes into a John human brain through a process of embriological growth (which John includes biological interaction from the mother -- why you can't John just grow a human being from an embryo in a petri dish), and John then a fairly long process of childhood development. John This is the way evolution created mind somewhat randomly over John three billion (and a half?) years. The human mind is the John pinnacle of this evolution. With this mind along with collective John intelligence, it shouldn't take another three billion years to John engineer intelligence. Evolution is slow -- human beings can John engineer. Yes, but (a) evolution had vastly more computational power than we did-- it had the ability to use this method to design the brain; and (b) plausible arguments (see What is Thought?) suggest that there may be no better way to design a mind; and (c) the supposition that evolution can't engineer is also unproven. You believe evolution designed us, and we engineer, so in a sense you believe evolution engineers. But I suggest, when we engineer what we basically do is a search over alternatives strongly constrained by knowledge evolution built in, and that the way evolution got to us was similarly by building knowledge that strongly constrained its search, recursively; so in fact it may make considerable sense to say that evolution engineers in basically the same way we do. Why do you think it looks so much like we are designed? John - Original Message - Eliezer S. Yudkowsky wrote: Eric Baum wrote: (Why should producing a human-level AI be cheaper than decoding the genome?) Because the genome is encrypted even worse than natural language. John - This list is sponsored by AGIRI: John http://www.agiri.org/email To unsubscribe or change your John options, please go to: http://v2.listbox.com/member/?list_id=303 - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
Eric Baum [EMAIL PROTECTED] wrote: Matt wrote: Changing one bit of the key or plaintext affects every bit of the cipherte= xt. That is simply not true of most encryptions. For example, Enigma.=20 Matt: Enigma is laughably weak compared to modern encryption, such as AES, RSA, S= HA-256, ECC, etc. Enigma was broken with primitive mechanical computers an= d pencil and paper. Enigma was broken without modern computers, *given access to the machine.* I chose Enigma as an example, because to break language it may be necessary to pay attention to the machine-- namely examining the genomics. But that is more work than you envisage ;^) It is true that much modern encryption is based on simple algorithms. However, some crypto-experts would advise more primitive approaches. RSA is not known to be hard, even if P!=NP, someone may find a number-theoretic trick tomorrow that factors. (Or maybe they already have it, and choose not to publish). If you use a mess machine like a modern version of enigma, that is much less likely to get broken, even though you may not have the theoretical results. Your response admits that for stream ciphers changing a bit of the plaintext doesn't affect many bits of the ciphertext, which was what I was mainly responding to. You may prefer other kinds of cipher, but your arguments about chaos are clearly not germane to concluding language is easy to decode. Incidentally, while no encryption scheme is provably hard to break (even assuming P!=NP) more is known about grammars: they are provably hard to decode given P!=NP. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: RE: [agi] Natural versus formal AI interface languages
Ben Jef wrote:
As I see it, the present key challenge of artificial intelligence
is to develop a fast and frugal method of finding fast and frugal
methods,
Ben However, this in itself is not possible. There can be a fast
Ben method of finding fast and frugal methods, or a frugal method of
Ben finding fast and frugal methods, but not a fast and frugal method
Ben of finding fast and frugal methods ... not in general ...
in other words to develop an efficient time-bound algorithm for
recognizing and compressing those regularities in the world
faster than the original blind methods of natural evolution.
Ben This paragraph introduces the key restriction -- the world,
Ben i.e. the particular class of environments in which the AI is
Ben biased to operate.
As I and Jef and you appear to agree, extant Intelligence works
because it exploits structure *of our world*;
there is and can be (unless P=NP or some such radical and
unlikely possibility) no such thing as as General Intelligence
that works in all worlds.
Ben It is possible to have a fast and frugal method of finding {fast
Ben and frugal methods for operating in environments in class X} ...
Ben [However, there can be no fast and frugal method for producing
Ben such a method based solely on knowledge of the environment X ;-)
Ben ]
I am unsure what you mean by this. Maybe what you are saying is, its not
going to be possible by writing down a simple algorithm and running it
for a week on a PC. This I agree with.
The challenge is to find a methodology
for producing fast enough and frugal enough code, where that
methodology is practicable. For example, as a rough upper bound,
it would be practicable if it required 10,000 programmer years and
1,000,000 PC-years (i.e a $3Bn budget).
(Why should producing a human-level AI be cheaper than decoding the
genome?) And of course, it has to scale, in the sense that you have to
be able to prove with $10^7 (preferably $10^6 ) that the
methodology works (as was the case more or less with the genome.)
This, it seems to me, requires a first project much more limited
than understanding most of English, yet of significant practical
benefit. I'm wondering if someone has a good proposal.
Ben One of my current sub-projects is trying to precisely formulate
Ben conditions on the environment under which it is the case that
Ben Novamente's particular combination of AI algorithms is fast and
Ben frugal at finding fast and frugal methods for solving
Ben environment-relevant problems I believe I know how to do
Ben so, but proving my intuitions rigorously will be a bunch of work
Ben which I don't have time for at the moment ... but the task will
Ben go on my (long) queue...
Ben -- Ben
Ben - This list is sponsored by AGIRI: http://www.agiri.org/email
Ben To unsubscribe or change your options, please go to:
Ben http://v2.listbox.com/member/?list_id=303
-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303
Re: Re: RE: [agi] Natural versus formal AI interface languages
Eric wrote: The challenge is to find a methodology for producing fast enough and frugal enough code, where that methodology is practicable. For example, as a rough upper bound, it would be practicable if it required 10,000 programmer years and 1,000,000 PC-years (i.e a $3Bn budget). (Why should producing a human-level AI be cheaper than decoding the genome?) And of course, it has to scale, in the sense that you have to be able to prove with $10^7 (preferably $10^6 ) that the methodology works (as was the case more or less with the genome.) This, it seems to me, requires a first project much more limited than understanding most of English, yet of significant practical benefit. I'm wondering if someone has a good proposal. I am afraid that it may not be possible to find an initial project that is both * small * clearly a meaningfully large step along the path to AGI * of significant practical benefit My observation is that for nearly all practical tasks, either a) it is a fairly large amount of work to get them done within an AGI architectre or b) narrow-AI methods can do them pretty well with a much smaller amount of work than it would take to do them within an AGI architecture I suspect there are fundamental reasons for this, even though current computer science and AI theory doesn't let us articulate these reasons clearly, at this stage. So, I think that, in terms of proving the value of AGI research, we wll likely have to settle for a combination of: a) an interim task that is relatively small, and is clearly along the path to AGI, and is impressive in itself but is not necessarily of large practical benefit unto itself. b) interim tasks that are of practical value, and utilize AGI-related ideas, but may also be achievable (with different strengths and weaknesses) using narrow-AI methods As an example of a, I suggest robustly learning to carry out a number of Piagetan concrete-operational level tasks in a simulation world. As an example of b, I suggest natural language question answering in a limited domain. Alternate suggestions of tasks are solicited and much valued ... any suggestions?? ;-) Ben - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
RE: RE: [agi] Natural versus formal AI interface languages
Eric Baum wrote: As I and Jef and you appear to agree, extant Intelligence works because it exploits structure *of our world*; there is and can be (unless P=NP or some such radical and unlikely possibility) no such thing as as General Intelligence that works in all worlds. I'm going to risk being misunderstood again over a subtle point of clarification: I think we are in practical agreement on the point quoted above, but I think that a more coherent view would avoid the binary distinction and instead place general intelligence at the end of a scale where with diminishing exploitation of regularities in the environment computational requirements become increasingly intractable. - Jef - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
Eric Baum wrote: (Why should producing a human-level AI be cheaper than decoding the genome?) Because the genome is encrypted even worse than natural language. -- Eliezer S. Yudkowsky http://singinst.org/ Research Fellow, Singularity Institute for Artificial Intelligence - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
Eliezer Eric Baum wrote: (Why should producing a human-level AI be cheaper than decoding the genome?) Eliezer Because the genome is encrypted even worse than natural Eliezer language. (a) By decoding the genome, I meant merely finding the sequence (should have been clear in context), which didn't involve any decryption at all. (b) why do you think so? - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: Re: RE: [agi] Natural versus formal AI interface languages
Ben Goertzel [EMAIL PROTECTED] wrote: I am afraid that it may not be possible to find an initial project that is both * small * clearly a meaningfully large step along the path to AGI * of significant practical benefit I'm afraid you're right. It is especially difficult because there is a long history of small (i.e narrow AI) projects that appear superficially to be meaningful steps toward AGI. Sometimes it is decades before we discover that they don't scale. -- Matt Mahoney, [EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
I think that natural language and the human genome have about the same order of magnitude complexity. The genome is 6 x 10^9 bits (2 bits per base pair) uncompressed, but there is a lot of noncoding DNA and some redundancy. By decoding, I assume you mean building a model and understanding the genome to the point where you could modify it and predict what will happen. The complexity of natural language is probably 10^9 bits. This is supported by: - Turing's 1950 estimate, which he did not explain. - Landauer's estimate of human long term memory capacity. - The quantity of language processed by an average adult, times Shannon's estimate of the entropy of written English of 1 bit per character. - Extrapolating the relationship between language model training set size and compression ratio in this graph: http://cs.fit.edu/~mmahoney/dissertation/ I don't think the encryption of the genome is any worse. Complex systems (that have high Kolmogorov complexity, are incrementally updatable, and do useful computation) tend to converge to the boundary between stability and chaos, where some perturbations decay while others grow. A characteristic of such systems (as studied by Kaufmann) is that the number of stable states or attractors tends to the square root of the size. The number of human genes is about the same as the size of the human vocabulary, about 30,000. Neither system is encrypted in the mathematical sense. Encryption cannot be an emergent property because it is at the extreme chaotic end of the spectrum. Changing one bit of the key or plaintext affects every bit of the ciphertext. The difference is that it is easier (faster and more ethical) to experiment with language models than the human genome. -- Matt Mahoney, [EMAIL PROTECTED] - Original Message From: Eliezer S. Yudkowsky [EMAIL PROTECTED] To: agi@v2.listbox.com Sent: Wednesday, November 8, 2006 3:23:10 PM Subject: Re: [agi] Natural versus formal AI interface languages Eric Baum wrote: (Why should producing a human-level AI be cheaper than decoding the genome?) Because the genome is encrypted even worse than natural language. -- Eliezer S. Yudkowsky http://singinst.org/ Research Fellow, Singularity Institute for Artificial Intelligence - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303 - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
Eric Baum wrote: Eliezer Eric Baum wrote: (Why should producing a human-level AI be cheaper than decoding the genome?) Eliezer Because the genome is encrypted even worse than natural Eliezer language. (a) By decoding the genome, I meant merely finding the sequence (should have been clear in context), which didn't involve any decryption at all. (b) why do you think so? (a) Sorry, didn't pick up on that. Possibly, more money has already been spent on failed AGI projects than on the human genome. (b) Relative to an AI built by aliens, it's possible that the human proteome annotated by the corresponding selection pressures (= the decrypted genome), is easier to reverse-engineer than the causal graph of human language. Human language, after all, takes place in the context of a complicated human mind. But relative to humans, human language is certainly a lot easier for us to understand than the human proteome! -- Eliezer S. Yudkowsky http://singinst.org/ Research Fellow, Singularity Institute for Artificial Intelligence - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
Fully decoding the human genome is almost impossible. Not only is there the problem of protein folding, which I think even supercomputers can't fully solve, but the purpose for the structure of each protein depends on interaction with the incredibly complex molecular structures inside cells. Also, the genetic code for a human being is basically made of the same elements that the genetic code for the lowliest single-celled creature is made of, and yet it somehow describes the initial structure of a system of neural cells that then developes into a human brain through a process of embriological growth (which includes biological interaction from the mother -- why you can't just grow a human being from an embryo in a petri dish), and then a fairly long process of childhood development. This is the way evolution created mind somewhat randomly over three billion (and a half?) years. The human mind is the pinnacle of this evolution. With this mind along with collective intelligence, it shouldn't take another three billion years to engineer intelligence. Evolution is slow -- human beings can engineer. - Original Message - Eliezer S. Yudkowsky wrote: Eric Baum wrote: (Why should producing a human-level AI be cheaper than decoding the genome?) Because the genome is encrypted even worse than natural language. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
I actually just stumbled on something, from a totally different work I was doing, but possibly interesting:http://simple.wikipedia.org/wiki/Main_PageAn entire wikipedia, using simple english, that should be much much easier to parse than its more complex brother.JamesBillK [EMAIL PROTECTED] wrote: On 11/6/06, James Ratcliff wrote: In some form or another we are going to HAVE to have a natural language interface, either a translation program that can convert our english to the machine understandable form, or a simplified form of english that is trivial for a person to quickly understand and write. Humans use natural speech to communicate and to have an effective AGI that we can itneract with, it will have to have easy communication with us. That has been a critcal problem with all software since the beginning, a difficulty in the human computer interface. I go further to propose that as much knowledge information should be stored in easily recognizable natural language as well, only devolving into more complex forms where the cases warrant it, such as complex motor-sensor data sets, and some lower logic levels.Anybody remember short wave radio?The Voice of America does worldwide broadcasts in Special English.Special English has a core vocabulary of 1500 words. Most are simplewords that describe objects, actions or emotions. Some words are moredifficult. They are used for reporting world events and describingdiscoveries in medicine and science.Special English writers use short, simple sentences that contain onlyone idea. They use active voice. They do not use idioms.--There is also Basic English:Basic English is a constructed language with a small number of wordscreated by Charles Kay Ogden and described in his book Basic English:A General Introduction with Rules and Grammar (1930). The language isbased on a simplified version of English, in essence a subset of it.Ogden said that it would take seven years to learn English, sevenmonths for Esperanto, and seven weeks for Basic English, comparablewith Ido. Thus Basic English is used by companies who need to makecomplex books for international use, and by language schools that needto give people some knowledge of English in a short time.Also see:Basic English is a selection of 850 English words, used in simplestructural patterns, which is both an international auxiliary languageand a self-contained first stage for the teaching of any form of wideror Standard English. A subset, no unlearning.BillK-This list is sponsored by AGIRI: http://www.agiri.org/emailTo unsubscribe or change your options, please go to:http://v2.listbox.com/member/?list_id=303___James Ratcliff - http://falazar.comNew Torrent Site, Has TV and Movie Downloads! http://www.falazar.com/projects/Torrents/tvtorrents_show.php Cheap Talk? Check out Yahoo! Messenger's low PC-to-Phone call rates. This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
RE: [agi] Natural versus formal AI interface languages
James Jef Allbright [EMAIL PROTECTED] wrote: Russell Wallace James wrote: Syntactic ambiguity isn't the problem. The reason computers don't understand English is nothing to do with syntax, it's because they don't understand the world. It's easy to parse The cat sat on the mat into sit cat James on mat past But the computer still doesn't understand the sentence, because it doesn't know what cats, mats and the act of sitting _are_. (The best test of such understanding is not language - it's having the computer draw an animation of the action.) James Russell, I agree, but it might be clearer if we point out that James humans don't understand the world either. We just process these James symbols within a more encompassing context. James, I would like to know what you mean by understand. In my view, what humans do is the example we have of understanding, the word should be defined so as to have a reasonably precise meaning, and to include the observed phenomenon. You apparently have something else in mind by understanding. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
RE: [agi] Natural versus formal AI interface languages
James Below Shouls be Jef, but I will respond as wellOrig Quotes: But the computer still doesn't understand the sentence, because it doesn't know what cats, mats and the act of sitting _are_. (The best test of such understanding is not language - it's having the computer draw an animation of the action.) Russell, I agree, but it might be clearer if we point out that humansdon't understand the world either. We just process these symbols withina more encompassing context.- JefMe, James: Understand is probably a red flag word, for computers and humans alike. We have no nice judge of what is understood, and I try not to use that term generally, as it devolves into vague phsycho talk, and nothing concrete.But basically, a computer can do one of two things to "show" that it has "understood" something; 1. either show its internal representation. You said cat, I know that cat is a mammal that is blah, and blah, and does blah, some cats I know are blah.2. It acts upon this information, "Bring me the cat" is followed by the robot bringing the cat to you, it obviously "understands" what you mean.I believe with a very rich frame system of memory that will start a fairly good understanding of "What" somethings "means" and allow some basic "understanding".At the basest level a "cat" can only mean a certain few things, maybe using the WordNet ontology for filtering that out.The depending on context and usage, we can possibly narrow it down, and use the Frames for some basic pattern matching to narrow it down to the one.And, maybe if it cant be narrowed successfully, something else should happen, either model internally both or multiple objects / processes, or get outside intervention where available.We should remember that there are almost always humans around, and SHOULD be used in my opinion.Either if they are standing by the robot, then they can be quizzed directly, or if it is not a immediate deceision to be made, ask them via email or a phone call or something, and try to learn that information given so next time it will not have to ask.EX: "Bring me the cat." Confusion in the AI, seeing 4 cats in front of it. AI: Which cat do you want? resolve abiguity thru interface.James RatcliffEric Baum [EMAIL PROTECTED] wrote: James Jef Allbright <[EMAIL PROTECTED]> wrote: Russell WallaceJames wrote: Syntactic ambiguity isn't the problem. The reason computers don't understand English is nothing to do with syntax, it's because they don't understand the world. It's easy to parse "The cat sat on the mat" into sit cat James on mat past But the computer still doesn't understand the sentence, because it doesn't know what cats, mats and the act of sitting _are_. (The best test of such understanding is not language - it's having the computer draw an animation of the action.)James Russell, I agree, but it might be clearer if we point out thatJames humans don't understand the world either. We just process theseJames symbols within a more encompassing context.James, I would like to know what you mean by "understand".In my view, what humans do is the example we have of understanding,the word should be defined so as to have a reasonably precise meaning,and to include the observed phenomenon.You apparently have something else in mind by understanding.-This list is sponsored by AGIRI: http://www.agiri.org/emailTo unsubscribe or change your options, please go to:http://v2.listbox.com/member/?list_id=303___James Ratcliff - http://falazar.comNew Torrent Site, Has TV and Movie Downloads! http://www.falazar.com/projects/Torrents/tvtorrents_show.php Sponsored Link Free Uniden 5.8GHz Phone System with Packet8 Internet Phone Service This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
RE: [agi] Natural versus formal AI interface languages
Eric Baum wrote: James Jef Allbright [EMAIL PROTECTED] wrote: Russell Wallace James wrote: Syntactic ambiguity isn't the problem. The reason computers don't understand English is nothing to do with syntax, it's because they don't understand the world. snip But the computer still doesn't understand the sentence, because it doesn't know what cats, mats and the act of sitting _are_. (The best test of such understanding is not language - it's having the computer draw an animation of the action.) James Russell, I agree, but it might be clearer if we point out that James humans don't understand the world either. We just process these James symbols within a more encompassing context. James, I would like to know what you mean by understand. In my view, what humans do is the example we have of understanding, the word should be defined so as to have a reasonably precise meaning, and to include the observed phenomenon. You apparently have something else in mind by understanding. Eric, you may refer to me as James ;-), but as with the topic at hand, it adds an unnecessary level of complexity and impedes understanding. It is common to think of machines as not possessing the faculty of understanding while humans do. Similarly, machines not possessing consciousness while humans do. This way of thinking is adequately effective for daily use, but it carries and propagates the implicit assumption that understanding and consciousness are somehow intrinsically distinct from other types of processing carried out by physical systems. It is simpler and more coherent to think in terms of a scale of processing within increasingly complex context, such that one might say that a vending machine understands the difference between certain coins, an infant understands that a nipple is a source of goodness, and most adults understand that cooperation is more productive than conflict. Alternatively we can say that a vending machine responds effectively to the insertion of proper coins, an infant responds effectively to the presence of a nipple, and most adults respond effectively by choosing cooperation over conflict. But let's rather not say that a vending machine doesn't really understand the difference between coins, an infant doesn't really understand the whys and wherefores of nipples, but most adults really do understand in all its significant implications why cooperation is more productive than conflict. Each of these examples is of a physical system responding with some degree of effectiveness based on an internal model that represents with some degree of fidelity its local environment. Its an unnecessary complication, and leads to endless discussions of qualia, consciousness, free will and the like, to assume that at some magical unspecified point there is a transition to true understanding. None of which is intended to deny that from a common-sense point of view, humans understand things that machines don't. But for computer scientists working on AI, I think such conceptualizing is sloppy and impedes effective discussion and progress. - Jef - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
RE: [agi] Natural versus formal AI interface languages
Jef wrote: Each of these examples is of a physical system responding with some degree of effectiveness based on an internal model that represents with some degree of fidelity its local environment. Its an unnecessary complication, and leads to endless discussions of qualia, consciousness, free will and the like, to assume that at some magical unspecified point there is a transition to true understanding. It occurred to me that my use of the term fidelity with respect to an agents internal model may have been misleading. Rather than say the model represents its environment with some degree of fidelity I should have said it represents its environment with some degree of effectiveness, since it's a model of what seems to work, rather than a model of what seems to be. - Jef - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
RE: [agi] Natural versus formal AI interface languages
James and Jef, my appologies for misattributing the question. There is a phenomenon colloquially called understanding that is displayed by people and at best rarely displayed within limitted domains by extant computer programs. If you want to have any hope of constructing an AGI, you are going to have to come to grips with what it is and how it is achieved. As to what I believe the answer is, I refer you to the top (new) paper at http://whatisthought.com/eric.html entitled A Working Hypothesis for General Intelligence (and to my book What is Thought? if you want more background.) Eric Baum http://whatisthought.com - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
RE: [agi] Natural versus formal AI interface languages
Eric - Thanks to the pointer to your paper. Upon reading I quickly saw what I think provoked your reaction to my observation about understanding. We were actually saying much the same thing there. My point was that no human understands the world, because our understanding, as with all examples of intelligence that we know of, is domain-specific. I used the word context as synonymous with domain. My point was that not that humans don't *understand* the world, but that humans don't understand the *world*. I tried to make that clear in my follow-up, but it appears I lost your interest very early on. In reading your paper, I see that you seem to use the terms world and domain quite synonymously, but I'm sure you can appreciate that domain connotes a limitation of scope while world connotes expanded or ultimate scope. Our domain specific knowledge is of the world, but one cannot derive the world from our domain-specific knowledge since a great deal of information is lost in the compression process, and that really speaks to the core of what it means to understand. When I read in your paper The claim is that the world has structure that can be exploited to rapidly solve problems which arise, and that underlying our thought processes are modules that accomplish this., that rang a familiar bell for me. I can remember the intellectual excitement I felt when I first came across this idea back in the 1990s, probably from Gigerenzer, Kahneman Tversky, Tooby Cosmides or some combination of their thinking on fast and frugal heuristics and bounded rationality. You might have deduced my bias toward the domain-specific theory of (evolved) intelligence by my statement that the internal model must represent what seems to work, rather than what seems to be, in the environment. As I see it, the present key challenge of artificial intelligence is to develop a fast and frugal method of finding fast and frugal methods, in other words to develop an efficient time-bound algorithm for recognizing and compressing those regularities in the world faster than the original blind methods of natural evolution. - Jef -Original Message- From: Eric Baum [mailto:[EMAIL PROTECTED] Sent: Tuesday, November 07, 2006 1:44 PM To: agi@v2.listbox.com Subject: RE: [agi] Natural versus formal AI interface languages James and Jef, my appologies for misattributing the question. There is a phenomenon colloquially called understanding that is displayed by people and at best rarely displayed within limitted domains by extant computer programs. If you want to have any hope of constructing an AGI, you are going to have to come to grips with what it is and how it is achieved. As to what I believe the answer is, I refer you to the top (new) paper at http://whatisthought.com/eric.html entitled A Working Hypothesis for General Intelligence (and to my book What is Thought? if you want more background.) Eric Baum http://whatisthought.com - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303 - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: RE: [agi] Natural versus formal AI interface languages
Jef wrote:
As I see it, the present key challenge of artificial intelligence is to
develop a fast and frugal method of finding fast and frugal methods,
However, this in itself is not possible. There can be a fast method
of finding fast and frugal methods, or a frugal method of finding fast
and frugal methods, but not a fast and frugal method of finding fast
and frugal methods ... not in general ...
in
other words to develop an efficient time-bound algorithm for recognizing
and compressing those regularities in the world faster than the
original blind methods of natural evolution.
This paragraph introduces the key restriction -- the world, i.e. the
particular class of environments in which the AI is biased to operate.
It is possible to have a fast and frugal method of finding {fast and
frugal methods for operating in environments in class X} ...
[However, there can be no fast and frugal method for producing such a
method based solely on knowledge of the environment X ;-) ]
One of my current sub-projects is trying to precisely formulate
conditions on the environment under which it is the case that
Novamente's particular combination of AI algorithms is fast and
frugal at finding fast and frugal methods for solving
environment-relevant problems I believe I know how to do so,
but proving my intuitions rigorously will be a bunch of work which I
don't have time for at the moment ... but the task will go on my
(long) queue...
-- Ben
-
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
Richard, The Blocks World (http://hci.stanford.edu/~winograd/shrdlu/) was over 36 years ago, and was a GREAT demonstration of what can be done with natural language. It handled a wide variety of items, albeit with a very limited environment. Currently MIT is doing work with robitics that uses the same types of systems, where they can talk to a grasper robot and tell it to pick up or move the yellow thing, and stuff like that. It is limited to its small environment, but that was also over 36 years ago.Today, we sould be able to take something like this and expand upwards. The harder part of the equation for a complex system like this is actually the robotics end, and image recognition tasks. In some form or another we are going to HAVE to have a natural language interface, either a translation program that can convert our english to the machine understandable form, or a simplified form of english that is trivial for a person to quickly understand and write. Humans use natural speech to communicate and to have an effective AGI that we can itneract with, it will have to have easy communication with us. That has been a critcal problem with all software since the beginning, a difficulty in the human computer interface.I go further to propose that as much knowledge information should be stored in easily recognizable natural language as well, only devolving into more complex forms where the cases warrant it, such as complex motor-sensor data sets, and some lower logic levels.James RatcliffRichard Loosemore [EMAIL PROTECTED] wrote: James Ratcliff wrote: Not necessarily childrens language, as tehy have their own problems and often use the wrong words and rules of grammar, but a simplified english, a reduced rule set. Something like no compound sentences for a start. I believe most everything can be written without compound sentences, and that would greatly reduce the processing complexity, and anaphora resolution as a part of the language rules, so if you reference something in one place it will stay the same throughout the section. Its not quite as natural, but could be understood simply enough by humans as well as computers. One problem I have with all of this, is the super-flowery writing styles of cramming as many words and complex topics all into one sentence.This is a question directed at this whole thread, about simplifying language to communicate with an AI system, so we can at least get something working, and then go from thereThis rationale is the very same rationale that drove researchers into Blocks World programs. Winograd and SHRDLU, etc. It was a mistake then: it is surely just as much of a mistake now.Richard Loosemore.-This list is sponsored by AGIRI: http://www.agiri.org/emailTo unsubscribe or change your options, please go to:http://v2.listbox.com/member/?list_id=303Thank YouJames Ratcliffhttp://falazar.com Sponsored Link Talk more and pay less. Vonage can save you up to $300 a year on your phone bill. Sign up now. This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: Re: Re: Re: [agi] Natural versus formal AI interface languages
Ben, I think it would be beneficial, at least to me, to see a list of tasks. Not as a "defining" measure in any way. But as a list of work items that a general AGI should be able to complete effectively. I started on a list, and pulled some information off the net before, but never completed one. My thoughts on a list like this is that is should be marked in increasing levels of difficulty, so an initial AGI should have the ability to complete the first level of tasks and so on. One problem I am running into here and in other discussions is the seperation between online AI, and robotic AI, and I have had to restrict alot of the research I am doing to online AI, and only a small amount of simulated robotic AI.There are many tasks that are not possible with non-robotic AI, and I would like to see the different classes of these tasks, so I could correctly model the system to handle the wide variety of the behaviors necessary.Ex: One item of AI task is a simple question answering ability, that can respond with an answer currently in the Knowledge base of the system.A more expansive item, would require the QA task to go and get outside information.James RatcliffBen Goertzel [EMAIL PROTECTED] wrote: I am happy enough with the long-term goal of independent scientific and mathematical discovery... And, in the short term, I am happy enough with the goals of carrying out the (AGISim versions of) the standard tasks used by development psychologists to study childrens' cognitive behavior... I don't see a real value to precisely quantifying these goals, though...To give an example of the kind of short-term goal that I think isuseful, though, consider the following.We are in early 2007 (if all goes according to plan) going to teachNovamente to carry out a game called "iterated Easter Egg hunt" --basically, to carry out an Easter Egg hunt in a room full of otheragents ... and then do so over and over again, modeling what the otheragents do and adjusting its behavior accordingly.Now, this task has a bit in common with the game Hide-and-Seek. So,you'd expect that a Novamente instance that had been taught iteratedEaster Egg Hunt, would also be good at hide-and-seek. So, we want tosee that the time required for an NM system to learn hide-and-seekwill be less if the NM system has previously learned to play iteratedEaster Egg hunt...This sort of goal is, I feel, good for infant-stage AGI educationHowever, I wouldn't want to try to turn it into an "objective IQtest." Our goal is not to make the best possible system for playingEaster Egg hunt or hide and seek or fetch or whateverAnd, in terms of language learning, our initial goal will not be tomake the best possible system for conversing in baby-talk...Rather, our goal will be to make a system that can adequately fulfillthese early-stage tasks, but in a way that we feel will beindefinitely generalizable to more complex tasks.This, I'm afraid, highlights a general issue with formal quantitativeintelligence measures as applied to immature AGI systems/minds. Oftenthe best way to achieve some early-developmental-stage task is goingto be an overfitted, narrow-AI type of algorithm, which is not easilyextendable to address more complex tasks.This is similar to my complaint about the Hutter Prize. Yah, asuperhuman AGI will be an awesome text compressor. But this doesn'tmean that the best way to achieve slightly better text compressionthan current methods is going to be **at all** extensible in thedirection of AGI.Matt, you have yet to convince me that seeking to optimize interimquantitative milestones is a meaningful path to AGI. I think it isprobably just a path to creating milestone-task-overfit narrow-AIsystems without any real AGI-related expansion potential...-- Ben-This list is sponsored by AGIRI: http://www.agiri.org/emailTo unsubscribe or change your options, please go to:http://v2.listbox.com/member/?list_id=303Thank YouJames Ratcliffhttp://falazar.com Sponsored Link Free Uniden 5.8GHz Phone System with Packet8 Internet Phone Service This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
On 11/6/06, James Ratcliff wrote: In some form or another we are going to HAVE to have a natural language interface, either a translation program that can convert our english to the machine understandable form, or a simplified form of english that is trivial for a person to quickly understand and write. Humans use natural speech to communicate and to have an effective AGI that we can itneract with, it will have to have easy communication with us. That has been a critcal problem with all software since the beginning, a difficulty in the human computer interface. I go further to propose that as much knowledge information should be stored in easily recognizable natural language as well, only devolving into more complex forms where the cases warrant it, such as complex motor-sensor data sets, and some lower logic levels. Anybody remember short wave radio? The Voice of America does worldwide broadcasts in Special English. http://www.voanews.com/specialenglish/about_special_english.cfm Special English has a core vocabulary of 1500 words. Most are simple words that describe objects, actions or emotions. Some words are more difficult. They are used for reporting world events and describing discoveries in medicine and science. Special English writers use short, simple sentences that contain only one idea. They use active voice. They do not use idioms. -- There is also Basic English: http://en.wikipedia.org/wiki/Basic_English Basic English is a constructed language with a small number of words created by Charles Kay Ogden and described in his book Basic English: A General Introduction with Rules and Grammar (1930). The language is based on a simplified version of English, in essence a subset of it. Ogden said that it would take seven years to learn English, seven months for Esperanto, and seven weeks for Basic English, comparable with Ido. Thus Basic English is used by companies who need to make complex books for international use, and by language schools that need to give people some knowledge of English in a short time. Also see: http://www.basic-english.org/ Basic English is a selection of 850 English words, used in simple structural patterns, which is both an international auxiliary language and a self-contained first stage for the teaching of any form of wider or Standard English. A subset, no unlearning. BillK - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: Re: Re: Re: Re: [agi] Natural versus formal AI interface languages
Hi, On 11/6/06, James Ratcliff [EMAIL PROTECTED] wrote: Ben, I think it would be beneficial, at least to me, to see a list of tasks. Not as a defining measure in any way. But as a list of work items that a general AGI should be able to complete effectively. I agree, and I think that this requires a lot of care. Carefully articulating such a list is on my agenda for the first half of next year (not that it will take full-time for 6 months, it will be a background task). My approach will be based on porting a number of basic ideas from human developmental psychology into the non-human-like-AGI-acting-in-a-simulation-world domain, but will also be useful beyond this particular domain... My thoughts on a list like this is that is should be marked in increasing levels of difficulty, so an initial AGI should have the ability to complete the first level of tasks and so on. Agreed, although most tasks will have the notion of partial completion rather than being binary in nature. Ex: One item of AI task is a simple question answering ability, that can respond with an answer currently in the Knowledge base of the system. A more expansive item, would require the QA task to go and get outside information. This seems not to be a very well-specified task ;-) ... the problem is that it refers to the internal state of the AI system (its knowledge base), whereas the tasks I will define will refer only to the system's external behaviors given various sets of stimuli... -- Ben - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
I dont believe that was the goal or lesson of the http://en.wikipedia.org/wiki/SHRDLU project.It was mainly centered aroudn a small test environment (the block world)and being able to create an interface that would allow the user to speak and be answered in a natural language.And in that goal it seemed to succeed fairly admirably.When we look at throwing the rest of the world into the simulation, we of course get problems, but that is understandable I believe.But this is a Type of goal that we would like to have (I think) is the ability in a much larger dynamic world environment to be able to talk and interact with the AI and have it understand you and be able to answer you.And as far as knowledge engineering (aka Cyc and others) it DOES work. The problem is just the size and complexity.I just finished a class here on Knowledge Engineering, and they worked on a project that allows you to do that and works very well on limited domains.One of the main drawbacks to the KE is the massive work-hours involved to get a decent knowledge base. and the limited scope of any of them.But the blocks world did show some sophisticated behavior for what it was:Person: PICK UP A BIG RED BLOCK. Computer: OK. Person: GRASP THE PYRAMID. Computer: I DON'T UNDERSTAND WHICH PYRAMID YOU MEAN. Person (changing their mind): FIND A BLOCK WHICH IS TALLER THAN THE ONE YOU ARE HOLDING AND PUT IT INTO THE BOX. Computer: BY "IT", I ASSUME YOU MEAN THE BLOCK WHICH IS TALLER THAN THE ONE I AM HOLDING. Computer: OK. Person: WHAT DOES THE BOX CONTAIN? Computer: THE BLUE PYRAMID AND THE BLUE BLOCK. Person: WHAT IS THE PYRAMID SUPPORTED BY? Computer: THE BOX. Person: HOW MANY BLOCKS ARE NOT IN THE BOX? Computer: FOUR OF THEM. Person: IS AT LEAST ONE OF THEM NARROWER THAN THE ONE WHICH I TOLD YOU TO PICK UP? Computer: YES, THE RED CUBE.including reason and memory and ambiguity.James RatcliffMatt Mahoney [EMAIL PROTECTED] wrote: Another important lesson from SHRDLU, aside from discovering that the approach of hand coding knowledge doesn't work, was how long it took to discover this. It was not at all obvious from the initial success. Cycorp still hasn't figured it out after over 20 years. -- Matt Mahoney, [EMAIL PROTECTED]- Original Message From: Charles D Hixson <[EMAIL PROTECTED]>To: agi@v2.listbox.comSent: Sunday, November 5, 2006 4:46:12 PMSubject: Re: [agi] Natural versus formal AI interface languagesRichard Loosemore wrote: ... This is a question directed at this whole thread, about simplifying language to communicate with an AI system, so we can at least get something working, and then go from there This rationale is the very same rationale that drove researchers into Blocks World programs. Winograd and SHRDLU, etc. It was a mistake then: it is surely just as much of a mistake now. Richard Loosemore. -Not surely. It's definitely a defensible position, but I don't see any evidence that it has even a 50% probability of being correct.Also I'm not certain that SHRDLU and Blocks World were mistakes. They didn't succeed in their goals, but they remain as important markers. At each step we have limitations imposed by both our knowledge and our resources. These limits aren't constant. (P.S.: I'd throw Eliza into this same category...even though the purpose behind Eliza was different.)Think of the various approaches taken as being experiments with the user interface...since that's a large part of what they were. They are, of course, also experiments with how far one can push a given technique before encountering a combinatorial explosion. People don't seem very good at understanding that intuitively. In neural nets this same problem re-appears as saturation, the point at which as you learn new things old things become fuzzier and less certain. This may have some relevance to the way that people are continually re-writing their memories whenever they remember something.-This list is sponsored by AGIRI: http://www.agiri.org/emailTo unsubscribe or change your options, please go to:http://v2.listbox.com/member/?list_id=303-This list is sponsored by AGIRI: http://www.agiri.org/emailTo unsubscribe or change your options, please go to:http://v2.listbox.com/member/?list_id=303Thank YouJames Ratcliffhttp://falazar.com Sponsored Link Free Uniden 5.8GHz Phone System with Packet8 Internet Phone Service This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: Re: Re: Re: Re: [agi] Natural versus formal AI interface languages
Ben Goertzel [EMAIL PROTECTED] wrote: Hi,On 11/6/06, James Ratcliff <[EMAIL PROTECTED]> wrote: Ben, I think it would be beneficial, at least to me, to see a list of tasks. Not as a "defining" measure in any way. But as a list of work items that a general AGI should be able to complete effectively.I agree, and I think that this requires a lot of care. Carefullyarticulating such a list is on my agenda for the first half of nextyear (not that it will take full-time for 6 months, it will be a"background task"). My approach will be based on porting a number ofbasic ideas from human developmental psychology into thenon-human-like-AGI-acting-in-a-simulation-world domain, but will alsobe useful beyond this particular domain... My thoughts on a list like this is that is should be marked in increasing levels of difficulty, so an initial AGI should have the ability to complete the first level of tasks and so on.Agreed, although most tasks will have the notion of "partialcompletion" rather than being binary in nature. Ex: One item of AI task is a simple question answering ability, that can respond with an answer currently in the Knowledge base of the system. A more expansive item, would require the QA task to go and get outside information.This seems not to be a very well-specified task ;-) ... the problem isthat it refers to the internal state of the AI system (its "knowledgebase"), whereas the tasks I will define will refer only to thesystem's external behaviors given various sets of stimuli...-- BenIm very focused here on its knowledge base as that is the main module I am working on. I am priming it with a large amount of extracted information from news and other texts, and its first task will be able to answer basic questions, and model the knowledge correctly there, before it can go on into deeper areas of reasoning and behaviour. I want the internal state to have asomewhat stable start before it goes forward into other areas.How much of the Novamente system is meant to be autonomous, and how much will be responding only from external stymulus such as a question or a task given externally.Is it intended after awhile to run "on its own" where it would be up 24 hours a day, exploring potentially some by itself, or more of a contained AI to be called up as needed?James RatcliffThank YouJames Ratcliffhttp://falazar.com Check out the all-new Yahoo! Mail - Fire up a more powerful email and get things done faster. This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: Re: Re: Re: Re: Re: [agi] Natural versus formal AI interface languages
How much of the Novamente system is meant to be autonomous, and how much will be responding only from external stymulus such as a question or a task given externally. Is it intended after awhile to run on its own where it would be up 24 hours a day, exploring potentially some by itself, or more of a contained AI to be called up as needed? Yes, it is intended to run permanently and autonomously, of course... Although at the moment it is being utilized in more of a task-focused way... this is because it is still in development... ben - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
- Original Message From: BillK [EMAIL PROTECTED] To: agi@v2.listbox.com Sent: Monday, November 6, 2006 10:08:09 AM Subject: Re: [agi] Natural versus formal AI interface languages Ogden said that it would take seven years to learn English, seven months for Esperanto, and seven weeks for Basic English, comparable with Ido. Basic English = 850 words = 10 words per day. Esperanto = 900 root forms or 17,000 words (http://www.freelang.net/dictionary/esperanto.html) = 4 to 80 words per day. English = 30,000 to 80,000 words = 12 to 30 words per day. SHRDLU = 200 words? = 0.3 words per day for 2 years. -- Matt Mahoney, [EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
Richard Loosemore wrote: ... This is a question directed at this whole thread, about simplifying language to communicate with an AI system, so we can at least get something working, and then go from there This rationale is the very same rationale that drove researchers into Blocks World programs. Winograd and SHRDLU, etc. It was a mistake then: it is surely just as much of a mistake now. Richard Loosemore. - Not surely. It's definitely a defensible position, but I don't see any evidence that it has even a 50% probability of being correct. Also I'm not certain that SHRDLU and Blocks World were mistakes. They didn't succeed in their goals, but they remain as important markers. At each step we have limitations imposed by both our knowledge and our resources. These limits aren't constant. (P.S.: I'd throw Eliza into this same category...even though the purpose behind Eliza was different.) Think of the various approaches taken as being experiments with the user interface...since that's a large part of what they were. They are, of course, also experiments with how far one can push a given technique before encountering a combinatorial explosion. People don't seem very good at understanding that intuitively. In neural nets this same problem re-appears as saturation, the point at which as you learn new things old things become fuzzier and less certain. This may have some relevance to the way that people are continually re-writing their memories whenever they remember something. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
Another important lesson from SHRDLU, aside from discovering that the approach of hand coding knowledge doesn't work, was how long it took to discover this. It was not at all obvious from the initial success. Cycorp still hasn't figured it out after over 20 years. -- Matt Mahoney, [EMAIL PROTECTED] - Original Message From: Charles D Hixson [EMAIL PROTECTED] To: agi@v2.listbox.com Sent: Sunday, November 5, 2006 4:46:12 PM Subject: Re: [agi] Natural versus formal AI interface languages Richard Loosemore wrote: ... This is a question directed at this whole thread, about simplifying language to communicate with an AI system, so we can at least get something working, and then go from there This rationale is the very same rationale that drove researchers into Blocks World programs. Winograd and SHRDLU, etc. It was a mistake then: it is surely just as much of a mistake now. Richard Loosemore. - Not surely. It's definitely a defensible position, but I don't see any evidence that it has even a 50% probability of being correct. Also I'm not certain that SHRDLU and Blocks World were mistakes. They didn't succeed in their goals, but they remain as important markers. At each step we have limitations imposed by both our knowledge and our resources. These limits aren't constant. (P.S.: I'd throw Eliza into this same category...even though the purpose behind Eliza was different.) Think of the various approaches taken as being experiments with the user interface...since that's a large part of what they were. They are, of course, also experiments with how far one can push a given technique before encountering a combinatorial explosion. People don't seem very good at understanding that intuitively. In neural nets this same problem re-appears as saturation, the point at which as you learn new things old things become fuzzier and less certain. This may have some relevance to the way that people are continually re-writing their memories whenever they remember something. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303 - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
I'll keep this short, just to weigh in a vote - I completelyagree with this. AGI will be measured by what we recognize as intelligent behavior andthe usefulness ofthat intelligence for tasks beyond the capabilities ofordinary software. Normal metrics don't apply. Russell Wallace wrote: Ben Goertzelwrote: I of course don't think that SHRDLU vs. AGISim is a fair comparison. Agreed. SHRDLU didn't even try to solve the real problems - for the simple and sufficient reason that it was impossible to make a credible attempt at such on the hardware of the day. AGISim (if I understand it correctly) does. Oh, I'm sure the current implementation makes fatal compromises to fit on today's hardware - but the concept doesn't have an _inherent_ plateau the way SHRDLU did, so it leaves room for later upgrade. It's headed in the right compass direction. And, deciding which AGI is smarter is not important either -- no moreimportant than deciding whether Ben, Matt or Pei is smarter.Who cares? Agreed. In practice the market will decide: which system ends up doing useful things in the real world, and therefore getting used? Academic judgements of which is smarter are, well, academic. This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: Re: Re: [agi] Natural versus formal AI interface languages
On 11/4/06, Russell Wallace [EMAIL PROTECTED] wrote: On 11/4/06, Ben Goertzel [EMAIL PROTECTED] wrote: I of course don't think that SHRDLU vs. AGISim is a fair comparison. Agreed. SHRDLU didn't even try to solve the real problems - for the simple and sufficient reason that it was impossible to make a credible attempt at such on the hardware of the day. AGISim (if I understand it correctly) does. Oh, I'm sure the current implementation makes fatal compromises to fit on today's hardware - but the concept doesn't have an _inherent_ plateau the way SHRDLU did, so it leaves room for later upgrade. It's headed in the right compass direction. Actually, I phrased my comment somewhat imprecisely. What I should have said is: I don't think that SHRDLU's Blocks World versus AGISim is a fair comparison These are both environments for AI's ... The absurd comparison between AI systems would be SHRDLU versus Novamente ;-) ben g - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
Ben, The test you described (Easter Egg Hunt) is a perfectly good example of the type of test I was looking for. When you run the experiment you will no doubt repeat it many times, adjusting various parameters. Then you will evaluate by how many eggs are found, how fast, and the extent to which it helps the system learns to play Hide and Seek (also a measurable quantity). Two other good qualities are that the test is easy to describe and obviously relevant to intelligence. For text compression, the relevance is not so obvious. I look forward to seeing a paper on the outcome of the tests. -- Matt Mahoney, [EMAIL PROTECTED] - Original Message From: Ben Goertzel [EMAIL PROTECTED] To: agi@v2.listbox.com Sent: Friday, November 3, 2006 10:51:16 PM Subject: Re: Re: Re: Re: [agi] Natural versus formal AI interface languages I am happy enough with the long-term goal of independent scientific and mathematical discovery... And, in the short term, I am happy enough with the goals of carrying out the (AGISim versions of) the standard tasks used by development psychologists to study childrens' cognitive behavior... I don't see a real value to precisely quantifying these goals, though... To give an example of the kind of short-term goal that I think is useful, though, consider the following. We are in early 2007 (if all goes according to plan) going to teach Novamente to carry out a game called iterated Easter Egg hunt -- basically, to carry out an Easter Egg hunt in a room full of other agents ... and then do so over and over again, modeling what the other agents do and adjusting its behavior accordingly. Now, this task has a bit in common with the game Hide-and-Seek. So, you'd expect that a Novamente instance that had been taught iterated Easter Egg Hunt, would also be good at hide-and-seek. So, we want to see that the time required for an NM system to learn hide-and-seek will be less if the NM system has previously learned to play iterated Easter Egg hunt... This sort of goal is, I feel, good for infant-stage AGI education However, I wouldn't want to try to turn it into an objective IQ test. Our goal is not to make the best possible system for playing Easter Egg hunt or hide and seek or fetch or whatever And, in terms of language learning, our initial goal will not be to make the best possible system for conversing in baby-talk... Rather, our goal will be to make a system that can adequately fulfill these early-stage tasks, but in a way that we feel will be indefinitely generalizable to more complex tasks. This, I'm afraid, highlights a general issue with formal quantitative intelligence measures as applied to immature AGI systems/minds. Often the best way to achieve some early-developmental-stage task is going to be an overfitted, narrow-AI type of algorithm, which is not easily extendable to address more complex tasks. This is similar to my complaint about the Hutter Prize. Yah, a superhuman AGI will be an awesome text compressor. But this doesn't mean that the best way to achieve slightly better text compression than current methods is going to be **at all** extensible in the direction of AGI. Matt, you have yet to convince me that seeking to optimize interim quantitative milestones is a meaningful path to AGI. I think it is probably just a path to creating milestone-task-overfit narrow-AI systems without any real AGI-related expansion potential... -- Ben - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303 - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
Eliezer S. Yudkowsky wrote: Pei Wang wrote: On 11/2/06, Eric Baum [EMAIL PROTECTED] wrote: Moreover, I argue that language is built on top of a heavy inductive bias to develop a certain conceptual structure, which then renders the names of concepts highly salient so that they can be readily learned. (This explains how we can learn 10 words a day, which children routinely do.) An AGI might in principle be built on top of some other conceptual structure, and have great difficulty comprehending human words-- mapping them onto its concepts, much less learning them. I think any AGI will need the ability to (1) using mental entities (concepts) to summarize percepts and actions, and (2) using concepts to extend past experience to new situations (reasoning). In this sense, the categorization/learning/reasoning (thinking) mechanisms of different AGIs may be very similar to each other, while the contents of their conceptual structures are very different, due to the differences in their sensors and effectors, as well as environments. Pei, I suspect that what Baum is talking about is - metaphorically speaking - the problem of an AI that runs on SVD talking to an AI that runs on SVM. (Singular Value Decomposition vs. Support Vector Machines.) Or the ability of an AI that runs on latent-structure Bayes nets to exchange concepts with an AI that runs on decision trees. Different AIs may carve up reality along different lines, so that even if they label their concepts, it may take considerable extra computing power for one of them to learn the other's concepts - it may not be natural to them. They may not be working in the same space of easily learnable concepts. Of course these examples are strictly metaphorical. But the point is that human concepts may not correspond to anything that an AI can *natively* learn and *natively* process. And when you think about running the process in reverse - trying to get a human to learn the AI's native language - then the problem is even worse. We'd have to modify the AI's concept-learning mechanisms to only learn humanly-learnable concepts. Because there's no way the humans can modify themselves, or run enough sequential serial operations, to understand the concepts that would be natural to an AI that used its computing power in the most efficient way. A superintelligence, or a sufficiently self-modifying AI, should not be balked by English. A superintelligence should carve up reality into sufficiently fine grains that it can learn any concept computable by our much smaller minds, unless P != NP and the concepts are genuinely encrypted. And a self-modifying AI should be able to natively run whatever it likes. This point, however, Baum may not agree with. This is just speculation. It is believable that different systems may have trouble if their experiences do not overlap (a Japanese friend of mine had great trouble with our conversations, even though her knowledge of the language per se was extremely good ... too many cultural references to British TV shows in typical Brit-speak). But the idea that there might be an effect due to the design of the thinking mechanism is not based on any evidence that I can see. For all we know, the concepts will converge if experiences are the same. Your conclusions therefore do not follow. Richard Loosemore - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
RE: [agi] Natural versus formal AI interface languages
Jef,Even given a hand created checked and correct small but comprehensive Knowledge Representation of the sample world, it is STILL not a trivial effort to get the sentences from the complicated form of english into some computer processable format. The cat example you gave is unfortunalty not the norm.An example from the texts I am working with is:"A draft United Nations resolution callingfor sanctions on Iran has been dealt a severe blowby China and Russia and, given the absence of anyevidence of nuclear-weapons proliferation by Iran,the momentum for UN action against Iran has begunto fizzle."Which becomes much harder to parse and put into machine readable format.There are two major interconnecting issues here, the Natural Langauge Processing, and the Knowledge Representation, which unfortunatly rely very heavily on eachother and must be solved together.JamesJef Allbright [EMAIL PROTECTED] wrote: Russell Wallace wrote: Syntactic ambiguity isn't the problem. The reason computers don't understand English is nothing to do with syntax, it's because they don't understand the world. It's easy to parse "The cat sat on the mat" into sit cat on mat past But the computer still doesn't understand the sentence, because it doesn't know what cats, mats and the act of sitting _are_. (The best test of such understanding is not language - it's having the computer draw an animation of the action.) Russell, I agree, but it might be clearer if we point out that humansdon't understand the world either. We just process these symbols withina more encompassing context.- Jef-This list is sponsored by AGIRI: http://www.agiri.org/emailTo unsubscribe or change your options, please go to:http://v2.listbox.com/member/[EMAIL PROTECTED]Thank YouJames Ratcliffhttp://falazar.com Want to start your own business? Learn how on Yahoo! Small Business. This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
Not necessarily childrens language, as tehy have their own problems and often use the wrong words and rules of grammar, but a simplified english, a reduced rule set.Something like no compound sentences for a start. I believe most everything can be written without compound sentences, and that would greatly reduce the processing complexity, and anaphora resolution as a part of the language rules, so if you reference something in one place it will stay the same throughout the section.Its not quite as natural, but could be understood simply enough by humans as well as computers.One problem I have with all of this, is the super-flowery writing styles of cramming as many words and complex topics all into one sentence.JamesMatt Mahoney [EMAIL PROTECTED] wrote: - Original Message From: Ben Goertzel <[EMAIL PROTECTED]>To: agi@v2.listbox.comSent: Tuesday, October 31, 2006 9:26:15 PMSubject: Re: Re: [agi] Natural versus formal AI interface languagesHere is how I intend to use Lojban++ in teaching Novamente. WhenNovamente is controlling a humanoid agent in the AGISim simulationworld, the human teacher talks to it about what it is doing. I wouldlike the human teacher to talk to it in both Lojban++ and English, atthe same time. According to my understanding of Novamente's learningand reasoning methods, this will be the optimal way of getting thesystem to understand English. At once, the system will get aperceptual-motor grounding for the English sentences, plus anunderstanding of the logical meaning of the sentences. I can think ofno better way to help a system understand English. Yes, this is notthe way humans do it. But so what? Novamente does not have a humanbrain, it has a different sort of infrastructure with differentstrengths and weaknesses.What about using "baby English" instead of an artificial language? By this I mean simple English at the level of a 2 or 3 year old child. Baby English has many of the properties that make artificial languages desirable, such as a small vocabulary, simple syntax and lack of ambiguity. Adult English is ambiguous because adults can use vast knowledge and context to resolve ambiguity in complex sentences. Children lack these abilities.I don't believe it is possible to map between natural and structured language without solving the natural language modeling problem first. I don't believe that having structured knowledge or a structured language available makes the problem any easier. It is just something else to learn. Humans learn natural language without having to learn structured languages, grammar rules, knowledge representation, etc. I realize that Novamente is different from the human brain. My argument is based on the structure of natural language, which is vastly different from artificial languages used for knowledge representation. To wit:- Artificial languages are designed to be processed (translated or compiled) in the order: lexical tokenization, syntactic parsing, semantic extraction. This does not work for natural language. The correct order is the order in which children learn: lexical, semantics, syntax. Thus we have successful language models that extract semantics without syntax (such as information retrieval and text categorization), but not vice versa.- Artificial language has a structure optimized for serial processing. Natural language is optimized for parallel processing. We resolve ambiguity and errors using context. Context detection is a type of parallel pattern recognition. Patterns can be letters, groups of letters, words, word categories, phrases, and syntactic structures. We recognize and combine perhaps tens or hundreds of patterns simultaneously by matching to perhaps 10^5 or more from memory. Artificial languages have no such mechanism and cannot tolerate ambiguity or errors.- Natural language has a structure that allows incremental learning. We can add words to the vocabulary one at a time. Likewise for phrases, idioms, classes of words and syntactic structures. Artificial languages must be processed by fixed algorithms. Learning algorithms are unknown.- Natural languages evolve slowly in a social environment. Artificial languages are fixed according to some specificiation.- Children can learn natural languages. Artificial languages are difficult to learn even for adults.- Writing in an artificial language is an iterative process in which the output is checked for errors by a computer and the utterance is revised. Natural language uses both iterative and forward error correction.By "natural language" I include man made languages like Esperanto. Esperanto was designed for communication between humans and has all the other properties of natural language. It lacks irregular verbs and such, but this is really a tiny part of a language's complexity. A natural language like English has a complexity of about 10^9 bits. How much information does it take to list all the irregularities in English like
Re: [agi] Natural versus formal AI interface languages
James Ratcliff wrote: Not necessarily childrens language, as tehy have their own problems and often use the wrong words and rules of grammar, but a simplified english, a reduced rule set. Something like no compound sentences for a start. I believe most everything can be written without compound sentences, and that would greatly reduce the processing complexity, and anaphora resolution as a part of the language rules, so if you reference something in one place it will stay the same throughout the section. Its not quite as natural, but could be understood simply enough by humans as well as computers. One problem I have with all of this, is the super-flowery writing styles of cramming as many words and complex topics all into one sentence. This is a question directed at this whole thread, about simplifying language to communicate with an AI system, so we can at least get something working, and then go from there This rationale is the very same rationale that drove researchers into Blocks World programs. Winograd and SHRDLU, etc. It was a mistake then: it is surely just as much of a mistake now. Richard Loosemore. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: Re: [agi] Natural versus formal AI interface languages
It does not help that words in SHRDLU are grounded in an artificial world. Its failure to scale hints that approaches such as AGI-Sim will have similar problems. You cannot simulate complexity. I of course don't think that SHRDLU vs. AGISim is a fair comparison. Among other counterarguments:the idea is that AGI systems trained in AGISim may then be able to use their learning to operate in the physical world, controlling robots similar to their AGISim simulated robots With this in mind, we have plans to eventually integrate the Pyro robotics control toolkit with AGISim (likely extending Pyro in the process), so that the same code can be used to control physical robots as AGISim simulated robots... Now, you can argue that this just won't work, because (you might say) there is nothing in common between learning perception-cognition-action in a simulated world like AGISim, and learning the same thing in the physical world. You might argue that the relative lack of richness in perceptual stimuli and motoric control makes a tremendous qualitative difference. OK, I admit I cannot rigorously prove this sort of argument false Nor can you prove it true. As with anything else in AGI, we must to some extent go on intuition until someone develops a real mathematical theory of pragmatic AGI, or someone finally creates a working AGI based on their intuition. But at least, you must admit there is a plausible argument to be made that effective AGI operation in a somewhat realistic simulation world can transfer to similar operation in the physical world. We are not talking about SHRDLU here. We are talking about a system that perceives simulated visual stimuli and has to recognize objects as patterns in these stimuli; that acts in the world by sending movement commands to joints; etc. Problems posed to the system need to be recognized by the system in terms of these sensory and motoric primitives, analogously to what happens with a system embedded in the physical world via a physical body. In a similar way, SHRDLU performed well in its artificial, simple world. But how would you measure its performance in a real world? I believe I have addressed this by noting that AGI performance is intended to be portable from AGISim into the physical world. Of course, with any simulated environment there is always the risk of creating an AGI or AI system that is overfit to that simulated environment. However, being aware of that risk, I feel it is going to be that difficult to avoid it. If we are going to study AGI, we need a way to perform tests and measure results. It is not just that we need to know what works and what doesn't. The systems we build will be too complex to know what we have built. How would you measure them? The Turing test is the most widely accepted, but it is somewhat subjective and not really appropriate for an AGI with sensorimotor I/O. I have proposed text compression. It gives hard numbers, but it seems limited to measuring ungrounded language models. What else would you use? Suppose that in 10 years, NARS, Novamente, Cyc, and maybe several other systems all claim to have solved the AGI problem. How would you test their claims? How would you decide the winner? I do not agree that having precise quantitative measures of system intelligence is critical, or even important to AGI. And, deciding which AGI is smarter is not important either -- no more important than deciding whether Ben, Matt or Pei is smarter. Who cares? Different systems may have different strengths and weaknesses, so that who is smarter often explicitly comes down to a subjective value judgment We may ask who is likely to be better at carrying out some particular problem-solving task; we may say that A is generically smarter than B if A is better than B at carrying out *every* problem-solving task (Pareto optimality, sorta), but this is not a very useful notion in practice. Once we have an AGI that can hold an English conversation that appears to trained human scientists to be intelligent and creative, and that makes original discoveries in science or mathematics, then the question of whether it is intelligent or not will cease to be very interesting. That is our mid-term goal with Novamente. I don't see why quantitative measures of intelligence are necessary or even useful along the path to getting there. -- Ben - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: Re: [agi] Natural versus formal AI interface languages
- Original Message From: Ben Goertzel [EMAIL PROTECTED] To: agi@v2.listbox.com Sent: Friday, November 3, 2006 9:28:24 PM Subject: Re: Re: [agi] Natural versus formal AI interface languages I do not agree that having precise quantitative measures of system intelligence is critical, or even important to AGI. The reason I ask is not just to compare different systems (which you can't really do if they serve different purposes), but also to measure progress. When I experiment with language models, I often try many variations, tune parameters, etc., so I need a quick test to see if what I did worked. I can do that very quickly using text compression. I can test tens or hundreds of slightly different models per day and make very precise measurements. Of course it is also useful that I can tell if my model works better or worse than somebody else's model that uses a completely different method. There does not seem to be much cooperation on this list toward the goal of achieving AGI. Everyone has their own ideas. That's OK. The purpose of having a metric is not to make it a race, but to help us communicate what works and what doesn't so we can work together while still pursuing our own ideas. Papers on language modeling do this by comparing different algorithms and reporting the results by word perplexity. So you don't have to re-experiment with various n-gram backoff models, LSA, statistical parsers, etc. You already know a lot about what works and what doesn't. Another reason for measurements is that it makes your goals concrete. How do you define general intelligence? Turing gave us a well defined goal, but there are some shortcomings. The Turing test is subjective, time consuming, isn't appropriate for robotics, and really isn't a good goal if it means deliberately degrading performance in order to appear human. So I am looking for better tests. I don't believe the approach of let's just build it and see what it does is going to produce anything useful. -- Matt Mahoney, [EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: Re: Re: [agi] Natural versus formal AI interface languages
Another reason for measurements is that it makes your goals concrete. How do you define general intelligence? Turing gave us a well defined goal, but there are some shortcomings. The Turing test is subjective, time consuming, isn't appropriate for robotics, and really isn't a good goal if it means deliberately degrading performance in order to appear human. So I am looking for better tests. I don't believe the approach of let's just build it and see what it does is going to produce anything useful. I am happy enough with the long-term goal of independent scientific and mathematical discovery... And, in the short term, I am happy enough with the goals of carrying out the (AGISim versions of) the standard tasks used by development psychologists to study childrens' cognitive behavior... I don't see a real value to precisely quantifying these goals, though... Ben G - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: Re: Re: Re: [agi] Natural versus formal AI interface languages
I am happy enough with the long-term goal of independent scientific and mathematical discovery... And, in the short term, I am happy enough with the goals of carrying out the (AGISim versions of) the standard tasks used by development psychologists to study childrens' cognitive behavior... I don't see a real value to precisely quantifying these goals, though... To give an example of the kind of short-term goal that I think is useful, though, consider the following. We are in early 2007 (if all goes according to plan) going to teach Novamente to carry out a game called iterated Easter Egg hunt -- basically, to carry out an Easter Egg hunt in a room full of other agents ... and then do so over and over again, modeling what the other agents do and adjusting its behavior accordingly. Now, this task has a bit in common with the game Hide-and-Seek. So, you'd expect that a Novamente instance that had been taught iterated Easter Egg Hunt, would also be good at hide-and-seek. So, we want to see that the time required for an NM system to learn hide-and-seek will be less if the NM system has previously learned to play iterated Easter Egg hunt... This sort of goal is, I feel, good for infant-stage AGI education However, I wouldn't want to try to turn it into an objective IQ test. Our goal is not to make the best possible system for playing Easter Egg hunt or hide and seek or fetch or whatever And, in terms of language learning, our initial goal will not be to make the best possible system for conversing in baby-talk... Rather, our goal will be to make a system that can adequately fulfill these early-stage tasks, but in a way that we feel will be indefinitely generalizable to more complex tasks. This, I'm afraid, highlights a general issue with formal quantitative intelligence measures as applied to immature AGI systems/minds. Often the best way to achieve some early-developmental-stage task is going to be an overfitted, narrow-AI type of algorithm, which is not easily extendable to address more complex tasks. This is similar to my complaint about the Hutter Prize. Yah, a superhuman AGI will be an awesome text compressor. But this doesn't mean that the best way to achieve slightly better text compression than current methods is going to be **at all** extensible in the direction of AGI. Matt, you have yet to convince me that seeking to optimize interim quantitative milestones is a meaningful path to AGI. I think it is probably just a path to creating milestone-task-overfit narrow-AI systems without any real AGI-related expansion potential... -- Ben - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: Re: [agi] Natural versus formal AI interface languages
On 11/4/06, Ben Goertzel [EMAIL PROTECTED] wrote: I of course don't think that SHRDLU vs. AGISim is a fair comparison.Agreed. SHRDLU didn't even try to solve the real problems - for the simple and sufficient reason that it was impossible to make a credible attempt at such on the hardware of the day. AGISim (if I understand it correctly) does. Oh, I'm sure the current implementation makes fatal compromises to fit on today's hardware - but the concept doesn't have an _inherent_ plateau the way SHRDLU did, so it leaves room for later upgrade. It's headed in the right compass direction. And, deciding which AGI is smarter is not important either -- no moreimportant than deciding whether Ben, Matt or Pei is smarter.Who cares?Agreed. In practice the market will decide: which system ends up doing useful things in the real world, and therefore getting used? Academic judgements of which is smarter are, well, academic. This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
Re: [agi] Natural versus formal AI interface languages
Pei (2) A true AGI should have the potential to learn any natural Pei language (though not necessarily to the level of native Pei speakers). This embodies an implicit assumption about language which is worth noting. It is possible that the nature of natural language is such that humans could not learn it if they did not have the key preprogrammed in genetically. Much data supports, and many authors would argue, that humans have preprogrammed genetically a predisposition, what I would call a strong inductive bias, to learn grammar of a certain type. It is likely that they would be unable to learn grammar nearly as fast as they do without it, indeed it might be computationally intractable even were they given many lifetimes. Moreover, I argue that language is built on top of a heavy inductive bias to develop a certain conceptual structure, which then renders the names of concepts highly salient so that they can be readily learned. (This explains how we can learn 10 words a day, which children routinely do.) An AGI might in principle be built on top of some other conceptual structure, and have great difficulty comprehending human words-- mapping them onto its concepts, much less learning them. Moreover, it is worth noting the possibility that the amount of computation that might in principle be necessary for learning a natural language can't be bounded as one might think. Historically, natural language was a creation of evolution (or of evolution plus human ingenuity, but since humans were a creation of evolution, and in my view evolution may often work by creating mechanisms that then lead to ``or make other discoveries, we can just consider this for some purposes as a creation of evolution.) Thus, you might posit that the amount of computation necessary for learning a natural language is bounded by the (truly vast) amount of computation that evolution could have devoted to the problem. *But this does not follow*. Evolution did not learn natural language; it created it. To the extent that language is an encryption system, evolution thus *chose* the encryption key, it did not have to decrypt it. Thus in principle at least, learning a natural language without being given the key could be a very hard problem indeed, not something that even evolution would have been capable of. This is discussed in more detail in What is Thought?, ch 12 I believe. Eric Baum - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Natural versus formal AI interface languages
Thats a totally different problem, and considering the massive knowledge whole currently about how the human brain works, we would have some major problems in that area, though it is interesting. One other problem there, what about two way communications? You are proposing to have the brain talk to the AGI and the AGI to the brain correct? You would need to put some pretty strict measures as to exactly what and where those communications would go because there is already some proven projects where they have the computer circuits activating human parts such as arms or legs. But if you turn a full AGI loose in someones brain, there is no guarentee of friendliness, but there is a possibility it could grow in ability to tap into all other areas of the body, physically and mentally, and that would be really bad.Interesting research on brain interactions though:http://news.bbc.co.uk/2/hi/technology/3485918.stmhttp://news.bbc.co.uk/2/hi/science/nature/1871803.stmThe last one here is about a great project that actually imbeds a circuit in a monkeys brain, then they go through a number of experiments, including playing a video game for rewards.About two minutes into one session the female monkey just stops moving the joystick in her hand, and keeps playing the game with her mind alone, she had noticed that she didnt need to use the joystick, just think about it.This is what I want now for typing papers and programming, and could also be used possibly as a teaching tool for AI, just start thinking all the information you know into the computer. Basically this should work very similarly, by thinking about typing, your brain could send the 'letters' to the computer instantly, just about as fast as you could think them. It will be intersting to see how accurate and how fast that would actually be, and if you could then transfer that upwards into words instead of just letters.James Ratcliff Gregory Johnson [EMAIL PROTECTED] wrote: Perhaps there is a shortcut to all of this. Provide the AGI with the hardware and software to jack into one or more human brains and let the bio-software of the human brain be the language interface development tool. I think we are creating some of this the hardware. This also puts AGI in a position to become reliant on humans to interface with other humans and perhaps also allows an AGI to learn the virtues of carbon technology and the value of continuing relationships with humans. Some of the drivers that bring humans together such as social relations and sexual relations perhaps can be learned by an AGI and perhaps we can pussywhip an antisocial AGI into a friendly AGI. Remember the KISS rule , sometimes you can focus only on key areas with enormous complexity and later discover that the result is far more simple than originally envisioned. Morris On 10/31/06, John Scanlon [EMAIL PROTECTED] wrote:One of the major obstacles to real AI is the belief thatknowledge ofa natural language is necessary for intelligence. Ahuman-level intelligent system should be expected to have the ability to learn a natural language, but it is not necessary. It is better to start with a formal language, with unambiguous formal syntax,as the primary interface between human beings and AI systems. This type of language could be called a "para-natural formallanguage." It eliminatesall of the syntactical ambiguity that makes competent use of a natural language so difficult to implement in an AI system. Such a language would also be a member of the class "fifth generation computer language." This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED] This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED] Thank YouJames Ratcliffhttp://falazar.com Get your email and see which of your friends are online - Right on the new Yahoo.com This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Natural versus formal AI interface languages
On 10/31/06, John Scanlon [EMAIL PROTECTED] wrote: One of the major obstacles to real AI is the belief thatknowledge ofa natural language is necessary for intelligence. Ahuman-level intelligent system should be expected to have the ability to learn a natural language, but it is not necessary. It is better to start with a formal language, with unambiguous formal syntax,as the primary interface between human beings and AI systems. This type of language could be called a para-natural formallanguage. It eliminatesall of the syntactical ambiguity that makes competent use of a natural language so difficult to implement in an AI system.Syntactic ambiguity isn't the problem. The reason computers don't understand English is nothing to do with syntax, it's because they don't understand the world.It's easy to parse The cat sat on the mat into sentenceverb sit /verbsubject cat /subjectpreposition on /prepositionobject mat /objecttense past /tense /sentenceBut the computer still doesn't understand the sentence, because it doesn't know what cats, mats and the act of sitting _are_. (The best test of such understanding is not language - it's having the computer draw an animation of the action.) Such a language would also be a member of the class fifth generation computer language.It might form the basis of one, but the hard part would be designing and implementing the functionality, the knowledge, that would need to be shipped with the language to make it useful. This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
RE: [agi] Natural versus formal AI interface languages
Russell Wallace wrote: Syntactic ambiguity isn't the problem. The reason computers don't understand English is nothing to do with syntax, it's because they don't understand the world. It's easy to parse The cat sat on the mat into sentence verb sit /verb subject cat /subject preposition on /preposition object mat /object tense past /tense /sentence But the computer still doesn't understand the sentence, because it doesn't know what cats, mats and the act of sitting _are_. (The best test of such understanding is not language - it's having the computer draw an animation of the action.) Russell, I agree, but it might be clearer if we point out that humans don't understand the world either. We just process these symbols within a more encompassing context. - Jef - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Natural versus formal AI interface languages
- Original Message From: Ben Goertzel [EMAIL PROTECTED] To: agi@v2.listbox.com Sent: Tuesday, October 31, 2006 9:26:15 PM Subject: Re: Re: [agi] Natural versus formal AI interface languages Here is how I intend to use Lojban++ in teaching Novamente. When Novamente is controlling a humanoid agent in the AGISim simulation world, the human teacher talks to it about what it is doing. I would like the human teacher to talk to it in both Lojban++ and English, at the same time. According to my understanding of Novamente's learning and reasoning methods, this will be the optimal way of getting the system to understand English. At once, the system will get a perceptual-motor grounding for the English sentences, plus an understanding of the logical meaning of the sentences. I can think of no better way to help a system understand English. Yes, this is not the way humans do it. But so what? Novamente does not have a human brain, it has a different sort of infrastructure with different strengths and weaknesses. What about using baby English instead of an artificial language? By this I mean simple English at the level of a 2 or 3 year old child. Baby English has many of the properties that make artificial languages desirable, such as a small vocabulary, simple syntax and lack of ambiguity. Adult English is ambiguous because adults can use vast knowledge and context to resolve ambiguity in complex sentences. Children lack these abilities. I don't believe it is possible to map between natural and structured language without solving the natural language modeling problem first. I don't believe that having structured knowledge or a structured language available makes the problem any easier. It is just something else to learn. Humans learn natural language without having to learn structured languages, grammar rules, knowledge representation, etc. I realize that Novamente is different from the human brain. My argument is based on the structure of natural language, which is vastly different from artificial languages used for knowledge representation. To wit: - Artificial languages are designed to be processed (translated or compiled) in the order: lexical tokenization, syntactic parsing, semantic extraction. This does not work for natural language. The correct order is the order in which children learn: lexical, semantics, syntax. Thus we have successful language models that extract semantics without syntax (such as information retrieval and text categorization), but not vice versa. - Artificial language has a structure optimized for serial processing. Natural language is optimized for parallel processing. We resolve ambiguity and errors using context. Context detection is a type of parallel pattern recognition. Patterns can be letters, groups of letters, words, word categories, phrases, and syntactic structures. We recognize and combine perhaps tens or hundreds of patterns simultaneously by matching to perhaps 10^5 or more from memory. Artificial languages have no such mechanism and cannot tolerate ambiguity or errors. - Natural language has a structure that allows incremental learning. We can add words to the vocabulary one at a time. Likewise for phrases, idioms, classes of words and syntactic structures. Artificial languages must be processed by fixed algorithms. Learning algorithms are unknown. - Natural languages evolve slowly in a social environment. Artificial languages are fixed according to some specificiation. - Children can learn natural languages. Artificial languages are difficult to learn even for adults. - Writing in an artificial language is an iterative process in which the output is checked for errors by a computer and the utterance is revised. Natural language uses both iterative and forward error correction. By natural language I include man made languages like Esperanto. Esperanto was designed for communication between humans and has all the other properties of natural language. It lacks irregular verbs and such, but this is really a tiny part of a language's complexity. A natural language like English has a complexity of about 10^9 bits. How much information does it take to list all the irregularities in English like swim-swam, mouse-mice, etc? -- Matt Mahoney, [EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: Re: [agi] Natural versus formal AI interface languages
Yes, teaching an AI in Esperanto would make more sense than teaching it in English ... but, would not serve the same purpose as teaching it in Lojban++ and a natural language in parallel... In fact, an ideal educational programme would probably be to use, in parallel -- an Esperanto-based, rather than English-based, version of Lojban++ -- Esperanto However, I hasten to emphasize that this whole discussion is (IMO) largely peripheral to AGI. The main point is to get the learning algorithms and knowledge representation mechanisms right. (Or if the learning algorithm learns its own KR's, that's fine too...). Once one has what seems like a workable learning/representation framework, THEN one starts talking about the right educational programme. Discussing education in the absence of an understanding of internal learning algorithms is perhaps confusing... Before developing Novamente in detail, I would not have liked the idea of using Lojban++ to help teach an AGI, for much the same reasons that you are now complaining. But now, given the specifics of the Novamente system, it turns out that this approach may actually make teaching the system considerably easier -- and make the system more rapidly approach the point where it can rapidly learn natural language on its own. To use Eric Baum's language, it may be that by interacting with the system in Lojban++, we human teachers can supply the baby Novamente with much of the inductive bias that humans are born with, and that helps us humans to learn natural languages so relatively easily I guess that's a good way to put it. Not that learning Lojban++ is a substitute for learning English, rather that the knowledge gained via interaction in Lojban++ may be a substitute for human babies' language-focused and spacetime-focused inductive bias. Of course, Lojban++ can be used in this way **only** with AGI systems that combine -- a robust reinforcement learning capability -- an explicitly logic-based knowledge representation But Novamente does combine these two factors. I don't expect to convince you that this approach is a good one, but perhaps I have made my motivations clearer, at any rate. I am appreciating this conversation, as it is pushing me to verbally articulate my views more clearly than I had done before. -- Ben G On 11/2/06, Matt Mahoney [EMAIL PROTECTED] wrote: - Original Message From: Ben Goertzel [EMAIL PROTECTED] To: agi@v2.listbox.com Sent: Tuesday, October 31, 2006 9:26:15 PM Subject: Re: Re: [agi] Natural versus formal AI interface languages Here is how I intend to use Lojban++ in teaching Novamente. When Novamente is controlling a humanoid agent in the AGISim simulation world, the human teacher talks to it about what it is doing. I would like the human teacher to talk to it in both Lojban++ and English, at the same time. According to my understanding of Novamente's learning and reasoning methods, this will be the optimal way of getting the system to understand English. At once, the system will get a perceptual-motor grounding for the English sentences, plus an understanding of the logical meaning of the sentences. I can think of no better way to help a system understand English. Yes, this is not the way humans do it. But so what? Novamente does not have a human brain, it has a different sort of infrastructure with different strengths and weaknesses. What about using baby English instead of an artificial language? By this I mean simple English at the level of a 2 or 3 year old child. Baby English has many of the properties that make artificial languages desirable, such as a small vocabulary, simple syntax and lack of ambiguity. Adult English is ambiguous because adults can use vast knowledge and context to resolve ambiguity in complex sentences. Children lack these abilities. I don't believe it is possible to map between natural and structured language without solving the natural language modeling problem first. I don't believe that having structured knowledge or a structured language available makes the problem any easier. It is just something else to learn. Humans learn natural language without having to learn structured languages, grammar rules, knowledge representation, etc. I realize that Novamente is different from the human brain. My argument is based on the structure of natural language, which is vastly different from artificial languages used for knowledge representation. To wit: - Artificial languages are designed to be processed (translated or compiled) in the order: lexical tokenization, syntactic parsing, semantic extraction. This does not work for natural language. The correct order is the order in which children learn: lexical, semantics, syntax. Thus we have successful language models that extract semantics without syntax (such as information retrieval and text categorization), but not vice versa. - Artificial language has a structure optimized for serial
Re: [agi] Natural versus formal AI interface languages
On 11/2/06, Eliezer S. Yudkowsky [EMAIL PROTECTED] wrote: Pei Wang wrote: On 11/2/06, Eric Baum [EMAIL PROTECTED] wrote: Moreover, I argue that language is built on top of a heavy inductive bias to develop a certain conceptual structure, which then renders the names of concepts highly salient so that they can be readily learned. (This explains how we can learn 10 words a day, which children routinely do.) An AGI might in principle be built on top of some other conceptual structure, and have great difficulty comprehending human words-- mapping them onto its concepts, much less learning them. I think any AGI will need the ability to (1) using mental entities (concepts) to summarize percepts and actions, and (2) using concepts to extend past experience to new situations (reasoning). In this sense, the categorization/learning/reasoning (thinking) mechanisms of different AGIs may be very similar to each other, while the contents of their conceptual structures are very different, due to the differences in their sensors and effectors, as well as environments. Pei, I suspect that what Baum is talking about is - metaphorically speaking - the problem of an AI that runs on SVD talking to an AI that runs on SVM. (Singular Value Decomposition vs. Support Vector Machines.) Or the ability of an AI that runs on latent-structure Bayes nets to exchange concepts with an AI that runs on decision trees. Different AIs may carve up reality along different lines, so that even if they label their concepts, it may take considerable extra computing power for one of them to learn the other's concepts - it may not be natural to them. They may not be working in the same space of easily learnable concepts. Of course these examples are strictly metaphorical. But the point is that human concepts may not correspond to anything that an AI can *natively* learn and *natively* process. That is why I tried to distinguish content from mechanism --- a robot with sonar as the only sensor and wheels as the only effectors surely won't categorize the environment in our concepts. However, I tend to believe that the relations among the robot's concepts are more or less what I call inheritance, similarity, and so on, and its reasoning rules are not that different from the ones we use. Can we understand such a language? I'd say yes, to a certain extent, though not fully, as far as there are ways for our experience to be related to that of the robot. A superintelligence, or a sufficiently self-modifying AI, should not be balked by English. A superintelligence should carve up reality into sufficiently fine grains that it can learn any concept computable by our much smaller minds, unless P != NP and the concepts are genuinely encrypted. And a self-modifying AI should be able to natively run whatever it likes. This point, however, Baum may not agree with. I'm afraid that there are no sufficiently fine grains that can serve as the common atoms of different sensorimotor systems. They may categorize the same environment in incompatible ways, which cannot be reduced to a common language with more detailed concepts. Pei -- Eliezer S. Yudkowsky http://singinst.org/ Research Fellow, Singularity Institute for Artificial Intelligence - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Natural versus formal AI interface languages
Eliezer unless P != NP and the concepts are genuinely encrypted. And I am of course assuming P != NP, which seems to me a safe assumption. If P = NP, and mind exploits that fact (which I don't believe) then we are at a serious handicap in producing an AGI till we understand why P = NP, but it will become a lot easier afterward! I'm not, of course, saying that there was some intent or evolutionary advantage to encryption, just that it very naturally occurs. Evolution picks a grammar bias, for example. One is as good as another, more or less, so it picks one. The AGI doesn't get the privilege, though, it has to solve a learning problem, and such learning problems are mostly known to be NP-hard. (We might, of course, give it the grammar bias, rather than requiring it to learn it, but alas, we don't know how to describe it... linguists study this problem, but it has been too hard to solve...) So Pei's comments are in some sense wishes. To be charitable-- maybe I should say beliefs supported by his experience. But they are not established facts. It remains a possibility, supported by reasonable evidence, that language learning may be an intractable additional step on top of building a program achieving other aspects of intelligence. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Natural versus formal AI interface languages
I don't know enough about Novamente to say if your approach would work. Using an artificial language as part of the environment (as opposed to a substitute for natural language) does seem to make sense. I think an interesting goal would be to teach an AGI to write software. If I understand your explanation, this is the same problem. I want to teach the AGI two languages (English and x86-64 machine code), one to talk to me and the other to define its environment. I would like to say to the AGI, write a program to print the numbers 1 through 100, are there any security flaws in this web browser? and ultimately, write a program like yourself, but smarter. This is obviously a hard problem, even if I substitute a more English-like programming language like COBOL. To solve the first example, the AGI needs an adult level understanding of English and arithmetic. To solve the second, it needs a comprehensive world model, including an understanding of how people think and the things they can experience. (If an embedded image can set a cookie, is this a security flaw?). When it can solve the third, we are in trouble (topic for another list). How could such an AGI be built? What would be its architecture? What learning algorithm? What training data? What computational cost? -- Matt Mahoney, [EMAIL PROTECTED] - Original Message From: Ben Goertzel [EMAIL PROTECTED] To: agi@v2.listbox.com Sent: Thursday, November 2, 2006 3:45:42 PM Subject: Re: Re: [agi] Natural versus formal AI interface languages Yes, teaching an AI in Esperanto would make more sense than teaching it in English ... but, would not serve the same purpose as teaching it in Lojban++ and a natural language in parallel... In fact, an ideal educational programme would probably be to use, in parallel -- an Esperanto-based, rather than English-based, version of Lojban++ -- Esperanto However, I hasten to emphasize that this whole discussion is (IMO) largely peripheral to AGI. The main point is to get the learning algorithms and knowledge representation mechanisms right. (Or if the learning algorithm learns its own KR's, that's fine too...). Once one has what seems like a workable learning/representation framework, THEN one starts talking about the right educational programme. Discussing education in the absence of an understanding of internal learning algorithms is perhaps confusing... Before developing Novamente in detail, I would not have liked the idea of using Lojban++ to help teach an AGI, for much the same reasons that you are now complaining. But now, given the specifics of the Novamente system, it turns out that this approach may actually make teaching the system considerably easier -- and make the system more rapidly approach the point where it can rapidly learn natural language on its own. To use Eric Baum's language, it may be that by interacting with the system in Lojban++, we human teachers can supply the baby Novamente with much of the inductive bias that humans are born with, and that helps us humans to learn natural languages so relatively easily I guess that's a good way to put it. Not that learning Lojban++ is a substitute for learning English, rather that the knowledge gained via interaction in Lojban++ may be a substitute for human babies' language-focused and spacetime-focused inductive bias. Of course, Lojban++ can be used in this way **only** with AGI systems that combine -- a robust reinforcement learning capability -- an explicitly logic-based knowledge representation But Novamente does combine these two factors. I don't expect to convince you that this approach is a good one, but perhaps I have made my motivations clearer, at any rate. I am appreciating this conversation, as it is pushing me to verbally articulate my views more clearly than I had done before. -- Ben G On 11/2/06, Matt Mahoney [EMAIL PROTECTED] wrote: - Original Message From: Ben Goertzel [EMAIL PROTECTED] To: agi@v2.listbox.com Sent: Tuesday, October 31, 2006 9:26:15 PM Subject: Re: Re: [agi] Natural versus formal AI interface languages Here is how I intend to use Lojban++ in teaching Novamente. When Novamente is controlling a humanoid agent in the AGISim simulation world, the human teacher talks to it about what it is doing. I would like the human teacher to talk to it in both Lojban++ and English, at the same time. According to my understanding of Novamente's learning and reasoning methods, this will be the optimal way of getting the system to understand English. At once, the system will get a perceptual-motor grounding for the English sentences, plus an understanding of the logical meaning of the sentences. I can think of no better way to help a system understand English. Yes, this is not the way humans do it. But so what? Novamente does not have a human brain, it has a different sort of infrastructure with different strengths and weaknesses. What about using
Re: Re: [agi] Natural versus formal AI interface languages
Hi, I think an interesting goal would be to teach an AGI to write software. If I understand your explanation, this is the same problem. Yeah, it's the same problem. It's a very small step from Lojban to a programming language, and in fact Luke Kaiser and I have talked about making a programming language syntax based on Lojban, using his Speagram program interpreter framework. The nice thing about Lojban is that it does have the flexibility to be used as a pragmatic programming language (tho no one has done this yet), **or** to be used to describe everyday situations in the manner of a natural language How could such an AGI be built? What would be its architecture? What learning algorithm? What training data? What computational cost? Well, I think Novamente is one architecture that can achieve this But I do not know what the computational cost will be, as Novamente is too complicated to support detailed theoretical calculations of its computational cost in realistic situations. I have my estimates of the computational cost, but validating them will have to wait till the project progresses further... Ben - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Natural versus formal AI interface languages
On 11/2/06, Eric Baum [EMAIL PROTECTED] wrote: So Pei's comments are in some sense wishes. To be charitable-- maybe I should say beliefs supported by his experience. But they are not established facts. It remains a possibility, supported by reasonable evidence, that language learning may be an intractable additional step on top of building a program achieving other aspects of intelligence. Of course you are right. We have no fact about AGI until someone build it, and convince the others that it is indeed an AGI, which may take longer than the former step. ;-) As I mentioned before, I haven't done any actual experiment in language learning yet, so my beliefs on this topic have relatively low confidence compared to some of my other beliefs. I'm just not convinced by the arguments about their impossibility. For example, I don't think we know a system that is intelligent in every sense, but cannot understand a human language, even after a reasonably long training period. Pei - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: Re: [agi] Natural versus formal AI interface languages
Hi. It's a very small step from Lojban to a programming language, and in fact Luke Kaiser and I have talked about making a programming language syntax based on Lojban, using his Speagram program interpreter framework. The nice thing about Lojban is that it does have the flexibility to be used as a pragmatic programming language (tho no one has done this yet), **or** to be used to describe everyday situations in the manner of a natural language Yes, in my opinion this **OR** should really be underlined. And I think this is a very big problem -- you can talk about programming *or* talk in everyday manner, but hardly both at the same time. I could recently feel the pain as a friend of mine worked on using Speagram in Wengo (an open source VoIP client) for language control of different commands and actions. The problem is that, even if you manage to get through parsing, context, disambiguation, add some meaningful interaction etc., you end up with a set of commands that is very hard to extend for non-programmer. So basically you can activate a few pre-programmed commands in a quite-natural language *and* you can add new commands in a naturally looking programming language. But, even though this is internally the same language, there is no way to say that you can program in a way that feels natural. It seems to be like this: when you start programming, even though the syntax is still natural, the language gets really awkward and does not resemble the way you would express the same thing naturally. For me it just shows that the real problem is somewhere deeper, in the semantic representation that is underlying it all. Simply the first-order logic or usual programming styles are different from everyday communication. Switching to Lojban might remove the remaining syntax errors, but I don't see how it can help with this bigger problem. Ben, do you think using Lojban can really substantially help or are you counting on Agi-Sim world and Novamente architecture in general, and want to use Lojban just to simplify language analysis? - lk - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: Re: Re: [agi] Natural versus formal AI interface languages
Luke wrote: It seems to be like this: when you start programming, even though the syntax is still natural, the language gets really awkward and does not resemble the way you would express the same thing naturally. For me it just shows that the real problem is somewhere deeper, in the semantic representation that is underlying it all. Simply the first-order logic or usual programming styles are different from everyday communication. Switching to Lojban might remove the remaining syntax errors, but I don't see how it can help with this bigger problem. Ben, do you think using Lojban can really substantially help or are you counting on Agi-Sim world and Novamente architecture in general, and want to use Lojban just to simplify language analysis? Above all I am counting on the Novamente architecture in general However, I do think the Lojban language, properly extended, has a lot of power. Following up on the excellent point you made: I do think that a mode of communication combining aspects of programming with aspects of commonsense natural language communication can be achieved -- and that this will be a fascinating thing. However, I think this can be achieved only AFTER one has a reasonably intelligent proto-AGI system that can take semantically slightly-imprecise statements and automatically map them into fully formalized programming-type statements. Lojban has no syntactic ambiguity but it does allow semantic ambiguity as well as extreme semantic precision. Using Lojban for programming would involve using its capability for extreme semantic precision; using it for commonsense communication involves using its capability for judiciously controlled semantic ambiguity. Using both these capabilities together in a creative way will be easier with a more powerful AI back end... E.g., you'd like to be able to outline the obvious parts of your code in a somewhat ambiguous way (but still, using Lojban, much less ambiguously than would be the case in English), and have the AI figure out the details. But then, the tricky parts of the code would be spelled out in detail using full programming-language-like precision. Of course, it may be that once the AGI is smart enough to be used in this way, it's only a short time after that until the AGI writes all its own code and we become obsolete as coders anyway ;-) -- Ben - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Natural versus formal AI interface languages
Matt, I totally agree with you on Cyc and LISP. To go further, I think Cyc is a dead end because of the assumption that intelligence is dependent on a vast store of knowledge, basically represented in a semantic net. Intelligence should start with the learning of simple patterns in images and some kind of language that can refer to them and their observed behavior. And this involves the training you are talking about. But you don't quite understand the difference between a natural-like formal language and something like LISP. I'm talking about a language that has formal syntax but most importantly has the full expressive power of a natural language (minus the idioms and aesthetic elements like poetry). Now the training of such a system is the problem, and that's the problem that we're all working on. I am just about finished with the parsing of my language, Jinnteera (in ANSI/ISO C++). I have bitmaps coming in from clients to the intelligence engine and some image processing. The next step is the semantic processing of the parse tree of incoming statements. This system, in no way, has any intelligence yet, but it provides the initial framework for experimentation and the developement of AI, using any internal intelligence algorithms of choice. It's basically an AI shell at the moment, and after some more development and polishing, I'm willing to share it with anyone whose interested. - Original Message - From: Matt Mahoney [EMAIL PROTECTED] To: agi@v2.listbox.com Sent: Tuesday, October 31, 2006 9:03 PM Subject: Re: [agi] Natural versus formal AI interface languages Artificial languages that remove ambiguity like Lojban do not bring us any closer to solving the AI problem. It is straightforward to convert between artificial languages and structured knowledge (e.g first order logic), but it is still a hard (AI complete) problem to convert between natural and artificial languages. If you could translate English - Lojban - English, then you could just as well translate, e.g. English - Lojban - Russian. Without a natural language model, you have no access to the vast knowledge base of the Internet, or most of the human race. I know people can learn Lojban, just like they can learn Cycl or LISP. Lets not repeat these mistakes. This is not training, it is programming a knowledge base. This is narrow AI. -- Matt Mahoney, [EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Natural versus formal AI interface languages
John Scanlon wrote: Ben, I did read your stuff on Lojban++, and it's the sort of language I'm talking about. This kind of language lets the computer and the user meet halfway. The computer can parse the language like any other computer language, but the terms and constructions are designed for talking about objects and events in the real world -- rather than for compilation into procedural machine code. Which brings up a question -- is it better to use a language based on term or predicate logic, or one that imitates (is isomorphic to) natural languages? A formal language imitating a natural language would have the same kinds of structures that almost all natural languages have: nouns, verbs, adjectives, prepositions, etc. There must be a reason natural languages almost always follow the pattern of something carrying out some action, in some way, and if transitive, to or on something else. On the other hand, a logical language allows direct translation into formal logic, which can be used to derive all sorts of implications (not sure of the terminology here) mechanically. The problem here is that when people use a language to communicate with each other they fall into the habit of using human, rather than formal, parsings. This works between people, but would play hob with a computer's understanding (if it even had reasonable referrents for most of the terms under discussion). Also, notice one major difference between ALL human languages and computer languages: Human languages rarely use many local variables, computer languages do. Even the words that appear to be local variables in human languages are generally references, rather than variables. This is (partially) because computer languages are designed to describe processes, and human languages are quasi-serial communication protocols. Notice that thoughts are not serial, and generally not translatable into words without extreme loss of meaning. Human languages presume sufficient understanding at the other end of the communication channel to reconstruct a model of what the original thought might have been. So. Lojban++ might be a good language for humans to communicate to an AI with, but it would be a lousy language in which to implement that same AI. But even for this purpose the language needs a verifier to insure that the correct forms are being followed. Ideally such a verifier would paraphrase the statement that it was parsing and emit back to the sender either an error message, or the paraphrased sentence. Then the sender would check that the received sentence matched in meaning the sentence that was sent. (N.B.: The verifier only checks the formal properties of the language to ensure that they are followed. It had no understanding, so it can't check the meaning.) - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Natural versus formal AI interface languages
John,One of the major obstacles to real AI is the belief thatknowledge ofa natural language is necessary for intelligence.I agree. And it's IMO nearly impossible for AGI to learn/understand NL when its only info source is NL. We get some extra [meta] data from our senses when learning NL (which NL itself wasn't designed to cover) and that extra info is often critical for plugging new concepts to our mental-model-of-the-world with all the important (ATM available) links to other concepts. BTW note that the ancient list of 5 senses (reportedly by Aristotle) is pretty obsolete. We just have a lot more than 5 and all of them help us to really understand NL-labeled and NL-not-covered concepts. So, practically, you IMO either need a bunch of (appropriately processed) human like senses (=LOTS of work for developers) OR (if it's [mostly] text I/O based AI) certain degree of formalization (higher than NL) for the input to get the meta data needed for decent understanding. The first alternative IMO requires resources most of us don't have so I go with the second option. Such systems need to learn a lot using some kind of formalized input = too much system-teaching for the dev team and I don't think a typical user would be eager to learn Lojban-like languages (which I see some issues with when it comes to meaning digging anyway) so I think an extra step is needed to really get the computer and the user to meet user-acceptable way (not exactly the halfway). As some of the above implies, languages get clumsy when describing certain types concepts. That's why in my wannabe AGI (which is still more on paper than in a version control system), I'm trying to design a user-AI interface that has a couple of specialized (but easy to use) editors in addition to its language itself.BTW a fellow coder just asked me Can I borrow your eyes?. Obviously, NL is a mess. Sure, AGI should be able to learn it but 1) to learn it well, it requires already having a significant well structured KB and 2) there is a LOT of very important problem solving that does not require being fluent in any NL. Matt,I guess the AI problem is solved, then. I can already communicate with my computer using formal, unambiguous languages. It already does a lot of things better than most humans, like arithmetic, chess, memorizing long lists and recalling them perfectly... AI is, AGI isn't. You are talking about domain specific systems that are unable to build mental models useful for general problem solving.Sorry I did not have a chance to read all the related posts so far.. I'll definitely get back to it later. This stuff is IMO really important for AGI. Sincerely,Jiri JelinekOn 10/31/06, John Scanlon [EMAIL PROTECTED] wrote: One of the major obstacles to real AI is the belief thatknowledge ofa natural language is necessary for intelligence. Ahuman-level intelligent system should be expected to have the ability to learn a natural language, but it is not necessary. It is better to start with a formal language, with unambiguous formal syntax,as the primary interface between human beings and AI systems. This type of language could be called a para-natural formallanguage. It eliminatesall of the syntactical ambiguity that makes competent use of a natural language so difficult to implement in an AI system. Such a language would also be a member of the class fifth generation computer language. This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED] This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Natural versus formal AI interface languages
On 11/1/06, Charles D Hixson wrote: So. Lojban++ might be a good language for humans to communicate to an AI with, but it would be a lousy language in which to implement that same AI. But even for this purpose the language needs a verifier to insure that the correct forms are being followed. Ideally such a verifier would paraphrase the statement that it was parsing and emit back to the sender either an error message, or the paraphrased sentence. Then the sender would check that the received sentence matched in meaning the sentence that was sent. (N.B.: The verifier only checks the formal properties of the language to ensure that they are followed. It had no understanding, so it can't check the meaning.) This discussion reminds me of a story about the United Nations assembly meetings. Normally when a representative is speaking, all the translation staff are jabbering away in tandem with the speaker. But when the German representative starts speaking they all fall silent and sit staring at him. The reason is that they are waiting for the verb to come along. :) Billk - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Natural versus formal AI interface languages
The AGI really does need to be able to read and write english or another natural language to be decently useful, people are just NOT goign to learn or be impressed with a machine that spurts out something incoherent (which they already can do)It is suprising how little actuall semantic ambiguity there is in well written language such as news articles and such. Especially when you take into account the statistical information of english. It may occur, but not often.The telescope/man example is the most ambigous, but even the other example:"He hit the boy with the bat"You can statistically show that "hitting with a bat" is statistically high, and assume it was the tool used.If not, and even so, the AI should model both scenarios as possible.Most of these ambiguities are removed though, with the additional context sentences around them, or people should just be trained to avoid these ambiguities in writing, but not another language indeed.Even without the ambiguity of the texts discussed here, there is no easy formula for mapping english or other sentences directly into any sort of database, using FOL or any others.This is something I am working on and am interested in currently.I am currently seeing how many simple statements can be pulled from the current news articles into an AI information center.JamesMatt Mahoney [EMAIL PROTECTED] wrote: Artificial languages that remove ambiguity like Lojban do not bring us any closer to solving the AI problem. It is straightforward to convert between artificial languages and structured knowledge (e.g first order logic), but it is still a hard (AI complete) problem to convert between natural and artificial languages. If you could translate English - Lojban - English, then you could just as well translate, e.g. English - Lojban - Russian. Without a natural language model, you have no access to the vast knowledge base of the Internet, or most of the human race. I know people can learn Lojban, just like they can learn Cycl or LISP. Lets not repeat these mistakes. This is not training, it is programming a knowledge base. This is narrow AI. -- Matt Mahoney, [EMAIL PROTECTED]-This list is sponsored by AGIRI: http://www.agiri.org/emailTo unsubscribe or change your options, please go to:http://v2.listbox.com/member/[EMAIL PROTECTED]Thank YouJames Ratcliffhttp://falazar.com Cheap Talk? Check out Yahoo! Messenger's low PC-to-Phone call rates. This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Natural versus formal AI interface languages
Forgot to add there is a large amount of syntactic and Word sense disambiguity, but there are some programs out there that handle that to a remarkable extent as well, and I believe can be improved upon.And for many tasks, I dont see any reason not to have some back and forth feedback in the loop for the AI.The "Smartest" response to the "I saw the man with the telescope." sentence to me would be simply:AI: "Did you have the telescope or did the man?"or "Was the man holding the telescope?"James RatcliffJames Ratcliff [EMAIL PROTECTED] wrote: The AGI really does need to be able to read and write english or another natural language to be decently useful, people are just NOT goign to learn or be impressed with a machine that spurts out something incoherent (which they already can do)It is suprising how little actuall semantic ambiguity there is in well written language such as news articles and such. Especially when you take into account the statistical information of english. It may occur, but not often.The telescope/man example is the most ambigous, but even the other example:"He hit the boy with the bat"You can statistically show that "hitting with a bat" is statistically high, and assume it was the tool used.If not, and even so, the AI should model both scenarios as possible.Most of these ambiguities are removed though, with the additional context sentences around them, or people should just be trained to avoid these ambiguities in writing, but not another language indeed.Even without the ambiguity of the texts discussed here, there is no easy formula for mapping english or other sentences directly into any sort of database, using FOL or any others.This is something I am working on and am interested in currently.I am currently seeing how many simple statements can be pulled from the current news articles into an AI information center.JamesMatt Mahoney [EMAIL PROTECTED] wrote: Artificial languages that remove ambiguity like Lojban do not bring us any closer to solving the AI problem. It is straightforward to convert between artificial languages and structured knowledge (e.g first order logic), but it is still a hard (AI complete) problem to convert between natural and artificial languages. If you could translate English - Lojban - English, then you could just as well translate, e.g. English - Lojban - Russian. Without a natural language model, you have no access to the vast knowledge base of the Internet, or most of the human race. I know people can learn Lojban, just like they can learn Cycl or LISP. Lets not repeat these mistakes. This is not training, it is programming a knowledge base. This is narrow AI. -- Matt Mahoney, [EMAIL PROTECTED]-This list is sponsored by AGIRI: http://www.agiri.org/emailTo unsubscribe or change your options, please go to:http://v2.listbox.com/member/[EMAIL PROTECTED]Thank YouJames Ratcliffhttp://falazar.com Cheap Talk? Check out Yahoo! Messenger's low PC-to-Phone call rates. This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED] Thank YouJames Ratcliffhttp://falazar.com Everyone is raving about the all-new Yahoo! Mail. This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Natural versus formal AI interface languages
BillK wrote: On 11/1/06, Charles D Hixson wrote: So. Lojban++ might be a good language for humans to communicate to an AI with, but it would be a lousy language in which to implement that same AI. But even for this purpose the language needs a verifier to insure that the correct forms are being followed. Ideally such a verifier would paraphrase the statement that it was parsing and emit back to the sender either an error message, or the paraphrased sentence. Then the sender would check that the received sentence matched in meaning the sentence that was sent. (N.B.: The verifier only checks the formal properties of the language to ensure that they are followed. It had no understanding, so it can't check the meaning.) This discussion reminds me of a story about the United Nations assembly meetings. Normally when a representative is speaking, all the translation staff are jabbering away in tandem with the speaker. But when the German representative starts speaking they all fall silent and sit staring at him. The reason is that they are waiting for the verb to come along. :) Billk Yeah, it wouldn't be ideal for rapid interaction. But it would help people to maintain adherence to the formal rules, and to notice when they weren't. If you don't have feedback of this nature, the language will evolve different rules, more closely similar to those of natural languages. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Natural versus formal AI interface languages
Perhaps there is a shortcut to all of this. Provide the AGI with the hardware and software to jack into one or more human brains and let the bio-software of the human brain be the language interface development tool. I think we are creating some of this the hardware. This also puts AGI in a position to become reliant on humans to interface with other humans and perhaps also allows an AGI to learn the virtues of carbon technology and the value of continuing relationships with humans. Some of the drivers that bring humans together such as social relations and sexual relations perhaps can be learned by an AGI and perhaps we can pussywhip an antisocial AGI into a friendly AGI. Remember the KISS rule , sometimes you can focus only on key areas with enormous complexity and later discover that the result is far more simple than originally envisioned. Morris On 10/31/06, John Scanlon [EMAIL PROTECTED] wrote: One of the major obstacles to real AI is the belief thatknowledge ofa natural language is necessary for intelligence. Ahuman-level intelligent system should be expected to have the ability to learn a natural language, but it is not necessary. It is better to start with a formal language, with unambiguous formal syntax,as the primary interface between human beings and AI systems. This type of language could be called a para-natural formallanguage. It eliminatesall of the syntactical ambiguity that makes competent use of a natural language so difficult to implement in an AI system. Such a language would also be a member of the class fifth generation computer language. This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED] This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Natural versus formal AI interface languages
- Original Message - From: Gregory Johnson Provide the AGI with the hardware and software to jack into one or more humanbrains and let the bio-software of the human brain be the language interface development tool. Jacking into the human brain? That is hardly a shortcut to AGI, if we are to invent AGI in the next 30 or 40 years. We are a long ways off from being able to use the human brain the way you mention. Some of the drivers that bring humans together such as social relations and sexual relations perhaps can be learned by an AGI andperhaps we can pussywhip an antisocial AGI into a friendly AGI. Could you elaborate on this? I don't see the reliability of comparing an AGI's motivations with human motivation. Mark N This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Natural versus formal AI interface languages
The development of real AI is a progressive evolutionary process. The ability to use natural languages, with even a minimum of fluency, is simply beyond the capacity of any AI technology that exists today. A para-natural language can communicate all the essential meanings of a natural language without the intractable messiness, and can be parsed easily like any other computer language. It's the best choice for the current primitive state of AI technology. The development of human-level natural-language abilities will take as much time as the development of human-level intelligence, and this will not happen right away. Dumb, to less dumb, to somewhat smart, to smart is a necessary progression. - Original Message - From: Richard Loosemore [EMAIL PROTECTED] To: agi@v2.listbox.com Sent: Tuesday, October 31, 2006 9:08 AM Subject: Re: [agi] Natural versus formal AI interface languages John Scanlon wrote: One of the major obstacles to real AI is the belief that knowledge of a natural language is necessary for intelligence. A human-level intelligent system should be expected to have the ability to learn a natural language, but it is not necessary. It is better to start with a formal language, with unambiguous formal syntax, as the primary interface between human beings and AI systems. This type of language could be called a para-natural formal language. It eliminates all of the syntactical ambiguity that makes competent use of a natural language so difficult to implement in an AI system. Such a language would also be a member of the class fifth generation computer language. Not true. If it is too dumb to acquire a natural language then it is too dumb, period. Richard Loosemore. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Natural versus formal AI interface languages
John Scanlon wrote: One of the major obstacles to real AI is the belief that knowledge of a natural language is necessary for intelligence. A human-level intelligent system should be expected to have the ability to learn a natural language, but it is not necessary. It is better to start with a formal language, with unambiguous formal syntax, as the primary interface between human beings and AI systems. This type of language could be called a para-natural formal language. It eliminates all of the syntactical ambiguity that makes competent use of a natural language so difficult to implement in an AI system. Such a language would also be a member of the class fifth generation computer language. Not true. If it is too dumb to acquire a natural language then it is too dumb, period. Richard Loosemore. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Natural versus formal AI interface languages
Let's don't confuse two statements: (1) To be able to use a natural language (so as to passing Turing Test) is not a necessary condition for a system to be intelligent. (2) A true AGI should have the potential to learn any natural language (though not necessarily to the level of native speakers). I agree with both of them, and I don't think they contradict to each other. Pei On 10/31/06, Richard Loosemore [EMAIL PROTECTED] wrote: John Scanlon wrote: One of the major obstacles to real AI is the belief that knowledge of a natural language is necessary for intelligence. A human-level intelligent system should be expected to have the ability to learn a natural language, but it is not necessary. It is better to start with a formal language, with unambiguous formal syntax, as the primary interface between human beings and AI systems. This type of language could be called a para-natural formal language. It eliminates all of the syntactical ambiguity that makes competent use of a natural language so difficult to implement in an AI system. Such a language would also be a member of the class fifth generation computer language. Not true. If it is too dumb to acquire a natural language then it is too dumb, period. Richard Loosemore. - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Natural versus formal AI interface languages
On 10/31/06, Matt Mahoney [EMAIL PROTECTED] wrote: I guess the AI problem is solved, then. I can already communicate with my computer using formal, unambiguous languages. It already does a lot of things better than most humans, like arithmetic, chess, memorizing long lists and recalling them perfectly... Ben G. brought up an excellent example of language ambiguity at a recent workshop: I saw the man with the telescope. Does that mean: (1) I saw the man and I used a telescope to do it. (2) I saw the man, he had a telescope. (3) I performed the action to saw using a telescope instead of using a saw (presumably because I'm a dummy). All three or completely different and also completely valid (unless you throw in life experience which knocks out 3). Just reforming the sentence in a more data-structure-like fashion helps immensely. Just making something up here: (1) I.saw(direct_object=man, using=telescope) (2) I.saw(direct_object=(man, with=telescope)) (3) I.saw_cut(direct_object=man, using=telescope) Getting more formal substantially lowers the work needed to obtain correct meaning. I imagine that's what lojban and its variants are intended to accomplish although I haven't had time to check them out. I also imagine they have a better approach to my off-the-cuff design. If a machine can't pass the Turing test, then what is your definition of intelligence? The ability to learn in a variety of situations without having to be re-engineered in each situation. Also, off-the-cuff, but I feel it's a good start. For example, if we had software that could learn: * List sorting * Go * Pong * Basic Algebra * etc. *without* being hard coded for them or being reprogrammed for anything other that access to input, that would feel pretty darn general to me. But without natural language, it would not be human level. I think human level intelligence is a bigger, harder goal than general intelligence and that the latter will come first. And I would be damned impressed if someone had an AGI capable of all the above even if I had to communicate in lojban to teach it new tricks. -Chuck - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Natural versus formal AI interface languages
I guess the AI problem is solved, then. I can already communicate with my computer using formal, unambiguous languages. It already does a lot of things better than most humans, like arithmetic, chess, memorizing long lists and recalling them perfectly...If a machine can't pass the Turing test, then what is your definition of intelligence?-- Matt Mahoney, [EMAIL PROTECTED]- Original Message From: John Scanlon [EMAIL PROTECTED]To: agi@v2.listbox.comSent: Tuesday, October 31, 2006 8:48:43 AMSubject: [agi] Natural versus formal AI interface languages One of the major obstacles to real AI is the belief thatknowledge ofa natural language is necessary for intelligence. Ahuman-level intelligent system should be expected to have the ability to learn a natural language, but it is not necessary. It is better to start with a formal language, with unambiguous formal syntax,as the primary interface between human beings and AI systems. This type of language could be called a "para-natural formallanguage." It eliminatesall of the syntactical ambiguity that makes competent use of a natural language so difficult to implement in an AI system. Such a language would also be a member of the class "fifth generation computer language." This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED] This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Natural versus formal AI interface languages
John -- See lojban.org and http://www.goertzel.org/papers/lojbanplusplus.pdf -- Ben G On 10/31/06, John Scanlon [EMAIL PROTECTED] wrote: One of the major obstacles to real AI is the belief that knowledge of a natural language is necessary for intelligence. A human-level intelligent system should be expected to have the ability to learn a natural language, but it is not necessary. It is better to start with a formal language, with unambiguous formal syntax, as the primary interface between human beings and AI systems. This type of language could be called a para-natural formal language. It eliminates all of the syntactical ambiguity that makes competent use of a natural language so difficult to implement in an AI system. Such a language would also be a member of the class fifth generation computer language. This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Natural versus formal AI interface languages
In the para-natural formal language I've developed, called Jinnteera, I saw the man with the telescope. would be expressed for each meaning in a declarative phrase as: 1. I did see with a telescope the_man 2. I did see the man which did have a telescope 3. I saw with a telescope the_man or I use a telescope for action (saw the_man) (where saw has the meaning of saw a 2x4, never see, which always takes the same form and means to view) - Original Message - From: Chuck Esterbrook [EMAIL PROTECTED] To: agi@v2.listbox.com Sent: Tuesday, October 31, 2006 12:58 PM Subject: Re: [agi] Natural versus formal AI interface languages On 10/31/06, Matt Mahoney [EMAIL PROTECTED] wrote: I guess the AI problem is solved, then. I can already communicate with my computer using formal, unambiguous languages. It already does a lot of things better than most humans, like arithmetic, chess, memorizing long lists and recalling them perfectly... Ben G. brought up an excellent example of language ambiguity at a recent workshop: I saw the man with the telescope. Does that mean: (1) I saw the man and I used a telescope to do it. (2) I saw the man, he had a telescope. (3) I performed the action to saw using a telescope instead of using a saw (presumably because I'm a dummy). All three or completely different and also completely valid (unless you throw in life experience which knocks out 3). Just reforming the sentence in a more data-structure-like fashion helps immensely. Just making something up here: (1) I.saw(direct_object=man, using=telescope) (2) I.saw(direct_object=(man, with=telescope)) (3) I.saw_cut(direct_object=man, using=telescope) Getting more formal substantially lowers the work needed to obtain correct meaning. I imagine that's what lojban and its variants are intended to accomplish although I haven't had time to check them out. I also imagine they have a better approach to my off-the-cuff design. If a machine can't pass the Turing test, then what is your definition of intelligence? The ability to learn in a variety of situations without having to be re-engineered in each situation. Also, off-the-cuff, but I feel it's a good start. For example, if we had software that could learn: * List sorting * Go * Pong * Basic Algebra * etc. *without* being hard coded for them or being reprogrammed for anything other that access to input, that would feel pretty darn general to me. But without natural language, it would not be human level. I think human level intelligence is a bigger, harder goal than general intelligence and that the latter will come first. And I would be damned impressed if someone had an AGI capable of all the above even if I had to communicate in lojban to teach it new tricks. -Chuck - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Natural versus formal AI interface languages
Ben, I did read your stuff on Lojban++, and it's the sort of language I'm talking about. This kind of language lets the computer and the user meet halfway. The computer can parse the language like any other computer language, but the terms and constructions are designed for talking about objects and events in the real world -- rather than for compilation into procedural machine code. Which brings up a question -- is it better to use a language based on term or predicate logic, or one that imitates (is isomorphic to) natural languages? A formal language imitating a natural language would have the same kinds of structures that almost all natural languages have: nouns, verbs, adjectives, prepositions, etc. There must be a reason natural languages almost always follow the pattern of something carrying out some action, in some way, and if transitive, to or on something else. On the other hand, a logical language allows direct translation into formal logic, which can be used to derive all sorts of implications (not sure of the terminology here) mechanically. - Original Message - From: Ben Goertzel [EMAIL PROTECTED] To: agi@v2.listbox.com Sent: Tuesday, October 31, 2006 12:24 PM Subject: Re: [agi] Natural versus formal AI interface languages John -- See lojban.org and http://www.goertzel.org/papers/lojbanplusplus.pdf -- Ben G On 10/31/06, John Scanlon [EMAIL PROTECTED] wrote: One of the major obstacles to real AI is the belief that knowledge of a natural language is necessary for intelligence. A human-level intelligent system should be expected to have the ability to learn a natural language, but it is not necessary. It is better to start with a formal language, with unambiguous formal syntax, as the primary interface between human beings and AI systems. This type of language could be called a para-natural formal language. It eliminates all of the syntactical ambiguity that makes competent use of a natural language so difficult to implement in an AI system. Such a language would also be a member of the class fifth generation computer language. This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: Re: [agi] Natural versus formal AI interface languages
For comparison, here are some versions of I saw the man with the telescope in Lojban++ ... [ http://www.goertzel.org/papers/lojbanplusplus.pdf ] 1) mi pu see le man sepi'o le telescope I saw the man, using the telescope as a tool 2) mi pu see le man pe le telescope I saw the man who was with the telescope, and not some other man 3) mi pu see le man ne le telescope I saw the man, and he happened to be with the telescope 4) mi pu saw le man sepi'o le telescope I carried out a sawing action on the man, using the telescope as a tool Each of these can be very simply and unambiguously translated into predicate logic, using the Lojban++ cmavo (function words) as semantic primitives. Some notes on Lojban++ as used in these very simple examples: -- pu is an article indicating past tense. -- mi means me/I -- sepi'o means basically the following item is used as a tool in the predicate under discussion -- le is sort of like the -- pe is association -- ne is incidental association -- in example 4, the parser must figure out that the action rather than object meaning of saw is intended because two arguments are provided (mi, and le man) Anyway, I consider the creation of a language that is suitable for human-computer communication about everyday or scientific phenomena, and that is minimally ambiguous syntactically and semantically, to be a solved problem. It was already basically solved by Lojban, but Lojban suffers from a shortage-of-vocabulary issue which Lojban++ remedies. There is a need for someone to write a Lojban++ parser and semantic mapper, but this is a straightforward though definitely not trivial task. As discussed before, I feel the use of Lojban++ may be valuable in order to help with the early stages of teaching an AGI. I disagree that if an AGI system is smart, it can just learn English. Human babies take a long time to learn English or other natural languages, and they have the benefit of some as yet unknown amount of inbuilt wiring (inductive bias) to help them. There is nothing wrong with taking explicit steps to make it easier to transform a powerful learning system into an intelligent, communicative mind... -- Ben G - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: Re: [agi] Natural versus formal AI interface languages
Hi, Which brings up a question -- is it better to use a language based on term or predicate logic, or one that imitates (is isomorphic to) natural languages? A formal language imitating a natural language would have the same kinds of structures that almost all natural languages have: nouns, verbs, adjectives, prepositions, etc. There must be a reason natural languages almost always follow the pattern of something carrying out some action, in some way, and if transitive, to or on something else. On the other hand, a logical language allows direct translation into formal logic, which can be used to derive all sorts of implications (not sure of the terminology here) mechanically. I think the Lojban strategy -- of parsing into formal logic -- is the best approach, because the NL categories that you mention are wrapped up with all sorts of irritating semantic ambiguities... -- Ben G - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Natural versus formal AI interface languages
Pei Wang wrote: Let's don't confuse two statements: (1) To be able to use a natural language (so as to passing Turing Test) is not a necessary condition for a system to be intelligent. (2) A true AGI should have the potential to learn any natural language (though not necessarily to the level of native speakers). I agree with both of them, and I don't think they contradict to each other. Natural language isn't. Humans have one specific idiosyncratic built-in grammar, and we might have serious trouble learning to communicate in anything else - especially if the language was being used by a mind quite unlike our own. Even a programming language is still something that humans made, and how many people do you know who can *seriously*, not-jokingly, think in syntactical C++ the way they can think in English? I certainly think that something could be humanish-level intelligent in terms of optimization ability, and not be able to learn English, if it had a sufficiently alien cognitive architecture - nor would we be able to learn its languge. Of course you can't be superintelligent and unable to speak English - *that* wouldn't make any sense. I assume that's what you mean by true AGI above. -- Eliezer S. Yudkowsky http://singinst.org/ Research Fellow, Singularity Institute for Artificial Intelligence - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: Re: [agi] Natural versus formal AI interface languages
Eliezer wrote: Natural language isn't. Humans have one specific idiosyncratic built-in grammar, and we might have serious trouble learning to communicate in anything else - especially if the language was being used by a mind quite unlike our own. Well, some humans have learned to communicate in Lojban quite effectively. It's slow and sometimes painful and sometimes delightful, but definitely possible, and there is no NL syntax involved... Even a programming language is still something that humans made, and how many people do you know who can *seriously*, not-jokingly, think in syntactical C++ the way they can think in English? One (and it's not me) ben g - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Natural versus formal AI interface languages
Artificial languages that remove ambiguity like Lojban do not bring us any closer to solving the AI problem. It is straightforward to convert between artificial languages and structured knowledge (e.g first order logic), but it is still a hard (AI complete) problem to convert between natural and artificial languages. If you could translate English - Lojban - English, then you could just as well translate, e.g. English - Lojban - Russian. Without a natural language model, you have no access to the vast knowledge base of the Internet, or most of the human race. I know people can learn Lojban, just like they can learn Cycl or LISP. Lets not repeat these mistakes. This is not training, it is programming a knowledge base. This is narrow AI. -- Matt Mahoney, [EMAIL PROTECTED] - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: Re: [agi] Natural versus formal AI interface languages
I know people can learn Lojban, just like they can learn Cycl or LISP. Lets not repeat these mistakes. This is not training, it is programming a knowledge base. This is narrow AI. -- Matt Mahoney, [EMAIL PROTECTED] You seem not to understand the purpose of using Lojban to help teach an AI. Of course it is not a substitute for teaching an AI a natural language. It is simply a tool to help beef up the understanding of certain types of AI systems to the point where they are ready to robustly understand natural language Just because humans don't learn this way doesn't mean some kinds of AI's shouldn't. And, just because Cyc is associated with a poor theory of AI education, doesn't mean that all logic-based AI systems are. (Similarly, just because backprop NN's are associated with a poor theory of AI education, doesn't mean all NN systems necessarily are.) Here is how I intend to use Lojban++ in teaching Novamente. When Novamente is controlling a humanoid agent in the AGISim simulation world, the human teacher talks to it about what it is doing. I would like the human teacher to talk to it in both Lojban++ and English, at the same time. According to my understanding of Novamente's learning and reasoning methods, this will be the optimal way of getting the system to understand English. At once, the system will get a perceptual-motor grounding for the English sentences, plus an understanding of the logical meaning of the sentences. I can think of no better way to help a system understand English. Yes, this is not the way humans do it. But so what? Novamente does not have a human brain, it has a different sort of infrastructure with different strengths and weaknesses. If it results in general intelligence, it is not narrow AI. The goal of this teaching methodology is to give Novamente a general conceptual understanding, using which it can flexibly generalize its understanding to progressively more and more complex situations. This is not what we are doing yet, mainly because we lack a Lojban++ parser still (just a matter of a few man-months of effort, but we have other priorities), but it is in the queue and we will get there in time, as resources permit... -- Ben G - This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/[EMAIL PROTECTED]
