Response to Mark Waser Mon 11/12/2007 2:42 PM post.
>MARK>>>> Remember that the brain is *massively* parallel. Novamente and any other linear (or minorly-parallel) system is *not* going to work in the same fashion as the brain. Novamente can be parallelized to some degree but *not* to anywhere near the same degree as the brain. I love your speculation and agree with it -- but it doesn't match near-term reality. We aren't going to have brain-equivalent parallelism anytime in the near future. ED>>>> I think in five to ten years there could be computers capable of providing every bit as much parallelism as the brain at prices that will allow thousands or hundreds of thousands of them to be sold. But it is not going to happen overnight. Until then the lack of brain level hardware is going to limit AGI. But there are still a lot of high value system that could be built on say $100K to $10M of hardware. You claim we really need experience with computing and controlling activation over large atom tables. I would argue that obtaining such experience should be a top priority for government funders. >MARK>>>> The node/link architecture is very generic and can be used for virtually anything. There is no rational way to attack it. It is, I believe, going to be the foundation for any system since any system can easily be translated into it. Attacking the node/link architecture is like attacking assembly language or machine code. Now -- are you going to write your AGI in assembly language? If you're still at the level of arguing node/link, we're not communicating well. ED>>>> nodes and links are what patterns are made of, and each static pattern can have an identifying node associated with it as well as the nodes and links representing its sub-patterns, elements, the compositions of which it is part, it associations, etc. The system automatically organize patterns into a gen/comp hierarchy. So, I am not just dealing at a node and link level, but they are the basic building blocks. >MARK>>>> ... I *AM* saying that the necessity of using probabilistic reasoning for day-to-day decision-making is vastly over-rated and has been a horrendous side-road for many/most projects because they are attempting to do it in situations where it is NOT appropriate. The "increased, almost ubiquitous adaptation of probabilistic methods" is the herd mentality in action (not to mention the fact that it is directly orthogonal to work thirty years older). Most of the time, most projects are using probabilistic methods to calculate a tenth place decimal of a truth value when their data isn't even sufficient for one. If you've got a heavy-duty discovery system, probabilistic methods are ideal. If you're trying to derive probabilities from a small number of English statements (like "this raven is white" and "most ravens are black"), you're seriously on the wrong track. If you go on and on about how humans don't understand Bayesian reasoning, you're both correct and clueless in not recognizing that your very statement points out how little Bayesian reasoning has to do with most general intelligence. Note, however, that I *do* believe that probabilistic methods *are* going to be critically important for activation for attention, etc. ED>>>> I agree that many approaches accord too much importance to the numerical accuracy and Bayesian purity of their approach, and not enough importance on the justification for the Bayesian formulations they use. I know of one case where I suggested using information that would almost certainly have improved a perception process and the suggestion was refused because it would not fit within the systems probabilistic framework. At an AAAI conference in 1997 I talked to a programmer for a big defense contractor who said he as a fan of fuzzy logic system; that they were so much more simple to get up an running because you didn't have to worry about probabilistic purity. He said his group that used fuzzy logic was getting things out the door that worked faster than the more probability limited competition. So obviously there is something to say for not letting probabilistic purity get in the way of more reasonable approaches. But I still think probabilities are darn important. Even your this raven is white and most ravens are black example involves notions of probability. We attribute probabilities to such statements based on experience with the source of such statements or similar sources of information, and the concept most is a probabilistic one. The reason we humans are so good at reasoning from small data is based on our ability to estimate rough probabilities from similar or generic patterns. >MARK>>>> ....The problem with probability-based conflict resolution is that it is a hack to get around insufficient knowledge rather than an attempt to figure out how to get more knowledge.... ED>>>> This agrees with what I said above about not putting enough emphasis on selecting what probabilistic formulas are appropriate. But it doesnt argue against the importance of probabilities It argues against using them blindly. >>ED>>>> So by operating with small amounts of data how small, very roughly, are you talking about. And are you only talking about the active goals or sources of activation, that will be small or are you saying that all the computation in the system will only be dealing with a small amount of data within, for example, one second of the processing of human-level system operating at human-level speed? >MARK>>>> I mean like the way humans reason, there is only concentration on a small number of objects -- which are only one link away from an almost inconceivable number of related things -- and then the brain can jump at least three of these links with lightning rapidity. ED>>>> So this implies you are not arguing against the idea that AGI will be dealing with massive data, just that that use will be focused by a concentration on a relatively small number of sources of activation at once. >MARK>>>> Ask Ben how much actual work has been done on activation control in very large, very sparse atom spaces in Novamente. He'll tell you that it's a project for when he's further along. I'll insist (as will Richard) that if it isn't baked in from the very beginning, you're probably going to have to go back to the beginning to repair the lack. ED>>>> It is exactly such research I want to see funded. It strikes me as one of the key things we must learn to do well to make powerful AGI. But I think even with some fairly dumb activation control systems you could get useful results. Such results would not be at all human-level in may ways, but in other ways they could be much more powerful because such systems could deal with many more explicit facts and could input and output information at a much higher rate than humans. For example, what is the equivalent of the activation control (or search) algorithm in Google sets. They operate over huge data. I bet the algorithm for calculating their search or activation is relatively simple (much, much, much less than a PhD theses) and look what they can do. So I think one path is to come up with applications that can use and reason with large data, having roughly world knowledge-like sparseness, (such as NL data) and start with relatively simple activation algorithms and develop then from the ground up. >MARK>>>> P.S. Oh yeah -- if you were public enemy number one, I wouldn't bother answering you (and I probably should lay off of the fan-boy crap :-). ED>>>> Thanks. I admit I am impressed with Novamente. Since its the best AGI architecture I currently know of; I am impressed with Ben; believe there is a high probability all the gaps you address could be largely fixed within five years with deep funding (which may never come); and since I want to get such deep funding for just the type of large atom-base work you say is so critical, I think it is important to focus on the potential for greatness that Novamente and somewhat similar systems have, rather than only think of its current gaps and potential problems. But of course, at the same time, we must look for and try to understand its gaps and potential problems so that we can remove them. Ed Porter -----Original Message----- From: Mark Waser [mailto:[EMAIL PROTECTED] Sent: Monday, November 12, 2007 2:42 PM To: [email protected] Subject: Re: [agi] What best evidence for fast AI? >> It is NOT clear that Novamente documentation is NOT enabling, or could not be made enabling, with, say, one man year of work. Strong argument could be made both ways. I believe that Ben would argue that Novamente documentation is NOT enabling even with one man-year of work. Ben? There is still way to much *research* work to be done. >> But the standard for non-enablement is very arguably weaker than not requiring a miracle. It would be more like "not requiring a leap of creativity that is outside the normal skill of talented PhDs trained in related fields". >> So although your position is reasonable, I hope you understand so is that on the other side. My meant-to-be-humorous miracle phrasing is clearly throwing you. The phrase "not requiring a leap of creativity that is outside the normal skill of talented PhDs trained in related fields" works for me. Novamente is *definitely* not there yet. I'm rather sure that Ben would agree -- as in, I'm not on the other side, *you* are on the other side from the system's designer. Again, Ben please feel free to chime in. >> <much scaling stuff> Remember that the brain is *massively* parallel. Novamente and any other linear (or minorly-parallel) system is *not* going to work in the same fashion as the brain. Novamente can be parallelized to some degree but *not* to anywhere near the same degree as the brain. I love your speculation and agree with it -- but it doesn't match near-term reality. We aren't going to have brain-equivalent parallelism anytime in the near future. >> with regard to serious review of memory design I dont know what you mean. Are you attacking the node, link architecture, or what? The node/link architecture is very generic and can be used for virtually anything. There is no rational way to attack it. It is, I believe, going to be the foundation for any system since any system can easily be translated into it. Attacking the node/link architecture is like attacking assembly language or machine code. Now -- are you going to write your AGI in assembly language? If you're still at the level of arguing node/link, we're not communicating well. >> I dont understand this. If there as been one major transformation in AI since the mid-80s it is the increased, almost ubiquitous adaptation of probabilistic methods. Are you claiming probabilistic reasoning is not important?. It depends upon what you mean by probabilistic reasoning. I *AM* saying that the necessity of using probabilistic reasoning for day-to-day decision-making is vastly over-rated and has been a horrendous side-road for many/most projects because they are attempting to do it in situations where it is NOT appropriate. The "increased, almost ubiquitous adaptation of probabilistic methods" is the herd mentality in action (not to mention the fact that it is directly orthogonal to work thirty years older). Most of the time, most projects are using probabilistic methods to calculate a tenth place decimal of a truth value when their data isn't even sufficient for one. If you've got a heavy-duty discovery system, probabilistic methods are ideal. If you're trying to derive probabilities from a small number of English statements (like "this raven is white" and "most ravens are black"), you're seriously on the wrong track. If you go on and on about how humans don't understand Bayesian reasoning, you're both correct and clueless in not recognizing that your very statement points out how little Bayesian reasoning has to do with most general intelligence. Note, however, that I *do* believe that probabilistic methods *are* going to be critically important for activation for attention, etc. >> With regard to knowledge-conflict-resolution, Novamentes probabilistic reasoning is designed to deal with it. Most of the other system I know of that deal with knowledge-conflict-resolution, such as constraint relaxation techniques, are probability based. This is where I believe that probabilistic reasoning is most often improperly used though I don't believe that "most" constraint-relaxation systems are probability-based (except, occasionally as an add-on to just why a given constraint was relaxed rather than another). The problem with probability-based conflict resolution is that it is a hack to get around insufficient knowledge rather than an attempt to figure out how to get more knowledge. It works because you always take the highest probability choice -- except when the system tells you that the sauna is hot because it doesn't know about the ice frozen over the top. In data-rich constrained environments, probabilistic reasoning works (and neural networks are very successful). In every day life . . . . it still works because all your probabilities are near 100% . . . . except when they suddenly aren't. >> So by operating with small amounts of data how small, very roughly, are you talking about. And are you only talking about the active goals or sources of activation, that will be small or are you saying that all the computation in the system will only be dealing with a small amount of data within, for example, one second of the processing of human-level system operating at human-level speed? I mean like the way humans reason, there is only concentration on a small number of objects -- which are only one link away from an almost inconceivable number of related things -- and then the brain can jump at least three of these links with lightning rapidity. Once again, the brain is *massively* parallel and operates with a *huge* sparse matrix. Activation is *far* more important than truth probabilities and much of the focus is the other way (and activation is a really tough nut to solve as you rightly point out with your comments about activation control). Ask Ben how much actual work has been done on activation control in very large, very sparse atom spaces in Novamente. He'll tell you that it's a project for when he's further along. I'll insist (as will Richard) that if it isn't baked in from the very beginning, you're probably going to have to go back to the beginning to repair the lack. Mark P.S. Oh yeah -- if you were public enemy number one, I wouldn't bother answering you (and I probably should lay off of the fan-boy crap :-). _____ This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/? <http://v2.listbox.com/member/?&; > & ----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=64591937-f9a609
