I think that there are two basic directions to better the Novamente architecture: the one Mark talks about more integration of MOSES with PLN and RL theory
On 11/13/07, Edward W. Porter <[EMAIL PROTECTED]> wrote: > Response to Mark Waser Mon 11/12/2007 2:42 PM post. > > > > >MARK>>>> Remember that the brain is *massively* parallel. Novamente and > any other linear (or minorly-parallel) system is *not* going to work in > the same fashion as the brain. Novamente can be parallelized to some > degree but *not* to anywhere near the same degree as the brain. I love > your speculation and agree with it -- but it doesn't match near-term > reality. We aren't going to have brain-equivalent parallelism anytime in > the near future. > > > > ED>>>> I think in five to ten years there could be computers capable of > providing every bit as much parallelism as the brain at prices that will > allow thousands or hundreds of thousands of them to be sold. > > > > But it is not going to happen overnight. Until then the lack of brain > level hardware is going to limit AGI. But there are still a lot of high > value system that could be built on say $100K to $10M of hardware. > > > > You claim we really need experience with computing and controlling > activation over large atom tables. I would argue that obtaining such > experience should be a top priority for government funders. > > > > >MARK>>>> The node/link architecture is very generic and can be used for > virtually anything. There is no rational way to attack it. It is, I > believe, going to be the foundation for any system since any system can > easily be translated into it. Attacking the node/link architecture is > like attacking assembly language or machine code. Now -- are you going to > write your AGI in assembly language? If you're still at the level of > arguing node/link, we're not communicating well. > > > > ED>>>> nodes and links are what patterns are made of, and each static > pattern can have an identifying node associated with it as well as the > nodes and links representing its sub-patterns, elements, the compositions > of which it is part, it associations, etc. The system automatically > organize patterns into a gen/comp hierarchy. So, I am not just dealing at > a node and link level, but they are the basic building blocks. > > > > > > >MARK>>>> ... I *AM* saying that the necessity of using probabilistic > reasoning for day-to-day decision-making is vastly over-rated and has been > a horrendous side-road for many/most projects because they are attempting > to do it in situations where it is NOT appropriate. The "increased, > almost ubiquitous adaptation of probabilistic methods" is the herd > mentality in action (not to mention the fact that it is directly > orthogonal to work thirty years older). Most of the time, most projects > are using probabilistic methods to calculate a tenth place decimal of a > truth value when their data isn't even sufficient for one. If you've got > a heavy-duty discovery system, probabilistic methods are ideal. If you're > trying to derive probabilities from a small number of English statements > (like "this raven is white" and "most ravens are black"), you're seriously > on the wrong track. If you go on and on about how humans don't understand > Bayesian reasoning, you're both correct and clueless in not recognizing > that your very statement points out how little Bayesian reasoning has to > do with most general intelligence. Note, however, that I *do* believe > that probabilistic methods *are* going to be critically important for > activation for attention, etc. > > > > ED>>>> I agree that many approaches accord too much importance to the > numerical accuracy and Bayesian purity of their approach, and not enough > importance on the justification for the Bayesian formulations they use. > I know of one case where I suggested using information that would almost > certainly have improved a perception process and the suggestion was > refused because it would not fit within the system's probabilistic > framework. At an AAAI conference in 1997 I talked to a programmer for a > big defense contractor who said he as a fan of fuzzy logic system; that > they were so much more simple to get up an running because you didn't have > to worry about probabilistic purity. He said his group that used fuzzy > logic was getting things out the door that worked faster than the more > probability limited competition. So obviously there is something to say > for not letting probabilistic purity get in the way of more reasonable > approaches. > > > > But I still think probabilities are darn important. Even your "this raven > is white" and "most ravens are black" example involves notions of > probability. We attribute probabilities to such statements based on > experience with the source of such statements or similar sources of > information, and the concept "most" is a probabilistic one. The reason we > humans are so good at reasoning from small data is based on our ability to > estimate rough probabilities from similar or generic patterns. > > > > >MARK>>>> ....The problem with probability-based conflict resolution is > that it is a hack to get around insufficient knowledge rather than an > attempt to figure out how to get more knowledge.... > > > > ED>>>> This agrees with what I said above about not putting enough > emphasis on selecting what probabilistic formulas are appropriate. But it > doesn't argue against the importance of probabilities It argues against > using them blindly. > > > > > >>ED>>>> So by "operating with small amounts of data" how small, very > roughly, are you talking about. And are you only talking about the active > goals or sources of activation, that will be small or are you saying that > all the computation in the system will only be dealing with a small amount > of data within, for example, one second of the processing of human-level > system operating at human-level speed? > > > > >MARK>>>> I mean like the way humans reason, there is only concentration > on a small number of objects -- which are only one link away from an > almost inconceivable number of related things -- and then the brain can > jump at least three of these links with lightning rapidity. > > > > ED>>>> So this implies you are not arguing against the idea that AGI will > be dealing with massive data, just that that use will be focused by a > concentration on a relatively small number of sources of activation at > once. > > > > > > >MARK>>>> Ask Ben how much actual work has been done on activation > control in very large, very sparse atom spaces in Novamente. He'll tell > you that it's a project for when he's further along. I'll insist (as will > Richard) that if it isn't baked in from the very beginning, you're > probably going to have to go back to the beginning to repair the lack. > > > > ED>>>> It is exactly such research I want to see funded. It strikes me > as one of the key things we must learn to do well to make powerful AGI. > But I think even with some fairly dumb activation control systems you > could get useful results. Such results would not be at all human-level in > may ways, but in other ways they could be much more powerful because such > systems could deal with many more explicit facts and could input and > output information at a much higher rate than humans. > > > > For example, what is the equivalent of the activation control (or search) > algorithm in Google sets. They operate over huge data. I bet the > algorithm for calculating their search or activation is relatively simple > (much, much, much less than a PhD theses) and look what they can do. So I > think one path is to come up with applications that can use and reason > with large data, having roughly world knowledge-like sparseness, (such as > NL data) and start with relatively simple activation algorithms and > develop then from the ground up. > > > > >MARK>>>> P.S. Oh yeah -- if you were public enemy number one, I > wouldn't bother answering you (and I probably should lay off of the > fan-boy crap :-). > > > > ED>>>> Thanks. > > > > I admit I am impressed with Novamente. Since it's the best AGI > architecture I currently know of; I am impressed with Ben; believe there > is a high probability all the gaps you address could be largely fixed > within five years with deep funding (which may never come); and since I > want to get such deep funding for just the type of large atom-base work > you say is so critical, I think it is important to focus on the potential > for greatness that Novamente and somewhat similar systems have, rather > than only think of its current gaps and potential problems. > > > > But of course, at the same time, we must look for and try to understand > its gaps and potential problems so that we can remove them. > > > > Ed Porter > > > > > -----Original Message----- > From: Mark Waser [mailto:[EMAIL PROTECTED] > Sent: Monday, November 12, 2007 2:42 PM > To: [email protected] > Subject: Re: [agi] What best evidence for fast AI? > > > >> It is NOT clear that Novamente documentation is NOT enabling, or could > not be made enabling, with, say, one man year of work. Strong argument > could be made both ways. > > I believe that Ben would argue that Novamente documentation is NOT > enabling even with one man-year of work. Ben? There is still way to much > *research* work to be done. > > >> But the standard for non-enablement is very arguably weaker than not > requiring a miracle. It would be more like "not requiring a leap of > creativity that is outside the normal skill of talented PhDs trained in > related fields". > > >> So although your position is reasonable, I hope you understand so is > that on the other side. > > > My meant-to-be-humorous miracle phrasing is clearly throwing you. The > phrase "not requiring a leap of creativity that is outside the normal > skill of talented PhDs trained in related fields" works for me. Novamente > is *definitely* not there yet. I'm rather sure that Ben would agree -- as > in, I'm not on the other side, *you* are on the other side from the > system's designer. Again, Ben please feel free to chime in. > > >> <much scaling stuff> > > Remember that the brain is *massively* parallel. Novamente and any > other linear (or minorly-parallel) system is *not* going to work in the > same fashion as the brain. Novamente can be parallelized to some degree > but *not* to anywhere near the same degree as the brain. I love your > speculation and agree with it -- but it doesn't match near-term reality. > We aren't going to have brain-equivalent parallelism anytime in the near > future. > > >> "with regard to serious review of memory design" I don't know what you > mean. Are you attacking the node, link architecture, or what? > > The node/link architecture is very generic and can be used for > virtually anything. There is no rational way to attack it. It is, I > believe, going to be the foundation for any system since any system can > easily be translated into it. Attacking the node/link architecture is > like attacking assembly language or machine code. Now -- are you going to > write your AGI in assembly language? If you're still at the level of > arguing node/link, we're not communicating well. > > >> I don't understand this. If there as been one major transformation in > AI since the mid-80's it is the increased, almost ubiquitous adaptation of > probabilistic methods. Are you claiming probabilistic reasoning is not > important?. > > It depends upon what you mean by probabilistic reasoning. I *AM* > saying that the necessity of using probabilistic reasoning for day-to-day > decision-making is vastly over-rated and has been a horrendous side-road > for many/most projects because they are attempting to do it in situations > where it is NOT appropriate. The "increased, almost ubiquitous adaptation > of probabilistic methods" is the herd mentality in action (not to mention > the fact that it is directly orthogonal to work thirty years older). Most > of the time, most projects are using probabilistic methods to calculate a > tenth place decimal of a truth value when their data isn't even sufficient > for one. If you've got a heavy-duty discovery system, probabilistic > methods are ideal. If you're trying to derive probabilities from a small > number of English statements (like "this raven is white" and "most ravens > are black"), you're seriously on the wrong track. If you go on and on > about how humans don't understand Bayesian reasoning, you're both correct > and clueless in not recognizing that your very statement points out how > little Bayesian reasoning has to do with most general intelligence. Note, > however, that I *do* believe that probabilistic methods *are* going to be > critically important for activation for attention, etc. > > >> With regard to knowledge-conflict-resolution, Novamente's probabilistic > reasoning is designed to deal with it. Most of the other system I know of > that deal with knowledge-conflict-resolution, such as constraint > relaxation techniques, are probability based. > > This is where I believe that probabilistic reasoning is most often > improperly used though I don't believe that "most" constraint-relaxation > systems are probability-based (except, occasionally as an add-on to just > why a given constraint was relaxed rather than another). The problem with > probability-based conflict resolution is that it is a hack to get around > insufficient knowledge rather than an attempt to figure out how to get > more knowledge. It works because you always take the highest probability > choice -- except when the system tells you that the sauna is hot because > it doesn't know about the ice frozen over the top. In data-rich > constrained environments, probabilistic reasoning works (and neural > networks are very successful). In every day life . . . . it still works > because all your probabilities are near 100% . . . . except when they > suddenly aren't. > > >> So by "operating with small amounts of data" how small, very roughly, > are you talking about. And are you only talking about the active goals or > sources of activation, that will be small or are you saying that all the > computation in the system will only be dealing with a small amount of data > within, for example, one second of the processing of human-level system > operating at human-level speed? > > I mean like the way humans reason, there is only concentration on a > small number of objects -- which are only one link away from an almost > inconceivable number of related things -- and then the brain can jump at > least three of these links with lightning rapidity. Once again, the brain > is *massively* parallel and operates with a *huge* sparse matrix. > Activation is *far* more important than truth probabilities and much of > the focus is the other way (and activation is a really tough nut to solve > as you rightly point out with your comments about activation control). > Ask Ben how much actual work has been done on activation control in very > large, very sparse atom spaces in Novamente. He'll tell you that it's a > project for when he's further along. I'll insist (as will Richard) that > if it isn't baked in from the very beginning, you're probably going to > have to go back to the beginning to repair the lack. > > Mark > > P.S. Oh yeah -- if you were public enemy number one, I wouldn't bother > answering you (and I probably should lay off of the fan-boy crap :-). > > _____ > > This list is sponsored by AGIRI: http://www.agiri.org/email > To unsubscribe or change your options, please go to: > http://v2.listbox.com/member/? > <http://v2.listbox.com/member/?&; > > & > > ----- > This list is sponsored by AGIRI: http://www.agiri.org/email > To unsubscribe or change your options, please go to: > http://v2.listbox.com/member/?&; ----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=64923307-824e30
