Hi, > The reason I asked the question was not to ask whether AIXI is > pragmatically better as a design strategy than Novamente. What I was > asking you rather is if, looking at AIXI, you see something > *missing* that > would be present in Novamente. In other words, *if* you had an > infinitely > powerful computer processor, is there a reason why you would *not* > implement AIXI on it, and would instead prefer Novamente, even if it had > to run on a plain old cluster?
These are deep and worthwhile questions that I can't answer thoroughly off the cuff, I'll have to put some thought into them and reply a little later. There are other less fascinating but more urgent things in the queue tonight, alas ;-p My intuitive feeling is that I'd rather implement Novamente but with AIXI plugged in as the "schema/predicate learning" component. In other words, it's clear that an infinitely capable procedure learning routine would be very valuable for AGI. But I don't really like AIXI's overall control structure, and I need to think a bit about why. ONE reason is that it's insanely inefficient, but even if you remove consideration of efficiency, there may be other problems with it too. > Actually, given the context of Friendliness, what we're interested in is > not so much "intelligence" as "interaction with humans"; under this view, > for example, giving humans a superintelligently deduced cancer cure is > just one way of interacting with humans. Looking at AIXI and Novamente, > do you see any way that Novamente interacts with humans in a way > that AIXI cannot? Well, off the cuff, I'm not sure because I've thought about Novamente a lot more than I've thought about AIXI. I'll need to mull this over.... It's certainly worth thinking about. Novamente is fundamentally self-modifying (NOT the current codebase but the long-term design). Based on feedback from humans and its own self- organization, it can completely revise its own codebase. AIXI can't do that. Along with self-modification comes the ability to modify its reward/punishment receptors, and interpret what formerly would have been a reward as a punishment... [This won't happen often but is in principle a possibility] I don't know if this behavior is in AIXI's repertoire... is it? > > Also, not all of Novamente's spontaneous behaviors are even implicitly > > goal-directed. Novamente is a goal-oriented but not 100% goal-directed > > system, which is one major difference from AIXI and AIXItl. > > I agree that it is a major difference; does it mean that Novamente can > interact with humans in useful or morally relevant ways of which AIXI is > incapable? Maybe... hmmm. > > In that case you cannot prove any of Hutter's > > theorems about them. And if you can't prove theorems about > them then they > > are nothing more than useless abstractions. Since AIXI can never be > > implemented and AIXItl is so inefficient it could never do > anything useful > > in practice. > > But they are very useful tools for talking about fundamental kinds of > intelligence. I am not sure whether they are or not. > > Well, sure ... it's *roughly analogous*, in the sense that it's > experiential > > reinforcement learning, sure. > > Is it roughly analogous, but not really analogous, in the sense that > Novamente can do something AIXI can't? Well, Novamente will not follow the expectimax algorithm. So it will display behaviors that AIXI will never display. I'm having trouble, off the cuff and in a hurry, thinking about AIXI in the context of a human saying to it "In my view, you should adjust your goal system for this reason...." If a human says this to Novamente, it may consider the request and may do so. It may do so if this human has been right about a lot of things in the past, for example. If a human says this to AIXI, how does AIXI react and why? AIXI doesn't have a "goal system" in the same sense that Novamente does. AIXI if it's smart enough co uld hypothetically figure out what the human meant and use this to modify its current operating program (but not its basic program-search mechanism, because AIXI is not self-modifying in such a strong sense)... if its history told it that listening to humans causes it to get rewarded. But, it seems to me intuitively that the modification AIXI would make in this case, would not constrain or direct AIXI's future development as strongly as the modification Novamente would make in response to the same human request. I'm not 100% sure about this though, because my mental model of AIXI's dynamics is not that good, and I haven't tried to do the math corresponding to this scenario. What do you think about AIXI's response to this scenario, Eliezer? You seem to have your head more fully wrapped around AIXI than I do, at the moment ;-) I really should reread the paper, but I don't have time right now. This little scenario I've just raised does NOT exhaust the potentially important differences between Novamente and AIXI, it's just one thing that happened to occur to me... I'll think on this stuff more... -- Ben G ------- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/?[EMAIL PROTECTED]
