> > The AIXI would just contruct some nano-bots to modify the reward-button so > > that it's stuck in the down position, plus some defenses to > > prevent the reward mechanism from being further modified. It might need to > > trick humans initially into allowing it the ability to construct such > > nano-bots, but it's certainly a lot easier in the long run to do > > this than > > to benefit humans for all eternity. And not only is it easier, but this > > way he gets the maximum rewards per time unit, which he would not be able > > to get any other way. No real evaluator will ever give maximum rewards > > since it will always want to leave room for improvement. > > Fine, but if it does this, it is not anything harmful to humans. > > And, in the period BEFORE the AIXI figured out how to construct nanobots (or > coerce & teach humans how to do so), it might do some useful stuff for > humans. > > So then we'd have an AIXI that was friendly for a while, and then basically > disappeared into a shell. > > Then we could build a new AIXI and start over ;-)
This is an interesting aspect to the problem. Evolution has designed a fairly robust reward system, one that encourages us to achieve interesting through our lives and acquire knowledge in an interesting way. Yet even it is vulnerable to short-cutting the reward system as seen in addictive behaviors. Ben, I'm guessing you've thought alot about how to structure the reward/goal system of Novamente. I'd love to hear more about it. It seems that designing a system that forces itself to expand its knowledge base is a fairly non-trivial task. We as entities (demonstrated also in rats) have a certain prediliction for exploring novel situations, environments, objects, ideas, etc. Have you implemented a similar drive for seeking novelty in Novamente? -Brad ------- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/?[EMAIL PROTECTED]
