Le 29-déc.-06, à 10:39, Stathis Papaioannou a écrit :

You seem to be including in your definition of the UM the *motivation*, not just the ability, to explore all mathematical objects. But you could also program the machine to do anything else you wanted, such as self-destruct when it solved a particular theorem. You could interview it and it might explain, "Yeah, so when I prove Fermat's Last Theorem, I'm going to blow my brains out. It'll be fun!" Unlike naturally evolved intelligences, which could be expected to have a desire for self-preservation, reproduction, etc., an AI can have any motivation and any capacity for emotion the technology allows. The difference between a machine that doesn't mind being a slave, a machine that wants to be free, and a machine that wants to enslave everyone else might be just a few lines of code.

You are right. I should have been clearer. I was still thinking to machine having been programmed with some "universal goal" like "help yourself", and actually I was refrerring to those who "succeed" in helping themselves. Surely the machine which blows itself in case of success (like some humans do BTW) does not belong the long run winner.

I tend to define a successful AI, as a machine which does succeed in the sharing of our evolutionary histories. What I was saying in that a (lucky!) universal machine driven by a universal goal will develop a taste for freedom. My point is that such a taste for freedom is not necessarily "human". I would be astonished if extraterrestrials does not develop such a taste. The roots of that attraction is the fact that when machine develop themselves (in some self-referentially correct way) they are more and more aware of their ignorance gap (which grows along with that development). By filling it, it grows more, but this provides the roots of the motivations too.

But then we are perhaps ok. "Help yourself" is indeed some line of code.



You received this message because you are subscribed to the Google Groups 
"Everything List" group.
To post to this group, send email to everything-list@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 

Reply via email to