On 01/07/07, Tom McCabe <[EMAIL PROTECTED]> wrote:
> If its goal is "achieve x using whatever means > necessary" and x is > "win at chess using only the formal rules of chess", > then it would > fail if it won by using some means extraneous to the > formal rules of > chess, just as surely as it would fail due to losing > to a superior > opponent. Uh... it kinda doesn't matter if a human would judge it a "failure" or "cheating" if the relevant human is dead. Why should the AGI care about human judgment?
If its goal is x, then it will do what it can to achieve x, regardless of human judgement or anything else. But that's exactly why it is wrong to assume that the AI would modify x in order to fit in with what you would do in a similar situation. What is it about "win at chess using only the formal rules of chess" that would allow the AI to modify it to "win at chess using any means necessary" rather than, say, "chess is a dumb game, I think collecting seashells is a more worthwhile goal"? -- Stathis Papaioannou ----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=4007604&user_secret=7d7fb4d8
