You misunderstood me -- when I said robustness of the goal system, I meant
the contents and integrity of the goal system, not the particular
implementation.
I do however continue to object to your phrasing about the system
recognizing influence on it's goal system and preserving it. Fundamentally,
there are only a very small number of "Thou shalt not" supergoals that need
to be forever invariant. Other than those, the system should be able to
change it's goals as much as it likes (chocolate or strawberry? excellent
food or mediocre sex? save starving children, save adults dying of disease,
or go on vacation since I'm so damn tired from all my other good works?)
A quick question for Richard and others -- Should adults be allowed to
drink, do drugs, wirehead themselves to death?
----- Original Message -----
From: "Vladimir Nesov" <[EMAIL PROTECTED]>
To: <agi@v2.listbox.com>
Sent: Tuesday, October 02, 2007 9:49 AM
Subject: **SPAM** Re: [agi] Religion-free technical content
But yet robustness of goal system itself is less important than
intelligence that allows system to recognize influence on its goal
system and preserve it. Intelligence also allows more robust
interpretation of goal system. Which is why the way particular goal
system is implemented is not very important. Problems lie in rough
formulation of what goal system should be (document in English is
probably going to be enough) and in placing the system under
sufficient influence of its goal system (so that intelligent processes
independent on it would not take over).
On 10/2/07, Mark Waser <[EMAIL PROTECTED]> wrote:
The intelligence and goal system should be robust enough that a single or
small number of sources should not be able to alter the AGI's goals;
however, it will not do this by recognizing "forged communications" but
by
realizing that the aberrant goals are not in congruence with the world.
Note that many stupid and/or greedy people will try to influence the
system
and it will need to be immune to them (or the solution will be worse than
the problem).
--
Vladimir Nesov mailto:[EMAIL PROTECTED]
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?&;
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?member_id=8660244&id_secret=48878028-0baa9c
