C. David Noziglia
Object Sciences Corporation
6359 Walker Lane, Alexandria, VA
(703) 253-1095
"What is true and what is not? Only God knows. And, maybe, America."
Dr. Khaled M. Batarfi, Special to Arab
News
"Just because something is obvious doesn't mean it's true."
--- Esmirelda Weatherwax, witch of Lancre
----- Original Message -----
From: "Philip Sutton" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Monday, November 04, 2002 8:02 PM
Subject: [agi] RE: Ethical drift
> Ben Goertzel wrote:
> > What if iterative self-revision causes the system's goal G to "drift"
> > over time...
>
> I think this is inevitable - it's just evolution keeping on going as it
always
> will. The key issue then is what processes can be set in train to operate
> throughout time to keep evolution re-inventing/re-committing AGIs (and
> humans too) to ethical behaviour. Maybe communities of AGIs can
> create this dynamic.
>
> Can isolated, non-socialised AGIs be ethical in relation to the whole?
>
> A book that I found facinating on the ethics issue in ealier evolutionaryu
> stages is:
>
> Good Natured: The Origins of Right and Wrong in Humans and Other
> Animals
> by Frans De Waal, Frans de Waal (Paperback - October 1997)
> Harvard Univ Pr; ISBN: 0674356616; Reprint edition (October 1997)
>
> It's well worth a read.
>
> Cheers, Philip
>
I would dare to add a note of perhaps naive optomism here.
It is a common belief that game theory has shown that it is advantageous to
be selfish and nasty. I assume that the members of this group know that is
wrong, that game theory has in fact shown that in a situation of repeated
interaction, it is more advantageous from a strictly self-interested
viewpoint to make nice and cooperate. This is a simplistic description of
the Nash Equilibrium.
Of course, Smith's Evolutionarily Stable Sets then show that there are
situations when betrayal then becomes of greater advantage to an individual,
so we can't count on a Nash calculation to lead any and all AGI's to make
nice and keep their human companions comfortable. I think that Jack
Williamson's The Humanoids
(http://www.amazon.com/exec/obidos/ASIN/0312852533/ref%3Dnosim/music2u/104-7
816878-7538303) is still the best and most thoughful cautionary tale in this
line.
David
-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/
