On Tue, Oct 02, 2007 at 01:20:54PM -0400, Richard Loosemore wrote:
> When the first AGI is built, its first actions will be to make sure that 
> nobody is trying to build a dangerous, unfriendly AGI.  

Yes, OK, granted, self-preservation is a reasonable character trait.

> After that 
> point, the first friendliness of the first one will determine the 
> subsequent motivations of the entire population, because they will 
> monitor each other.

Um, why, exactly, are you assuming that the first one will be freindly?
The desire for self-preservation, by e.g. rooting out and exterminating
all (potentially unfreindly) competing AGI, would not be what I'd call
"freindly" behavior.

There's also a strong sense that winnner-takes-all, or
first-one-takes-all, as the first one is strongly motivated,
by instinct for self-preservation, to make sure that no other
AGI comes to exist that could threaten, dominate or terminate it.

In fact, the one single winner, out of sheer loneliness and boredom,
might be reduced to running simulations a la Nick Bostrom's simulation
argument (!)


This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:

Reply via email to