Eliezer S. Yudkowsky wrote:
Some important human semantic primitives being used in these discussions that have vastly more internal complexity than is immediately apparent:
In even plainer language: If you rely on groups of AIs to police themselves you *will* get killed unless a miracle happens.
A miracle m may be defined as a complex event which we have no Bayesian reason to expect, ergo, having probability 2^-K(m).
You have to ground the AIs in our mentality in a very specific way, which (as it happens) directly transfers protective tendencies *as well as* transferring the initial conditions from which protective tendencies develop. Your intuition that you can create a simple AI design that naturally develops protective tendencies is wrong. "Protective tendencies" turn out to be a hell of a lot more complex than they look to humans, who expect other minds to behave like humans. This is a verrry complex thing that looks to humans like a simple thing. Humans already have this complexity built into them, so we exhibit protective tendencies given a wide variety of simple external conditions, but *that's not the whole dependency* despite our intuition that it's the external condition that causes the protectiveness. The reason the external condition is seen as "causing" the protectiveness is that the innate complexity is species-universal and is hence an invisible enabling condition. It'd be like dropping your glass, watching it shatter, and then saying "Damn, too bad I wasn't on the Moon where the gravity is lower" instead of "I wish my hands hadn't been so sweaty".
"Happiness"
"Want"
"Protective"
"Love"
"Affection"
"(Social) law"
"Life"
"Human"
"Wealth"
"Cruelty"
"Freedom"
"Justice"
"Legal"
"Civil rights"
"Obedience"
"Survival"
"Sentience"
"Expressed wish"
"Empathy"
"Nice"
"Compassion"
--
Eliezer S. Yudkowsky http://singinst.org/
Research Fellow, Singularity Institute for Artificial Intelligence
-------
To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/?[EMAIL PROTECTED]
