Stefan Pernar wrote:
On 10/26/07, *Richard Loosemore* <[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>> wrote:
Stefan can correct me if I am wrong here, but I think that both yourself
and Aleksei have misunderstood the sense in which he is pointing to a
circularity.
If you build an AGI, and it sets out to discover the convergent desires
(the CEV) of all humanity, it will be doing this because it has the goal
of using this CEV as the basis for the "friendly" motivations that will
henceforth guide it.
But WHY would it be collecting the CEV of humanity in the first
phase of
the operation? What would motivate it to do such a thing? What exactly
is it in the AGI's design that makes it feel compelled to be friendly
enough toward humanity that it would set out to assess the CEV of
humanity?
The answer is: its initial feelings of friendliness toward humanity
would have to be the motivation that drove it to find out the CEV.
The goal state of its motivation system is assumed in the initial state
of its motivation system. Hence: circular.
Interesting point and I guess asking if a programmers CEV would be to
let an AGI find the CEV of humanity is another aspect of finding
circularity in the concept.
Yup.
What really matters is finding out how to dispose the AGI to have
friendliness from the outset, so that it can then seek the specific
needs of humanity.
The way out of the circularity is to understand how to build
motivational systems in an AGI, and how to give global feelings of
(e.g.) empathy to the AGI.
As I have said before, I have an approach to that problem. I also
believe that the standard AI understanding of motivation, in which goals
are represented in the same semantic format as the rest of the system's
explicit knowledge, are such a bad way to drive an AGI that it will (a)
not actually work if you want the system to be generally intelligent,
and (b) would in any case be a disastrous way to ensure the motivations
of an AGI.
Richard Loosemore
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?member_id=4007604&id_secret=57841333-afba0f
