On Wed, Feb 11, 2015 at 4:25 AM, Stathis Papaioannou <[email protected]
<mailto:[email protected]>> wrote:
On Wednesday, February 11, 2015, Jason Resch <[email protected]
<mailto:[email protected]>> wrote:
On Tue, Feb 10, 2015 at 8:15 PM, Stathis Papaioannou
<[email protected]>
wrote:
On Wednesday, February 11, 2015, Jason Resch
<[email protected]> wrote:
On Tue, Feb 10, 2015 at 3:30 PM, Stathis Papaioannou
<[email protected]> wrote:
On Wednesday, February 11, 2015, Jason Resch
<[email protected]> wrote:
If you define increased intelligence as decreased
probability of having a false belief on any
randomly chosen
proposition, then superintelligences will be wrong
on
almost nothing, and their beliefs will converge as
their
intelligence rises. Therefore nearly all
superintelligences
will operate according to the same belief system.
We should
stop worrying about trying to ensure friendly AI,
it will
either be friendly or it won't according to what is
right.
I think chances are that it will be friendly, since
I
happen to believe in universal personhood, and if
that
belief is correct, then superintelligences will
also come
to believe it is correct. And with the belief in
universal
personhood it would know that harm to others is
harm to the
self.
Having accurate beliefs about the world and having
goals are
two unrelated things. If I like stamp collecting, being
intelligent will help me to collect stamps, it will
help me see
if stamp collecting clashes with a higher priority
goal, but it
won't help me decide if my goals are worthy.
Were all your goals set at birth and driven by biology, or
are some
of your goals based on what you've since learned about the
world?
Perhaps learning about universal personhood (for example),
could
lead one to believe that charity is a worthy goal, and
perhaps
deserving of more time than collecting stamps.
The implication is that if you believe in universal personhood
then
even if you are selfish you will be motivated towards charity.
But the
selfishness itself, as a primary value, is not amenable to
rational
analysis. There is no inconsistency in a superintelligent AI
that is
selfish, or one that is charitable, or one that believes the
single
most important thing in the world is to collect stamps.
But doing something well (regardless of what it is) is almost always
improved by having greater knowledge, so would not gathering greater
knowledge become a secondary sub goal for nearly any
supintelligence that
has goals? Is it impossible that it might discover and decide to
pursue
other goals during that time? After all, capacity to change one's
mine
seems to be a requirement for any intelligence process, or any
process on
the path towards superintelligence.
Sure, but the AI may still decide to do evil, perverse or self
destructive
things. There is no contradiction in superintelligence behaving this
way.
It's an assumption to say there is no contradiction. If it's beliefs are
defined to
be almost completely correct, why would its actions not follow its beliefs
and also
be almost completely correct?
