On 2/12/2015 3:19 AM, Telmo Menezes wrote:
On Thu, Feb 12, 2015 at 10:25 AM, Bruno Marchal <[email protected]
<mailto:[email protected]>> wrote:
On 10 Feb 2015, at 22:26, Telmo Menezes wrote:
On Tue, Feb 10, 2015 at 9:07 PM, Jason Resch <[email protected]
<mailto:[email protected]>> wrote:
On Tue, Feb 10, 2015 at 12:59 PM, Telmo Menezes <[email protected]
<mailto:[email protected]>> wrote:
On Tue, Feb 10, 2015 at 6:21 PM, Jason Resch <[email protected]
<mailto:[email protected]>> wrote:
On Tue, Feb 10, 2015 at 12:04 PM, Telmo Menezes
<[email protected]
<mailto:[email protected]>> wrote:
On Tue, Feb 10, 2015 at 4:47 PM, Jason Resch
<[email protected]
<mailto:[email protected]>> wrote:
If you define increased intelligence as decreased
probability
of having a false belief on any randomly chosen
proposition,
then superintelligences will be wrong on almost
nothing, and
their beliefs will converge as their intelligence rises.
Therefore nearly all superintelligences will operate
according
to the same belief system. We should stop worrying
about trying
to ensure friendly AI, it will either be friendly or it
won't
according to what is right.
I wonder if this isn't prevented by Gödel's incompleteness.
Given
that the superintelligence can never be certain of its own
consistency, it must remain fundamentally agnostic. In this
case,
we might have different superintelligences working under
different
hypothesis, possibly occupying niches just like what
happens with
Darwinism.
Interesting point. Yes a true super intelligence may never
perform any
actions, as its trapped in never being certain (and knowing it
never
can be certain) that its actions are right. Fitness for
survival may
play some role in how intelligent active agents can be before
they
become inactive.
Yes, that's an interesting way to put it. I wonder.
I think chances are that it will be friendly, since I
happen to
believe in universal personhood, and if that belief is
correct,
then superintelligences will also come to believe it is
correct. And with the belief in universal personhood it
would
know that harm to others is harm to the self.
I agree with you, with the difference that I try to assume
universal personhood without believing in it, to avoid
becoming a
religious fundamentalist.
Interesting. Why do you think having beliefs can lead to
religious
fundamentalism. Would you not say you belief the Earth is
round? Could
such a belief lead to religious fundamentalism and if not why
not?
This leads us back to a recurring discussion on this mailing list.
I would
say that you can believe the Earth to be round in the informal
sense of the
word: your estimation of the probability that the earth is round is
very
close to one. I don't think you can believe the earth to be round
with 100%
certainty without falling into religious fundamentalism. This
implies a
total belief in your senses, for example. That is a strong position
about
the nature of reality that is not really backed up by anything.
Just like
believing literally in the Bible or the Quran or Atlas Shrugged.
I see. I did not mean it in the sense of absolute certitude, merely that
universal personhood is one of my current working hypotheses derived
from my
consideration of various problems of personal identity.
Right. We are in complete agreement then.
Universal personhood is also one of my main working hypotheses. I wonder if
it
could be considered a "preferable belief": it may be true and we are all
better off
assuming it to be true.
It might be useful after death, but I am not sure if it is a preferable
belief/assumption on the terrestrial (effective) plane. It makes sense only
through
a personal understanding, for example of the universal person that all
machine can
recognized by themselves to be when introspecting, in case they are enough
self-referentially correct. If not, it will becomes a statement that the
parrots
will repeat and impose without understanding, and that will quickly lead to
a threat
to freedom.
If you are honest about your belief in universal personhood you won't be interested into
manipulating the other versions of you into servitude.
This reminds me of Nietzschean slave morality: the slave cannot conceive of true
freedom, so he can only desire to become the oppressor. But this is because he does not
really believe in universal personhood, otherwise he would understand true freedom.
And he won't mind being a slave because he knows he's also a slave holder. At least
that's what I'd tell him if I were a slave holder.
It's essentially the story told by fascists: Sure you're a lowly cog in the great
machinery of the nation, BUT you're part of a great and glorious nation. And also by
ISIL: Sure you're fighting and suffering and you're gonna get killed, BUT your part of a
glorious new Caliphate.
Brent
--
You received this message because you are subscribed to the Google Groups
"Everything List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/everything-list.
For more options, visit https://groups.google.com/d/optout.
