On 10 Sep 2016, at 19:14, John Clark wrote:
On Sat, Sep 10, 2016 at 9:06 AM, Telmo Menezes
<[email protected]> wrote:
> published this working paper on arxiv, same title as this
email: http://arxiv.org/abs/1609.02009
Criticisms most welcome!
This may have some bearing on what you're saying, more than
twenty years ago on May 13 1996 I sent this letter to
Nick Bostrom commenting on some of his ideas:
I think that's the central flaw in your argument. You're making an
assumption that can not be true. We don't understand
ourselves, a Jupiter Brain wouldn't understand itself either. It may
be able to produce internal models that are far more complex than
anything we can do, but the thing it is trying to model,
itself, is also far more complex. A Jupiter Brain could make
improvements in itself, but it would have to be very careful,
and could certainly not "transform itself" to any mental state
"at will" because it wouldn't know how.
Science and understanding are about prediction, and the only way a
computer program can know how it will react if it is given a
particular input is to run it, and even that will not always work.
Turing proved in 1935 that a computer program or a Turing Machine
can not predict its own behavior, and unless you believe in a
mystical soul, that's exactly what we or a Jupiter brain are, a
computer program running on a Turing Machine.
Today some human beings become drug addicts and accomplish nothing,
in the future some Jupiter Brains may decide to directly, and
crudely, stimulate the pleasure centers of it's brain with the
equivalent of a drug like Heroin and accomplish nothing. That will
be a problem, but I don't see why it will be a bigger problem than
it is now, and not everyone is a junkie. Your assumption is that the
Jupiter Brain will have access to something infinitely more powerful
than Heroin that will be impossible to resist, but that is not true.
I agree with you that if it was possible to change your emotions to
anything you wanted, alter modes of thought, radically change your
personality, swap your goals as well as your philosophy of life at
the drop of a hat it would be very dangerous. Once you change
yourself you may not want to change back, even if your behavior
became bizarre or suicidal.
Ever want to accomplish something but been unable to because it's
difficult, well just change your goal in life to something simple
and do that; better yet, flood your mind with a feeling of pride and
self satisfaction and don't bother accomplishing anything at all.
Think all this is a terrible idea and stupid as well, no problem,
just change your mind (and I do mean CHANGE YOUR MIND)
now you think it's a wonderful idea. I don't have the blueprints for
a Jupiter brain in my pocket but I do know that complex mechanisms
don't do well in a positive feedback loop, not electronics, not
animals, not people and not Jupiter brains.
There is another reason, that may strike closer to home, to be happy
that total self awareness is impossible, if it weren't AI
researchers would be out of a job. If we understood how our mind
worked nobody would study computer science in school because it
would be too trivial, we'd all know instinctively how to make an AI,
but that's not how the world works. For the things the mind does
best, creativity, language recognition, depth perception, pattern
identification, muscle coordination, we have no self awareness at
all. It's only in things that were lousy at, like solving
differential equations, that we become aware of our mental activity.
We only become aware of mental processes when we're confused.
People always asked creative men like Feynman how he came up with
such brilliant ideas, but he didn't know, all he could say was
things like "the idea just popped into my head". If he could really
answer that question then we could just follow his instructions
and we'd all be as smart as he was. Self awareness means knowing
what goes on inside your mind, but when you try to think about your
present mental state you immediately change it, and if you don't
completely understand something then you can't change it to anything
you want at will.
I mainly agree. The arithmetical reality is inexhaustible (Torkel
Franzen terming), and infinitely complex, and we can't escape it, no
drugs can do that.
The use of psychotropic, in good environment, can only make doubt that
some reality are ultimate, but that is how science and religion (well
understood) always progress (by doubting ideas previously taken for
granted). Once we have artificial brains, our notion of identity can
evolve a lot, and the pleasure will remain in sharing experiences,
including amnesia.
Addiction is not a fatality. Contemplation is not productive, sure,
but productivity is not a goal per se (today I would say it is a
perverted tool).
Bruno
John K Clark
Criticisms most welcome!
Best,
Telmo.
--
You received this message because you are subscribed to the Google
Groups "Everything List" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/everything-list.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "Everything List" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/everything-list.
For more options, visit https://groups.google.com/d/optout.
http://iridia.ulb.ac.be/~marchal/
--
You received this message because you are subscribed to the Google Groups
"Everything List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/everything-list.
For more options, visit https://groups.google.com/d/optout.
