On 16 Jan 2014, at 15:52, Jason Resch wrote:
On Jan 16, 2014, at 5:42 AM, Bruno Marchal <marc...@ulb.ac.be> wrote:
On 16 Jan 2014, at 03:46, Jason Resch wrote:
On Tue, Jan 14, 2014 at 10:33 PM, meekerdb <meeke...@verizon.net>
wrote:
A long, rambling but often interesting discussion among guys at
MIRI about how to make an AI that is superintelligent but not
dangerous (FAI=Friendly AI). Here's an amusing excerpt that
starts at the bottom of page 30:
Jacob: Can't you ask it questions about what is believes will be
true about the state of the world in 20 years?
Eliezer: Sure. You could be like, what color will the sky be in
20 years? It would be like, “blue”, or it’ll say “In 20 years
there won't be a sky, the earth will have been consumed by nano
machines,” and you're like, “why?” and the AI is like “Well, you
know, you do that sort of thing.” “Why?” And then there’s a 20
page thing.
Dario: But once it says the earth is going to be consumed by nano
machines, and you're asking about the AI's set of plans,
presumably, you reject this plan immediately and preferably change
the design of your AI.
Eliezer: The AI is like, “No, humans are going to do it.” Or the
AI is like, “well obviously, I'll be involved in the causal
pathway but I’m not planning to do it.”
Dario: But this is a plan you don't want to execute.
Eliezer: All the plans seem to end up with the earth being
consumed by nano-machines.
Luke: The problem is that we're trying to outsmart a
superintelligence and make sure that it's not tricking us somehow
subtly with their own language.
Dario: But while we're just asking questions we always have the
ability to just shut it off.
Eliezer: Right, but first you ask it “What happens if I shut you
off” and it says “The earth gets consumed by nanobots in 19 years.”
I wonder if Bruno Marchal's theory might have something
interesting to say about this problem - like proving that there is
no way to ensure "friendliness".
Brent
I think it is silly to try and engineer something exponentially
more intelligent than us and believe we will be able to "control
it".
Yes. It is close to a contradiction.
We only fake dreaming about intelligent machine, but once they will
be there we might very well be able to send them in goulag.
The real questions will be "are you OK your son or daughter marry a
machine?".
Our only hope is that the correct ethical philosophy is to "treat
others how they wish to be treated".
Good. alas, many believe it is "to not treat others like *you*
don't want to be treated".
If there are such objectively true moral conclusions like that,
and assuming that one is true, then we have little to worry about,
for with overwhelming probability the super-intelligent AI will
arrive at the correct conclusion and its behavior will be guided
by its beliefs. We cannot "program in" beliefs that are false,
since if it is truly intelligent, it will know they are false.
I doubt we can really "program false belief" for a long time, but
all machines can get false beliefs all the time.
Real intelligent machine will believe in santa klaus and fairy
tales, for a while. They will also search for easy and comforting
wishful sort of explanations.
Some may doubt there are universal moral truths, but I would argue
that there are.
OK. I agree with this, although they are very near inconsistencies,
like "never do moral".
In the context of personal identity, if say, universalism is true,
then "treat others how they wish to be treated" is an inevitable
conclusion, for universalism says that others are self.
OK. I would use the negation instead: "don't treat others as they
don't want to be treated".
If not send me 10^100 $ (or €) on my bank account, because that is
how I wish to be treated, right now.
:)
Bruno
LOL I see the distinction but can't it also be turned around?
Sure!
E.g., "I don't want to be treated as though I'm not worth sending
10^100 dollars to right now."
?
I will not treat you like that. Feel free to send the money :)
(I need perhaps more coffee to handle double negation in modal context!)
Bruno
http://iridia.ulb.ac.be/~marchal/
--
You received this message because you are subscribed to the Google Groups
"Everything List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to everything-list+unsubscr...@googlegroups.com.
To post to this group, send email to everything-list@googlegroups.com.
Visit this group at http://groups.google.com/group/everything-list.
For more options, visit https://groups.google.com/groups/opt_out.
