Edward W. Porter wrote:
In response to Richard Loosemore’s Post of Sun 11/4/2007 12:15 PM
responding to my prior message of Sat 11/3/2007 3:28 PM
ED’s prior msg>>>>> For example, humans might for short sighted personal
gain (such as when using them in weapon systems)
RL>>>> Whoaa! You assume that it would be possible to "use" an AGI for
personal gain, or in a weapon system. If it starts out with the
supposed empathic motivational system, it would not allow this.
ED>>>> First, there is good reason to believe that military and/or
selfish use of computers will continue into the future and that a lot of
AGI’s will be made with motivational systems other that the one’s you
put so much trust in.
If I may interrupt here: this assumes that things carry on much as they
are, but with AGI as well.
That is an extremely unlikely scenario, once you examined the
consequences of having even one safe AGI in existence. As I have said
before, as soon as you have one, you will want to get it to recursively
self-improve until it is superintelligent, and then it will immediately
look around, see *all* of the dangers that you foresee below, and
quietly, nonviolently deal with them. At that point there will not be
any more unfriendly AGIs around.
I have summarized, in the above paragraph, a rather large body of
thought, so please accept my apologies for everything left out... But
even if you do not see the validity of this argument as it stands,
please do accept the fact that this is a very real possibility, and that
for that reason nobody can come to further conclusions (in particular
the ones you list below) until the viability of that scenario has been
carefully examined. Everything (but *everything*) depends on the
whether or not the scenario I just described is the one that will prevail.
For a variety of reasons, I believe it is overwhelmingly likely, but
that is a long story.
This is supported by: (A) a lot of funding of AI is being done, not only
by America’s defense department, but also by other defense departments,
including in countries like South Korea and Israel, and probably also in
many other countries; and (B) there is a tremendous amount of use of
computers for selfish reasons, such as for stealing information, sending
spam emails, stealing the use of other people’s computers, and
performing extortions by threatening denial of service attacks and
hacking into financial transaction systems,
Second, even even if early AGI’s are given your motivational system, it
would only seem to make sense that human would want the ability to
modify such systems. Even if the human’s making such system’s had good
motivations they might not know exactly how to best create such a
motivational system and they would almost certain want the ability to
improve it, or override it if it was misbehaving. So, until we trust
AGIs so much that we put their motivational systems beyond our control,
they will be designed so that humans could modify such systems, and that
would mean humans culd modify them as I have suggested above.
Too much depends on exactly how the motivational systems are structured.
I do not believe, in practice, that people will *want* to modify them
in any significant way. Such modifications as do occur will be done by
global consultation: the initial design will be such that *only* global
consultation (amongs the AGIs and the humans) would allow modifications
to be made. That is part of the security mechanism that makes it safe:
all the AGIs will be watching one another, and any attempt to make
unauthorized changes will be instantly spotted.
Third, you have not indicated how AGI’s with your motivational systems
could insure they were not hacked. A superintelligence under its own
malicious motivations or the control of a malicious human could code,
and understand code and its flaws a millions of times better than
humans, so it is going to be hard to keep your “white-hat” machines from
being hacked by “black-hat” machines. About ten years ago I talked with
an MIT profession who was one of DARPA’s head guy on computer security
and he told me it was a problem much bigger than any solution he knew of
other than keeping a machine totally disconnected. That would apply to
AGI’s
Forth, and most importantly, you have not shown how your system would
deal with all the problems I have raised in prior posts on this subject
about, possible conflicting goals, conflicts between application of the
same goal to different parts of the world, the need for a system dealing
in a complex and rapidly changing world(as the near- and
post-singularity world will be) to change its sub-goals and its
understanding of how to best serve its motivations in such a rapidly
changing world. You have only provided verbal handwaving. Nothing that
even begins to answer such hard and real questions. The below
interchanges provides some examples of such handwaving.
Did you read my previous posts on Motivational Systems That Are Stable?
Unfortunately, I could only summarize the position briefly in the post
you are quoting here, so I think that is why some of the arguments look
like handwaving.
ED’s prior msg>>>>> Or over time the inherent biases that were designed
to make AGI’s have empathy for humans, might cause it to have empathy
for some humans more than others
RL>>>> As part of its initial (assumed) empathy, it will set up
mechanisms to monitor such things. It could not possibly start having
"more empathy for some humans more than others" without *also* being
aware of the fact that, by being so biassed, it was in conflict with its
general motivation. So it would not do such a thing. (We have to
remember not to assume it would be both superintelligent, and also
suscetible to such easily-caught problems).
ED>>>> Assuming for the moment your hypothetical system has stayed loyal
to people, I understand how in the situation described above it probably
would know it had a conflict.
But if the computer treats all people equally, then how good would it be
if it aids all people equally regardless of the relative good and/or
harm they are doing. And if it aids people in doing harm, is it or is
it not being empathetic to people, and who is to decide what is good and
what is harm.
Sorry, I have to interrupt here, because you are in the midst of a
particular type of (how can I put it?) fallacy (?). The "fallacy" is to
first assume the AGI would be superintelligent (which is the assumption
that the discussion is based on), and then quietly slip in a scenario in
which the AGI does something incredibly, bizarrely stupid: namely, this
superintelligent AGI, with its enormous, carefully balanced empathy for
the human species, suddenly decides that because it is trying to "be
balanced" in the way it empathizes with people it must give a homicidal
maniac an equal chance to fulfil his dreams of genocide!
That kind of "dumb computer" AI is exactly the type of science fiction
mistake that we must get away from, surely? We cannot assume it is
smart, and then say "But what it ..." and then insert a mind-bogglingly
dumb behavior.
So, in this case, "empathy for the human race" means that the human race
says that genocidal dictators are not acceptible. The AGI is perfectly
well aware of this, so it just quietly tells the hypthetical Hitler
that, awfully sorry but no genocide today thankyou.
Would it help a new Hitler, or would it fight him, or
would it equally aid both sides, or would it stay out of the struggle
completely. Would it help fight Al Quida? Would it help American in
Iraq? Would it help company A try to develop a product that will help
out compete company B? Is that empathetic to mankind? Who defines what
is empathetic to mankind.
And if they will not help companies, nations, or ideologies compete, how
likely is it that your “empathic” AGIs will be the one that governments
and corporations will pay to have built, when other, more nationally,
corporately, or personally loyal and useful AGI can be built?
ED’s prior msg>>>>> , or might cause them to make decisions that they
think are in our best interest, but would not.
RL>>>> Again, this assumes (implicitly) that it would both be generally and
broadly empathic -- which means sensitive to our wishes -- and at the
same time, for some inexplicable reason, decide to do something that it
thinks is in our best interests, without consulting us. Effectively
assumed that it would spontaneously *stop* being empathic, without
explaining how it could happen.
ED>>>> You talk about being “broadly empathic” to humans as if that
answer all questions. It doesn’t. In fact, it avoids almost all of the
most relevant ones.
There is no broad consensus among humans what life is about, what its
main purposes should be, what the social contract should be, what are
our duties to each other, etc. So the question become which of the
beliefs and value systems that are disputed among people would it allow
itself to be used to support, and if it refused to be used for any
purposes about which there is dispute among humans, what good would it be.
Would machines that are empathetic to people addict all of humanity to
the better-that-crystal-meth-eternal-rush machine Jiri Jelinek has been
pushing in the Nirvana? Manyana? Never! Thread, or would it not. If
they shared Jiri’s values they would. If they shared mine they would
not. If they shared mine they would actively discourage people from
using such a machine, assuming it was shown to have the totally
consuming addictive power Jiri assumes they would have.
No they would not: this is an outrageous distortion of what "broadly
empathic" means. Does it really take a brain the size of a planet to
realize that "... addicting all of humanity... " to anything is
obviously not an act of empathy for the species as a whole? You
postulate this crazy scenario of it suddenly deciding that it "should"
take the action of addicting everyone to something, against their will,
when it is obvious that most people would consider this the very
opposite of empathy (the first rule of empathy is to let people make
their own decisions, after all!).
If it is obvious to most people that this would not be empathic, why do
you insert the out-of-left-field assertion that the AGI might do this?
Again, you implictly assume that, for some bizarre reason, the AGI would
be both superintelligent and so dumb as not to be able to understand
word one about what empathy actually is.
So it would appear that your concept of “generally and broadly empathic”
is such a broad generality as to be totally useless for most human
endeavors in our competitive world.
Sorry: completely false conclusion.
ED’s prior msg>>>>> The world is too complicated and is going to change
too rapidly in the next one hundred, one thousand, or ten thousand years
for any goal system designed circa 2015 to remain appropriate until the
end of history – unless history ends pretty soon.
RL>>>> Not true: the statement was that it would stay empathic to our
motivations.
Only if the goal system were particularly rigid would this be a problem,
and by assumption I am talking about motivational systems that are
stable (diffuse systems, along the lines of my previously mentioned post).
ED>>>> Your approach to motivational system might be more stable than
others, but it is not clear that it would reliably deal with issues of
the types I have discussed above (people, themselves, often don’t) and
it is not clear it would remainloyal to the best interests of human (if
it were possible to know exactly what that entailed) to the end of
history, as you claim. Again you have refused to answer the central
thrust of my question.
The previous post of mine was long and detailed: did you look it up,
read it and fully understand it before you assembled these criticisms?
I must say that it does not appear so.
Richard Loosemore
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?member_id=8660244&id_secret=61212037-5ae4a0
