rg wrote:
Hi
I made some responses below.
Richard Loosemore wrote:
rg wrote:
Hi
Is anyone discussing what to do in the future when we
have made AGIs? I thought that was part of why
the singularity institute was made ?
Note, that I am not saying we should not make them!
Because someone will regardless of what we decide.
I am asking for what should do to prepare for it!
and also how we should affect the creation of AGIs?
Here's some questions, I hope I am not the first to come up with.
* Will they be sane?
* Will they just be smart enough to pretend to be sane?
until...they do not have to anymore.
* Should we let them decide for us ?
If not should we/can we restrict them ?
* Can they feel any empathy for us ?
If not, again should we try to manipulate/force them to
act like they do?
* Our society is very dependent on computer systems
everywhere and its increasing!!!
Should we let the AGIs have access to the internet ?
If not is it even possible to restrict an AGI that can think super fast
is a super genious and also has a lot of raw computer power?
That most likely can find many solutions to get internet access...
(( I can give many crazy examples on how if anyone doubts))
* What should we "stupid" organics do to prepare ?
Reduce our dependency?
* Should a scientist, that do not have true ethical values be allowed
to do AGI research ?
Someone that just pretend to be ethical, someone that just wants the
glory and the
Nobel price....someone that answers the statement: It is insane
With: Oh its just needs
some adjustment, don't worry :)
* What is the military doing ? Should we raise public awareness to
gain insight?
I guess all can imagine why this is important..
The only answers I have found to what can truly control/restrict an
AGI smarter than us
are few..
- Another AGI
- Total isolation
So anyone thinking about this?
Hi
You should know that there are many people who indeed are deeply
concerned about these questions, but opinions differ greatly over what
the dangers are and how to deal with them.
This sounds good :)
I have been thinking about these questions for at least the last 20
years, and I am also an AGI developer and cognitive psychologist. My
own opinion is based on a great deal of analysis of the motivations of
AI systems in general, and AGI systems in particular.
I have two conclusions to offer you.
1) Almost all of the discussion of this issue is based on assumptions
about how an AI would behave, and the depressing truth is that most of
those assuptions are outrageously foolish. I say this, not to be
antagonistic, but because the degree of nonsense talked on this
subject is quite breathtaking, and I feel at a loss to express just
how ridiculous the situation has become.
It is not just that people make wrong assumptions, it is that people
make wrong assumptions very, very loudly: declaring these wrog
assumptions to be "obviously true". Nobody does this out of personal
ignorance, it is just that our culture is saturated with crazy ideas
on the subject.
This is probably true.
Therefore I try to make very few assumptions, except one: They will
eventually be much smarter than us.
(If you want I can justify this, based on scalability.)
Your comments are interesting, because they give me some opportunities
to illustrate the extreme difficulty of analysing these questions
without making hidden assumptions.
To begin with your above remark: it is fair to assume that they will be
much smarter than us, but the consequences of this are not as obvious as
they might appear.
For example: what if the inevitable outcome were that "they" would give
us the option of elevating our intelligence up to their level, at will
(albeit with the proviso that when going up to their level we would
leave the dangerous human motivations on ice for that time)? Under
these circumstances there would not be any meaningful "them" and "us"
but actually one population of beings, some of whom would be
superintelligent some of the time, but with a flexibility in the level
of intelligence of any given individual that is completely impossible today.
Second, we have to consider not their intelligence level as such, but
their motivations. More on this in a moment.
2) I believe it is entirely possible to build a completely safe AGI.
I also beelieve that this completely safe AGI would be the simplest
one to build, so it is likley to be built first. Lastly, I believe
that it will not matter a great deal who builds the first AGI (within
limits) because an AGI will "self-stabilize" toward a benevolent state.
Why is it simplest to make a safe AGI?
A long argument, the shortest version of which is: you have to give a
motivation system of some sort (NOT a conventional goal stack, which
does not work for full AGI systems) and the motivation system will have
a set of drives .... if you try to make it violent or aggressive, this
will tend to destabilize it, meaning that some other team which is
building a simpler AGI with neutral motivations will be able to get
their system to work first. But even they have to make the AGI want to
do something, and I submit that they will give it simple desire to want
to be empathic to the human race, because this will facilitate the
learning process. After all, we know a good deal about how to do the
teacher-student thing when the teacher and student are in harmony with
one another (e.g. patient adult and cheerful, affectionate, cooperative
child), but we find it very difficult to imagine teaching something that
does not relate to us in any way whatsoever (ever tried to teach a wild
cat how to do algebra?).
This is a complex subject, so my point in a short post like this is to
show you that:
a) Some people (i.e. me) have thought some of these issues through in
great depth,
b) The factors that determine the outcome are, at the very least, not
quite as obvious as they seem, and
c) There is some prima facie argument to back up the idea that the
simplest type of AGI might (suprisingly, perhaps) also be the
friendliest and therefore safest.
Is it not more difficult to make something that is guaranteed to be in
some way?
This is true of a "technology" that is not able to think for itself at a
high level of intelligence. But the crux of my argument is that this is
not really "technology" we are talking about here, so the assumption
that it is always *intrinsically* more difficult to make something
guaranteed safe does not necessarily apply.
Two factors contribute to the safety feature. One is that because the
AGI can think, it will be able to understand its own safety. No mere
machine could ever do that (which is why machines are not safe). The
second (and perhaps more important) is that the design of the motivation
system can be carried out in such a way that it is "stable" in the sense
that a system constrained by a very large number of constraints is
stable. For example: the sun is stable with respect to its physical
location because, to go anywhere, rather a lot of atoms have to
simultaneously decide to move in the same direction.... its position is
constrained by the fact that an impossibly large number of 'accidents'
would need to occur for it to go anywhere spontaneously.
Is it not easier to just make something that can potentially be safe,
unsafe and whatever.
Only for ordinary 'technology'. The usual arguments, as I say, do not
apply for a 'technology' that thinks.
When you say it will "self-stabilize" toward a benevolent state do you
not make a large assumption.
It will exist in the same world as humans, do we all stabilize into
benevolent states?
This is the most interesting of your questions. We are humans. We have
motivations systems designed by evolution to be aggressive, selfish, etc
etc. When you generalize from our behavior to the behavior of an AGI,
you implicitly assume that the AGI would have those same motivations.
This is, at the very least, a questionable assumption. In fact I have
argued that it is much more than just questionable, it is very likely to
be 100% wrong (see above).
The self-stabilization idea has many components to it (some outlined
above). One that I have not mentioned so far is an intriguing effect
that occurs in humans with severe, but episodic mental disorders: in a
nutshell, if your brain periodically makes you insane, then during the
periods when you are not suffering an insane episode you "want" to do
something to not be the insane person any more. If you had a switch to
get rid of the pathoogical 'you', you would use it. So if someone made
a pathologically violent AGI, that AGI would have to have some episodes
during which it was calm, otherwise it would never develop into an
intelligent (let alone superintelligent) entity. But during the quiet
episodes it would know enough to modify itself to eliminate the
pathological side.
This is one important aspect of what I mean by self-stabilization.
Unless you introduce this artificially during the evolutionary process
of said AGI.
Rewarding certain behavior. But what happens when the AGI realizes it
has been
designed in an evolutionary process with this goals in mind, what will
it do?
We can not know can we ?
Suppose you knew that evolution designed you in such a way that you have
a predesigned module in your motivation system that makes you feel
affection for your parents? Suppose also that you are not afflicated by
violent emotions or motivations.
Would your knowledge of the *fact* of this innate desiugn feature make
you suddenly feel inclined to rip it out and replace it with one that
gave you the opposite motivation?
I submit that, if that is what drives you, you will feel no such
inclination. An AGI would be the same way. It could understand
everything about our intentions and actions in designing it, but it
would *be* the peaceful, empathic creature that we designed, so it would
feel no inclination to abandon that motivation system just because we
put it there.
Richard Loosemore
-------------------------------------------
agi
Archives: http://www.listbox.com/member/archive/303/=now
RSS Feed: http://www.listbox.com/member/archive/rss/303/
Modify Your Subscription:
http://www.listbox.com/member/?member_id=8660244&id_secret=95818715-a78a9b
Powered by Listbox: http://www.listbox.com
