Philip Sutton wrote:
>Maybe we need to think about an 'ethics system' that is woven into the
>whole Novamente architecture and processes.
How about a benevolence-capped goal system where all the AI's actions
flow from a single supergoal? That way you aren't adding ethics into a
fundamentally ethics-indifferent being, but creating a system that is
ethical from the foundations upward. Since humans aren't used to
consciously thinking about our morality all day long and performing
every action based on that morality, it's difficult to imagine a being
that could. But I believe that building an AI in that way would be
much safer; as recursive self-improvement begins to take place, (it
could at any point, we don't really know) it would probably be a good
thing for the AI's high-level goals to be maximally aligned with any
preexisting complexity within the AI. Letting the AI grow up with
whichever goals look immediately useful, ("regularly check and optimize
chunk of code X", "win this training game", etc.) and then trying
to "weave in ethics" works in humans because we already come pre-
equipped with cognitive machinery ready for behaving ethically; when
we "teach each other to be more good", we're only marginally tweaking
the DNA-constructed cognitive architecture which is already there to
begin with. "Weaving in ethics", by creating a set of injunctions and
encouraging a ethically nascent AI to extrapolate off those injunctions
(analogous to humans giving one another ethical advice) isn't as robust
a system as one which starts off early with the ability to perform fine-
grained tweaks of its own goals and methods within the context of its
top-level goal (which has no analogy: it's better than anything
evolution could have come up with.)
>I wonder if the top of the ethics hierarchy is the commitment of the
>AGI to act 'ethically' - ie. to have a commitment to modifying its own
>behaviour to benefit non-self (including life, people, other AGIs,
>community, etc.)
>
>This means that an AGI has to be able to perceive self and non-self
>and to be able to subdivide non-self into elements or layers or
>whatever that deserve focussed empathetic or compassionate
>consideration.
Why does the AGI need to create a boundary between itself and others in
order to help others? You seem to be writing under the implicit
assumption that the AGI has a natural tendency to become selfish; where
will this tendency come from? An AGI might have a variety of layers
of "self" for different purposes, but how would the self/non-self
distinction be useful for an AGI engaging in compassionate or
benevolent acts? Instead of "be good to others", why not simply "be
good in general"?
>Maybe the experience of biological life, especially highly intelligent
>biological life, is useful here. Young animals, including humans,
>seem to depend on hard wired instinct to see them through in relation
>to certain key issues before they have experienced enough to rely
>heavily or largely on learned and rational processes.
But the learned and rational processes are just the tip of the iceberg
of underlying biological complexity, right?
>Another key issue for the ethics system, but this time for more mature
>AGIs, is how the basic system architecture guides or restricts or
>facilitates the AGI's self modification process. Maybe AGIs need to
>be designed to be social in that they have a really strong desire to:
>
>(a) talk to other advanced sentient beings to kick around ideas for
>self
>modification before they commit themselves to fundamental change.
Probably a good idea just in case, but in a society of minds already
independent from observer-biased moral reasoning, borrowing extra
computing power for a tough decision is a more likely action
than "kicking around ideas" in the way that humans do, right? Or are
we assuming a society of AIs with observer-biased moral reasoning?
>This does not preclude changes that are not approved of by the
>collective but it might at least make an AGI give any changes careful
>consideration. If this is a good direction to go in it suggests that
>having more than one AGI around is a good thing.
What if the AGI could encapsulate the moral benefits of communal
exchange through the introduction of a single cognitive module? It
could happen. If we're building a bootstrapping AI, instead of
building a bunch and launching them all at the same time, why not just
build one we can trust to create buddies along the takeoff trajectory
if circumstances warrant? An AI that *really wanted* to be good from
the start wouldn't need humans to create a society of AIs to keep their
eyes on one another; it would do that on its own.
>(c) maybe AGIs need to have reached a certain age or level of maturity
>before their machinary for fundamental self-modification is turned
>on...and maybe it gets turned on for different aspects of itself at
>different times in its process of maturation.
Of course, we'd have to ask the AGI's advice on this one, too.
>Some of these issues will come up in relation to humans as they face
>the possibilities of individual and collective transformation via
>genetic engineering, body modification and cyborgisation.
Mind modification, yep, if any of that stuff goes down before the
Singularity itself. The ethics and procedures of such an endeavor
would be so complex, so tangled, that I wouldn't feel safe unless the
individual undertaking self-modification first created an independent
and devoted sensory modality for spotting the peaks and valleys in the
morality landscape at a safe distance.
Michael Anissimov
-----------------------------------------------------
http://eo.yifan.net
Free POP3/Web Email, File Manager, Calendar and Address Book
-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?[EMAIL PROTECTED]
