Most people are about as happy as they make up their minds to be.
-- Abraham Lincoln
In our society, after a certain point where we've taken care of our
immediate needs, arguably we humans are and should be subject to the Nirvana
effect.
Deciding that you can settle for something (if your subconscious truly can
handle it) definitely makes you more happy than not.
If, like a machine, you had complete control over your subconscious/utility
functions, you *could* Nirvana yourself by happily accepting anything.
This is why pleasure and lack of pain suck as goals. They are not goals,
they are status indicators. If you accept them as goals, nirvana is clearly
the fastest, cleanest, and most effective way to fulfill them.
Why is this surprising or anything to debate about?
----- Original Message -----
From: "J Storrs Hall, PhD" <[EMAIL PROTECTED]>
To: <agi@v2.listbox.com>
Sent: Friday, June 13, 2008 11:58 AM
Subject: Re: [agi] Nirvana
There've been enough responses to this that I will reply in generalities,
and
hope I cover everything important...
When I described Nirvana attractractors as a problem for AGI, I meant that
in
the sense that they form a substantial challenge for the designer (as do
many
other features/capabilities of AGI!), not that it was an insoluble
problem.
The hierarchical fixed utility function is probably pretty good -- not
only
does it match humans (a la Maslow) but Asimov's Three Laws. And it can be
more subtle than it originally appears:
Consider a 3-Laws robot that refuses to cut a human with a knife because
that
would harm her. It would be unable to become a surgeon, for example. But
the
First Law has a clause, "or through inaction allow a human to come to
harm,"
which means that the robot cannot obey by doing nothing -- it must weigh
the
consequences of all its possible courses of action.
Now note that it hasn't changed its utility function -- it always believed
that, say, appendicitis is worse than an incision -- but what can happen
is
that its world model gets better and it *looks like* it's changed its
utility
function because it now knows that operations can cure appendicitis.
Now it seems reasonable that this is a lot of what happens with people,
too.
And you can get a lot of mileage out of expressing the utility function in
very abstract terms, e.g. "life-threatening disease" so that no utility
function update is necessary when you learn about a new disease.
The problem is that the more abstract you make the concepts, the more the
process of learning an ontology looks like ... revising your utility
function! Enlightenment, after all, is a Good Thing, so anything that
leads
to it, nirvana for example, must be good as well.
So I'm going to broaden my thesis and say that the nirvana attractors lie
in
the path of *any* AI with unbounded learning ability that creates new
abstractions on top of the things it already knows.
How to avoid them? I think one very useful technique is to start with the
kind
of knowledge and introspection capability to let the AI know when it faces
one, and recognize that any apparent utility therein is fallacious.
Of course, none of this matters till we have systems that are capable of
unbounded self-improvement and abstraction-forming, anyway.
Josh
-------------------------------------------
agi
Archives: http://www.listbox.com/member/archive/303/=now
RSS Feed: http://www.listbox.com/member/archive/rss/303/
Modify Your Subscription:
http://www.listbox.com/member/?&;
Powered by Listbox: http://www.listbox.com
-------------------------------------------
agi
Archives: http://www.listbox.com/member/archive/303/=now
RSS Feed: http://www.listbox.com/member/archive/rss/303/
Modify Your Subscription:
http://www.listbox.com/member/?member_id=8660244&id_secret=103754539-40ed26
Powered by Listbox: http://www.listbox.com
