----- Original Message -----
From: "Charles D Hixson" <[EMAIL PROTECTED]>
Sent: Thursday, June 08, 2006 7:26 PM
Subject: Re: [agi] Four axioms (Was Two draft
papers: AI and existential risk; heuristics and biases)
> I think that Axiom 2 needs a bit of
work.
Agreed.
> as I read it, it appeared to be a
recommendation to "wish early and often".
It can certainly be taken that way. I don't
see anything at all wrong with "wish early and often". Remember, however,
that Axiom 2 is explicitly NOT meant to recognize multiple copies of the same
wish. Wish early and often but don't try to game the system by repeating
the same wish with different wording or other minor differences -- and, as
I said before, Axiom 2 should certainly not be read as a support for a simple
majority rule either.
> N.B.: I disagree with the axioms as
stated (even given the interpretation), because I *don't* think that the wishes
of each individual are all of equal value. The wish of a guy with a broken
leg to have medical attention appears to me as more important than the wish of
some four-year old for a bag of candy, e.g.)
Excellent! A lead in to my next major chunk
of concepts . . . . Here's the teaser -- If your goal is
maximizing volition fulfillment then there is clearly a
substantial difference in the importance of
*fulfilling a volition* and *maintaining the integrity of a source of
volition*. The guy with a broken leg is a source of volition. If he
dies, there are that many volitions that are never created to be
fulfilled. While I still am not supporting simple majority rule, it should
be quite clear that all those possibly annihilated volitions easily
outweigh any single, ten, hundred, or more volitions that do not
affect the integrity of a source of volition.
> OTOH, if this is seen as an engineering
simplification of something that might be too complex to properly accomplish
otherwise, it may be justifiable.
While I might make engineering simplifications as
necessary, I will attempt to clearly identify them as such. Personally,
however, treating as equal the cases of a guy with a broken leg and a
four-year-old wanting candy would definitely be the signs of a
really bad design.
> But I do wonder
about corner cases and how you define "individual" and "separate" especially in
combination with "volition" and "wish". Do wishes count if they aren't
expressed? What if they are expressed, but censored?
For morality purposes, wishes certainly do count
even if not expressed. For the purposes of an AI, the AI should ask if
possible and assume common and reasonable wishes if it's not possible to ask and
if it has no evidence to the contrary.
If they are expressed but willingly censored by the
individual then the censorship is considered their volition. If they are
expressed and censored by someone else then they should be simply consired as
expressed.
> There's no
two/three-year old who hasn't occasionally wished his parents were "dead".
(I'm not clear what's meant by that, but it's not nice. It's probably also
not equivalent to what the word normally means.)
There are multiple reasons why this isn't a
problem. First, see the argument about volitions vs. sources of volition
-- the parents are in no danger at all. Second, still not supporting
simple majority rule, even if this was a not threat to a source
of volition, it is clearly contrary to an overwhelming number of other
volitions. Third, and here comes another set of teasers, since no one
is obligated to take an affirmative action to fulfill another
individual's volition, the two/three-year-old would
have to find someone willing to make his parents dead. And, in the case of
an AI or any other friendly entity, that
entity should always be fulfilling society's goals by not only looking
for informed consent by saying "Here are the consequences . . . . Are you
REALLY sure that you want to do this?" but also by generally declining to take
an action that it knows to be a bad idea (from the point of view of
society/maximizing volition). Even in the case of a two/three-year-old
wanting candy, the AI should most often conclude that candy is a bad idea and
decline to give it to the child.
> Here we have the problem. Exactly *WHAT*
does axiom 2 mean?
Axiom 2 means that wishes from different
individuals carry exactly the same weight unless part of the wish is that it
doesn't (i.e. you can prioritize someone else's wish above your own). It
also (apparently poorly) attempts to rule out the consideration of multiple
identical wishes from the same individual or from a group of identical
people.
> (I also don't even accept that all wishes
from the same individual are of equal significance or importance.)
Nor do I. I assume that individuals can
certainly prioritize their own wishes. (I definitely have the fix the
phrasing on Axiom 2)
> And you
don't express how the AI is to know that the wish exists
No, I don't. For the time being, I'm going to
assume that the AI is either going to ask or be told (or, in extreme
circumstances where is isn't possible, assume normal and common wishes unless it
has evidence to the contrary).
>> Axiom 3 is simply an attempt to clear away all the clutter . . .
> Be careful here. The purpose is sound, but it implies that you have gotten everything right the first time. I don't immediately see what's wrong (other that as I've said), but this explicitly says that there will be no way to patch the system.
It's an interesting question as to whether expanding it to something
like "There is no other inherent good other than
what I tell you in the future when in sound mind and body and not coerced by
others" is as bad an idea as I think it is. Honestly, if I vet this
through enough people, I believe that the odds of it being wrong are virtually
non-existant and certainly less than the odds of something horrible
happening if I put such a clause in. If you were an super-intelligent AI,
wouldn't you have a problem with a clause like this?
> Except, of course, be a 50% + 1 tally of wishes....actually, not even
that's needed if there are a bunch of entities who don't have a known wish
against the system being patched in a certain way.
Nope. Neither of those are even close to a valid criteria for
getting even an opposed volition actualized (much less a change to the entire
goal/morality system). As I've said, simple majority rule doesn't cut it
in this system. And I would never suggest a system where it is possible
for such a patch to be effected without everyone being asked
effectively and given a reasonable chance to respond.
>> Axiom 4 (Colloquial/Selfish
Version).
>> or, more formally . . . .
> That does need to be expressed as an axiom. It does seem to be implied
> by logic from the first three...but only if instead of calling these
> assertions axioms you call them goals. I also would not state it in a
> form that implied that it's existence as a goal depended on it's ability
> to be derived as a result of the other goals.
>> or, more formally . . . .
> That does need to be expressed as an axiom. It does seem to be implied
> by logic from the first three...but only if instead of calling these
> assertions axioms you call them goals. I also would not state it in a
> form that implied that it's existence as a goal depended on it's ability
> to be derived as a result of the other goals.
I'm not sure what you're saying. Could you provide a specific
rephrasing?
> Should it be impossible to modify the major goals? If not, then it
> should certainly be quite difficult, but if you can't then major
> problems that are discovered later can't be fixed.
If by major goals, you mean the first three axioms, then I would argue
extremely strenuously that the first two can't be changed and reasonably
strenuously that the third can't be changed.
> I'd say "Have a good trip!", but seventh graders? Come
back sane.
:-) Actually, I'm pretty resilient to the attempted
tyranies of that age group (mostly finding them humorous). It was actually
fun (or is that proof of my insanity? :-).
Thanks.
Mark
To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
