Mark Waser wrote: > .. > The first thing that is necessary is to define your goals. It is > my contention that there is no good and no bad (or evil) except in the > context of a goal and that those who believe that there is some > absolute morality out there have been fooled by the unconscious > assumption of the most common human goals. Thus, the first three axioms: > > > Axiom 1. Volition actualization/wish fulfillment is good > > Axiom 2. Each separate/individual volition/wish is of equal value > > Axiom 3. There is */no/* other inherent good > I think that Axiom 2 needs a bit of work. From what you say later on, it appears that you mean something quite reasonable, but as I read it, it appeared to be a recommendation to "wish early and often".
N.B.: I disagree with the axioms as stated (even given the interpretation), because I *don't* think that the wishes of each individual are all of equal value. The wish of a guy with a broken leg to have medical attention appears to me as more important than the wish of some four-year old for a bag of candy, e.g. OTOH, if this is seen as an engineering simplification of something that might be too complex to properly accomplish otherwise, it may be justifiable. It does remind me of the Athenian ostracism, where someone could be exiled for 20 years just because he was unpopular, so perhaps it needs to be thought through with a bit of care. Still, it's a place to start which is simple, intelligible, and probably implementable. But I do wonder about corner cases and how you define "individual" and "separate" especially in combination with "volition" and "wish". Do wishes count if they aren't expressed? What if they are expressed, but censored? There's no two/three-year old who hasn't occasionally wished his parents were "dead". (I'm not clear what's meant by that, but it's not nice. It's probably also not equivalent to what the word normally means.) > > Axiom 1 is the primary axiom. Few people would dispute the core of > it but everyone wants to raise questions about and exceptions to it -- > the most common of which is "What happens when two volitions > collide?" Note that when the word volition is used, it refers only to > current volition (and explicitly not to future or "extrapolated" > volitions). For the moment, don't worry about edge-cases like teenage > suicide, etc. that look like they may be incorrectly solved because of > axiom 1. They ARE correctly solved from extrapolation from all four > axioms (and, if not, I'm sure that everyone will let me know :-) > I have no trouble with axiom 1. > > Axiom 2 is some critical clarification that needs a lot of filling > out. It also may be objectionable to many people. If you don't agree > with axiom 2, you're not going to agree with my solution because it > critically depends upon it. Axiom 2 does explicitly mean that it is > NOT good (or moral or whatever) for a Jupiter brain to forcibly > "uplift" an entity that doesn't want to be uplifted. Also, Axiom 2 is > meant to explicitly NOT recognize multiple copies of the exact, same > entity as having separate/individual volitions/wishes so it is not > possible for an AI to play number games by making numerous identical > copies of itself (although, as it relinquishes control over those > copies and they grow apart as they have different experiences, this > changes). Axiom 2 should certainly not be read as a support for a > simple majority rule either. > Here we have the problem. Exactly *WHAT* does axiom 2 mean? As stated I have to disagree with every interpretation I've come up with, but your own equivocations imply that you have some of the same concerns that I have. (I also don't even accept that all wishes from the same individual are of equal significance or importance.) And you don't express how the AI is to know that the wish exists > > Axiom 3 is simply an attempt to clear away all the clutter of > previous beliefs. Religions have many good ideas. Religions have > many bad ideas. I don't want to deal with it and clutter makes for > bad (error-prone) design. > Be careful here. The purpose is sound, but it implies that you have gotten everything right the first time. I don't immediately see what's wrong (other that as I've said), but this explicitly says that there will be no way to patch the system. Except, of course, be a 50% + 1 tally of wishes....actually, not even that's needed if there are a bunch of entities who don't have a known wish against the system being patched in a certain way. > > - - - - - - - - - - - - > > The next thing that we need to do is to define how we're going to > */start/* fulfilling our goals . . . . the seed, as it were . . . . > and thus: > > > Axiom 4 (Colloquial/Selfish Version). The best way to ensure the > optimization of my good is to ensure the optimization of > */everybody/*'s good. > > > > or, more formally . . . . > That does need to be expressed as an axiom. It does seem to be implied by logic from the first three...but only if instead of calling these assertions axioms you call them goals. I also would not state it in a form that implied that it's existence as a goal depended on it's ability to be derived as a result of the other goals. Should it be impossible to modify the major goals? If not, then it should certainly be quite difficult, but if you can't then major problems that are discovered later can't be fixed. > > ... > > Have at it . . . . > > > > Mark > > > > P.S. I am away from my e-mail for the next day or so as a chaperone > on a seventh-grade trip of epic proportions. Please do not interpret > my silence during that time as implying anything other than that . . . > . :-) > I'd say "Have a good trip!", but seventh graders? Come back sane. ------- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
