Hi,
I'll
respond to this on the SL4 list, which is better suited for discussion of the
more futurist rather than AI-architecture-oriented aspects of my
essay
ben
g
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]On Behalf Of Philip Sutton
Sent: Thursday, February 19, 2004 4:30 PM
To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
Subject: Re: Positive Transcension 2Ben,I've just finished reading your 14 February version of "Encouraging a Positive Transcension".It's taken me two reads of the paper to become clear on a few issues.It seems to me that there are really three separate ethical issues at the heart of the paper that have been conflated and they are: how can we ensure that the next big advance in cognitive capacity in our neck of the universe-- is not a disaster for existing sentient beings (humans being theonly ones we know of presently),- doesn't fail to carry forward the gains made so far by existingcreative sentient beings (including humans) and- helps to drive (and does not prevent) further wondrous flowering ofthe universe.While these issues clearly interrelate (will protecting existing sentient beings lead to a stagnation in the flowering of the universe?) I think there is something to be gained from being clear about each one.And there is a special aspect to the first issue that shouldn't be overlooked. The emergence of AGI is not some inevitable process that Fate deals up to us. On the earth at least, it is the outcome of deliberate actions by a few humans that could impact on the rest of humanity (and perhaps a lot of the rest of the universe as well). So while we discuss the ethics we want to see AGIs apply, we also need to also think about the ethics of what we ourselves are doing. If we can't get our own ethics sorted out then I'm not too hopeful we'll be able to generate appropriate and adequate ethics in our AGI progeny.So let's start with how some humans might feel about some other humans creating a 'thing' which could wipe out humans without their agreement.Ben you said: "And this may or may not lead to the demise of humanity - which may or may not be a terrible thing." At best loose language like this means one thing to most people - somebody else is being cavalier about their future - at worst they are likely to perceive an active threat to their existence.Frankly I doubt if anyone will care if humanity evolves or transcends to a higher state of being so long as it's voluntary. To a timeless observer it might be arguable that the humanity of 2004 (or whatever) is no longer to be found - but the people who have evolved/transcended will still feel like humanity of the new era - they will not have been obliterated. To mix this sort of change up with the death of humanity via, for example, rather un-necessary discussions of Nietzsche's notions of "a good death" and "Man is something to be overcome" seems to me to be pointless and dangerous. After the "bad death" of many thousands of people in the Twin Towers the US has rained death on many more thousands of people in the rest of the world. For AGI- advocates to be cavalier about the lives of billions of people is to my mind to, very understandably, invite similar very nasty reactions.To withhold concern for other humans lives because theoretically some AGI might form the view that our mass/energy could be deployed more beautifully/usefully seems simply silly. The universe is a big place with, most likely, a mind bogglingly large amount of mass/energy not used by any sentient beings - so having a few billion humans on the Earth or the nearby planets is hardly going to cramp the style of any self- respecting AGI with a big brain.I think the first step in creating safe AGI is for the would-be creators of AGI to themselves make an ethical commitment to the protection of humans - not because humans are the peak of creation or all that stunningly special from the perspective of the universe as a whole but simply because they exist and they deserve respect - especially from their fellow humans. If AGI developers cannot give their fellow humans that commitment or that level of respect, then I think they demonstrate they are not safe parents for growing AGIs! I was actually rather disturbed by your statement towards the end of your paper where you said: "In spite of my own affection for Voluntary Joyous Growth, however, I have strong inclinations toward both the Joyous Growth Guided Voluntarism and pure Joyous Growth variants as well." My reading of this is that you would be prepared to inflict Joyous Growth future on people whether they wanted it or not and even if this resulted in the involuntary elimination of people or other sentients that somehow were seen by the AGI or AGIs pursuing Joyous Growth as being an impediment in the way of the achievement of joyous growth. If I've interpreted what you are saying correctly that's pretty scary stuff!I think the next step is to consider what values we would like AGIs to hold in order for them to be sound citizens in a community of sentients. I think the minimum that is needed is for them to have a tolerant, respectful, compassionate, live-and-let-live attitude. This is what I personally would hope for from all sentients - no matter how low or mighty their intellectual powers. This doesn't mean that all human behaviours or all AGI behaviours should be accepted. Cruel or exploitative or oppressive behaviours by any sentient or group of sentients would seem to me to be behaviours that should be resisted or prevented.I think AGIs that had a tolerant, respectful, compassionate, live-and-let- live ethic would not intrude excessively on human society. They might, for example, try to discourage female circumcision or even go so far as stopping capital punishment in human societies (I can't see that these actions would conform to the ethics that the AGIs were given [under my scenario] their human creators/carers). As far as I can see I don't think that AGIs need to have ported into them a sort of general digest of human-ness or even an idiosyncratic (renormalised) essence of general humane-ness. I think we should be able to be more transparent than that and to identify the key ethical drivers that lead to tolerant, respectful, compassionate, live-and-let-live behaviour.I think these notions are sufficiently abstract to be able to pass your test of being likely to "survive successive self-modification". They are not tied to a form of humanity that is frozen in time and they not tied conceptually to any particular form of life or sentience. And I think this base ethic would be useful in guiding how AGIs relate to each other.If AGIs adopted a tolerant, respectful, compassionate, live-and-let-live ethic then I think that we would have pretty good assurance that the emergence of AGIs was not going to be a disaster for any existing sentient beings (including humans beings) and that the gains made so far by existing creative sentient beings would not be lost due to the cavalier (or otherwise) annihilation of sentient societies by more powerful AGIs.Now I want to move on to the issue how ethical systems might ensure that AGIs help to drive (and do not prevent) further wondrous flowering of the universe.Ben, you proposed that AGIs should have an ethic of promoting voluntary, joyous growth. The way you discussed this issue it made it sound as if all AGIs should have this goal/ethical structure. It's not clear to me that all AGIs need such a goal structure for there to be a wondrous flowering of the universe. The development of art/science etc. that we love so much on the earth was the work of one species in 20 million. Perhaps only a small minority of AGIs need to be creative or promoting "voluntary joyous growth" for there to be the unfolding that you are hoping for.My guess is that if we avoid human or AGI dictatorship, then humans of all sorts will facilitate the creation of all sorts of AGIs. So long as these AGIs all practice a tolerant, respectful, compassionate, live-and-let-live ethic and so long as *some* AGIs pursue 'voluntary joyous growth' or 'growth in knowledge and development and application of creativity' then I think a positive transcension will occur. I think we should look to a plurality of AGI ethics as much or more than we should expect and support a plurality of human goals and ethics. If people or AGIs are happy with relatively unchanging lives (as we perceive it), then good on them if that's what makes them happy or transcendent. It only takes a small percentage of 'driven creatives' (whether human or AGI) to keep evolution moving along. In a healthy mix of sentients it's probably also a good idea if at least a reasonable percentage (5-10%???) are driven by an urge to improve wellbeing for themselves and others - ie. not driven pricipally by growth in knowledge/patterns or hedonistic joy! This nead for a complementary mix of motivations within a human/AGI/other sentient meta population is another reason for having a plurality of AGIs rather than just one. (I know that mindplexes could be formed by groups of AGIs - but I still think that even a mindplex will be better quality/wiser if we encourage the creation of many, diverse AGIs with distinct perspectives.)In your paper you suggest that we need AGIs to save humanity from our destructive urges (applied via advanced technology). If having AGIs around could increase the risk of humanity being wiped out to achieve a more beautiful deployment of mass/energy then it might be a good idea to go back and check to see just exactly how dangerous the other feared technologies are. While nanotech and genetic engineering could produce some pretty virulent and deadly entities I'm not sure that they are likely to be much more destructive than bubonic plague, eboloa virus, small pox have been in their time etc. There are a lot of people around so that even if these threats killed millions? billions? they are unlikely to wipe out even most people. So should we seek help from this scale of threat by creating something that might arbitarily decide to wipe out the lot of us on a whim?I don't think that AGIs are inevitably a threat, but this will only be if AGIs are imbued with tolerant, respectful, compassionate, live-and-let- live ethics and I think these will only be imbued if their human makers are similarly moved by the same ethic.-------By the way, your interpretation of my idea of 'sustainability' as a form of notalgia misses the point I was trying to make. I don't think one has to be a Luddite or a back-to-the land/cave person to find some value in the concept of sustainability. I think life for any creative sentient is made up of three processes:- changing things for the better driven by need or delight- engaging in a journey of life where change occurs but the changecannot be characterised as better or worse than what went before- retaining conditions/things that are valued because they are needed(from a utilitarian point of view) or they are valuedexistentially/morally etc.These three processes combine a concern for both change and continuity. At any particular moment not *everything* is changed nor is *everything* retained (sustained).The notion of combining continuity and change is particularly important if you think that the purpose of change includes creating better situations/things. If things can be improved then it is sensible to protect these improvements from back-sliding when yet further changes are made.Nostalgia plays a part in this for some people but generally I suspect a fairly small part in this whole process of managing/fostering continuity and change.----Oh well, I hope what I've said is of some use!Cheers, Philip
To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
