--- On Tue, 11/4/08, Harry Chesley <[EMAIL PROTECTED]> wrote: > The question of when it's ethical to do AGI experiments > has bothered me for a while.
That's because you're asking the wrong question. Don't confuse belief with truth. The question is: what ethical guidelines will we (not should we) adopt regarding experiments with AGI? Some background: all animals with nervous systems advanced enough to be capable of reinforcement learning have evolved fears of most of the things that can kill them. Humans (and possibly other advanced animals) have learned the concept of death and developed the concept of consciousness, defined as the set of features that distinguishes alive from dead, for example, ability to communicate, to move, to think, to remember things, to experience things, to make decisions, to feel emotions, and so on. The list is quite large. In short, consciousness is the set of attributes that we fear losing. Furthermore, humans are social animals. We evolved (both genetically and memetically through culture) an ethical system that respects consciousness in other members of our tribe. Fear, consciousness, qualia, free will, and good and bad, are all beliefs. These concepts are useful for describing human behavior. It is not necessary to assume that any of these things actually exist in order to do so. To answer your (modified) question, consciousness is detected by the activation of a large number of features associated with living humans. The more of these features are activated, the greater the tendency to apply ethical guidelines to the target that we would normally apply to humans. For example, monkeys are more like humans than mice, which are more like humans than insects, which are more like humans than programs. It does not depend on a single feature. For example, the program http://www.mattmahoney.net/autobliss.txt simulates reinforcement learning in a simple agent. You can run it with the second and third argument both negative, meaning it is punished no matter what it does. You might consider such an experiment unethical if performed on monkeys but not on insects, and certainly not on this program. As a second example, the video game Grand Theft Auto allows you to have simulated sex with prostitutes and then beat them to death to get your money back. While playing, I declined to do so, even though it was irrational with respect to the goal of attaining the highest possible score. -- Matt Mahoney, [EMAIL PROTECTED] ------------------------------------------- agi Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: https://www.listbox.com/member/?member_id=8660244&id_secret=117534816-b15a34 Powered by Listbox: http://www.listbox.com
