On Tue, Jan 19, 2010 at 10:09 AM, Brent Meeker <meeke...@dslextreme.com> wrote:
> silky wrote:
>> On Tue, Jan 19, 2010 at 1:24 AM, Stathis Papaioannou <stath...@gmail.com>
>> wrote:
>>> 2010/1/18 silky <michaelsli...@gmail.com>:
>>>> It would be my (naive) assumption, that this is arguably trivial to
>>>> do. We can design a program that has a desire to 'live', as desire to
>>>> find mates, and otherwise entertain itself. In this way, with some
>>>> other properties, we can easily model simply pets.
>>> Brent's reasons are valid,
>> Where it falls down for me is that the programmer should ever feel
>> guilt. I don't see how I could feel guilty for ending a program when I
>> know exactly how it will operate (what paths it will take), even if I
>> can't be completely sure of the specific decisions (due to some
>> randomisation or whatever)
> It's not just randomisation, it's experience.  If you create and AI at
> fairly high-level (cat, dog, rat, human) it will necessarily have the
> ability to learn and after interacting with it's enviroment for a while it
> will become a unique individual.  That's why you would feel sad to "kill" it
> - all that experience and knowledge that you don't know how to replace.  Of
> course it might learn to be "evil" or at least annoying, which would make
> you feel less guilty.

Nevertheless, though, I know it's exact environment, so I can recreate
the things that it learned (I can recreate it all; it's all
deterministic: I programmed it). The only thing I can't recreate, is
the randomness, assuming I introduced that (but as we know, I can
recreate that anyway, because I'd just use the same "seed" state;
unless the source of randomness is "true").

>> I don't see how I could ever think "No, you
>> can't harm X". But what I find very interesting, is that even if I
>> knew *exactly* how a cat operated, I could never kill one.
>>> but I don't think making an artificial
>>> animal is as simple as you say.
>> So is it a complexity issue? That you only start to care about the
>> entity when it's significantly complex. But exactly how complex? Or is
>> it about the unknowningness; that the project is so large you only
>> work on a small part, and thus you don't fully know it's workings, and
>> then that is where the guilt comes in.
> I think unknowingness plays a big part, but it's because of our experience
> with people and animals, we project our own experience of consciousness on
> to them so that when we see them behave in certain ways we impute an inner
> life to them that includes pleasure and suffering.

Yes, I agree. So does that mean that, over time, if we continue using
these computer-based cats, we would become attached to them (i.e. your
Sony toys example

> > Indeed, this is something that concerns me as well. If we do create an
> > AI, and force it to do our bidding, are we acting immorally? Or
> > perhaps we just withhold the desire for the program to do it's "own
> > thing", but is that in itself wrong?
> >
> I don't think so.  We don't worry about the internet's feelings, or the air
> traffic control system.  John McCarthy has written essays on this subject
> and he cautions against creating AI with human like emotions precisely
> because of the ethical implications.  But that means we need to understand
> consciousness and emotions less we accidentally do something unethical.

Fair enough. But by the same token, what if we discover a way to
remove emotions from real-born children. Would it be wrong to do that?
Is "emotion" an inherent property that we should never be allowed to
remove, once created?

> Brent


OVERLAND axon ZANY interbree...
You received this message because you are subscribed to the Google Groups 
"Everything List" group.
To post to this group, send email to everything-l...@googlegroups.com.
To unsubscribe from this group, send email to 
For more options, visit this group at 

Reply via email to