On Thu, Mar 6, 2008 at 11:23 PM, Mark Waser <[EMAIL PROTECTED]> wrote: > > Friendliness must include reasonable protection for sub-peers or else there > is no "enlightened self-interest" or "attractor-hood" to it -- since any > rational entity will realize that it could *easily* end up as a sub-peer. > The value of having that protection in Friendliness in case the super-entity > needs it should be added to my innate value (which it probably dwarfs) when > considering whether I should be snuffed out. Friendliness certainly allows > the involuntary conversion of sub-peers under dire enough circumstances (or > it wouldn't be "enlightened self-interest" for the super-peer) but there is > a *substantial* value barrier to it (to be discussed later). >
This is different from what I replied to (comparative advantage, which J Storrs Hall also assumed), although you did state this point earlier. I think this one is a package deal fallacy. I can't see how whether humans conspire to weed out wild carrots or not will affect decisions made by future AGI overlords. ;-) -- Vladimir Nesov [EMAIL PROTECTED] ------------------------------------------- agi Archives: http://www.listbox.com/member/archive/303/=now RSS Feed: http://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: http://www.listbox.com/member/?member_id=8660244&id_secret=95818715-a78a9b Powered by Listbox: http://www.listbox.com
