On 03/10/2008 06:17 AM, Mark Waser wrote:
Which, if either, of 'including all of humanity' rather than just
'friendly humanity', or 'excluding non-human friendlies (initially)'
do you see as the greater risk?
I see 'excluding non-human friendlies (initially)' as a tremendously
greater risk.
As of now, we are aware of no non-human friendlies, so the set of
excluded beings will in all likelihood be the empty set.
Actually, let me rephrase your question and turn it around -- Which,
if either, of 'not protecting all of humanity from Friendlies rather
than just friendly humanity' or 'being actively unfriendly' do you see
as a greater risk?
To answer your question, I don't see the "people are evil and will screw
it all up" scenario as being even remotely likely, for reasons of
self-interest among others. And I think it very likely that if it turns
out that including non-human friendlies is the right thing to do, that
the system will do as designed and renormalize accordingly.
As for the other option, my question was not about the dangers relating
to *who is or is not protected*, but rather *whose volition is taken
into account* in calculating the CEV, since your approach considers only
the volition of friendly humanity (and non-human friendlies but not
non-friendly humanity), while Eliezer's includes all of humanity.
joseph
-------------------------------------------
agi
Archives: http://www.listbox.com/member/archive/303/=now
RSS Feed: http://www.listbox.com/member/archive/rss/303/
Modify Your Subscription:
http://www.listbox.com/member/?member_id=8660244&id_secret=95818715-a78a9b
Powered by Listbox: http://www.listbox.com
