On 03/09/2008 02:43 PM, Mark Waser wrote:
Why do you believe it likely that Eliezer's CEV of humanity would not
recognize your approach is better and replace CEV1 with your improved
CEV2, if it is actually better?
If it immediately found my approach, I would like to think that it
would do so (recognize that it is better and replace Eliezer's CEV
with mine).
Unfortunately, it is doesn't immediately find/evaluate my approach, it
might traverse some *really* bad territory while searching (with the
main problem being that I perceive the proportionality attractor as
being on the "uphill" side of the revenge attractor and Eliezer's
initial CEV as being "downhill" of all that).
It *might* get stuck in bad territory, but can you make an argument why
there is a *significant* chance of that happening? Given that humanity
has many times expanded the set of 'friendlies deserving friendly
behavior', it seems an obvious candidate for further research. And of
course, those smarter, better, more ... ones will be in a better
position than us to determine that.
One thing that I think most of will agree on is that if things did work
as Eliezer intended, things certainly could go very wrong if it turns
out that the vast majority of people -- when smarter, more the people
they wish they could be, as if they grew up more together ... -- are
extremely unfriendly in approximately the same way (so that their
extrapolated volition is coherent and may be acted upon). Our
meanderings through state space would then head into very undesirable
territory. (This is the "people turn out to be evil and screw it all up"
scenario.) Your approach suffers from a similar weakness though, since
it would suffer under the "seeming friendly people turn out to be evil
and screw it all up before there are non-human intelligent friendlies to
save us" scenario.
Which, if either, of 'including all of humanity' rather than just
'friendly humanity', or 'excluding non-human friendlies (initially)' do
you see as the greater risk? Or is there some other aspect of Eliezer's
approach that especially concerns you and motivates your alternative
approach?
Thanks for continuing to answer my barrage of questions.
joseph
-------------------------------------------
agi
Archives: http://www.listbox.com/member/archive/303/=now
RSS Feed: http://www.listbox.com/member/archive/rss/303/
Modify Your Subscription:
http://www.listbox.com/member/?member_id=8660244&id_secret=95818715-a78a9b
Powered by Listbox: http://www.listbox.com
