On Tue, Aug 26, 2008 at 9:54 PM, Terren Suydam <[EMAIL PROTECTED]> wrote: > > I didn't say the algorithm needs to be simple, I said the goal of > the algorithm ought to be simple. What are you trying to compute? > > Your answer is, "what is the right thing to do?" > > The obvious next question is, what does "the right thing" mean?
This is a part where you begin answering that question. > The only way that the answer to that is not context-dependent is > if there's such a thing as objective morality, something you've already > dismissed by referring to the "there are no universally compelling > arguments" post on the Overcoming Bias blog. > > You have to concede here that Friendliness is not objective. > Therefore, it cannot be expressed formally. It can only be approximated, > with error. The question itself doesn't exist in vacuum. When *you*, as a human, ask it, there is a very specific meaning associated with it. You don't search for the "meaning" that the utterance would call in a mind-in-general, you search for meaning that *you* give to it. Or, to make the it more reliable, for the meaning given by the idealized dynamics implemented in you ( http://www.overcomingbias.com/2008/08/computations.html ). -- Vladimir Nesov [EMAIL PROTECTED] http://causalityrelay.wordpress.com/ ------------------------------------------- agi Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: https://www.listbox.com/member/?member_id=8660244&id_secret=111637683-c8fa51 Powered by Listbox: http://www.listbox.com
