On Sat, Aug 30, 2008 at 8:54 PM, Terren Suydam <[EMAIL PROTECTED]> wrote: > --- On Sat, 8/30/08, Vladimir Nesov <[EMAIL PROTECTED]> wrote: > >> You start with "what is right?" and end with >> Friendly AI, you don't >> start with "Friendly AI" and close the circular >> argument. This doesn't >> answer the question, but it defines Friendly AI and thus >> "Friendly AI" >> (in terms of "right"). > > In your view, then, the AI never answers the question "What is right?". > The question has already been answered in terms of the algorithmic process > that determines its subgoals in terms of Friendliness.
There is a symbolic string "what is right?" and what it refers to, the thing that we are trying to instantiate in the world. The whole process of answering the question is the meaning of life, it is what we want to do for the rest of eternity (it is roughly a definition of "right" rather than over-the-top extrapolation from it). It is an immensely huge object, and we know very little about it, like we know very little about the form of a Mandelbrot set from the formula that defines it, even though it entirely unfolds from this little formula. What's worse, we don't know how to safely establish the dynamics for answering this question, we don't know the formula, we only know the symbolic string, "formula", that we assign some fuzzy meaning to. There is no final answer, and no formal question, so I use question-answer pairs to describe the dynamics of the process, which flows from question to answer, and the answer is the next question, which then follows to the next answer, and so on. With Friendly AI, the process begins with the question a human asks to himself, "what is right?". From this question follows a technical solution, initial dynamics of Friendly AI, that is a device to make a next step, to initiate transferring the dynamics of "right" from human into a more reliable and powerful form. In this sense, Friendly AI answers the question of "right", being the next step in the process. But initial FAI doesn't embody the whole dynamics, it only references it in the humans and learns to gradually transfer it, to embody it. Initial FAI doesn't contain the content of "right", only the structure of absorb it from humans. Of course, this is simplification, there are all kinds of difficulties. For example, this whole endeavor needs to be safeguarded against mistakes made along the way, including the mistakes made before the idea of implementing FAI appeared, mistakes in everyday design that went into FAI, mistakes in initial stages of training, mistakes in moral decisions made about what "right" means. Initial FAI, when it grows up sufficiently, needs to be able to look back and see why it turned out to be the way it did, was it because it was intended to have a property X, or was it because of some kind of arbitrary coincidence, was property X intended for valid reasons, or because programmer Z had a bad mood that morning, etc. Unfortunately, there is no objective morality, so FAI needs to be made good enough from the start to eventually be able to recognize what is valid and what is not, reflectively looking back at its origin, with all the depth of factual information and optimization power to run whatever factual queries it needs. I (vainly) hope this answered (at least some of the) other questions as well. -- Vladimir Nesov [EMAIL PROTECTED] http://causalityrelay.wordpress.com/ ------------------------------------------- agi Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: https://www.listbox.com/member/?member_id=8660244&id_secret=111637683-c8fa51 Powered by Listbox: http://www.listbox.com
