On Thu, Aug 21, 2008 at 5:33 PM, Valentina Poletti <[EMAIL PROTECTED]> wrote: > Sorry if I'm commenting a little late to this: just read the thread. Here is > a question. I assume we all agree that intelligence can be defined as the > ability to achieve goals. My question concerns the establishment of those > goals. As human beings we move in a world of limitations (life span, ethical > laws, etc.) and have inherent goals (pleasure vs pain) given by evolution. > An AGI in a different embodyment might not have any of that, just a pure > meta system of obtaining goals, which I assume, we partly give the AGI and > partly it establishes. Now, as I understand, the point of Singularity is > that of building an AGI more intelligent than humans so it could solve > problems for us that we cannot solve. That entails that the goal system of > the AGI and ours must be interconnected somehow. I find it difficult > to understand how that can be achieved with an AGI with a different type of > embodyment. I.e. planes are great in achieving flights, but are quite > useless to birds as their goal system is quite different. Can anyone > clarify? >
This is the question of Friendly AI: how to construct AGI that are good to have around, that are a right thing to launch Singularity with, what do we mean by goals, what do we want AGI to do and how to communicate this in implementation of AGI. Read CFAI ( http://www.singinst.org/upload/CFAI/index.html ) and the last arc of Eliezer's posts on Overcoming Bias to understand what the problem is about. This is a tricky question, not in the least because everyone seems to have a deep-down intuitive confidence that they understand what the problem is and how to solve it, out of hand, without seriously thinking about it. It takes much reading to even get what the question is and why it won't be answered "along the way", as AGI itself gets understood better, or by piling lots of shallow rules, hoping that AGI will construct what we want from these rules by the magical power of its superior intelligence. For example, "inherent goals (pleasure vs pain) given by evolution" doesn't even begin cut it, leading the investigation in the wrong direction. Hedonistic goal is answered by the universe filled with doped humans, and it's certainly not what is right, no more than a universe filled with paperclips. -- Vladimir Nesov [EMAIL PROTECTED] http://causalityrelay.wordpress.com/ ------------------------------------------- agi Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: https://www.listbox.com/member/?member_id=8660244&id_secret=111637683-c8fa51 Powered by Listbox: http://www.listbox.com
