I don't think it's necessary to be self-aware to do self-modifications. Self-awareness implies that the entity has a model of the world that separates self from other, but this kind of distinction is not necessary to do self-modifications. It could act on itself without the awareness that it was acting on itself. (Goedelian machines would qualify, imo).
The reverse is true, as well. Humans are self aware but we cannot improve ourselves in the dangerous ways we talk about with the hard-takeoff scenarios of the Singularity. We ought to be worried about self-modifying agents, yes, but self-aware agents that can't modify themselves are much less worrying. They're all around us. --- On Tue, 8/26/08, David Hart <[EMAIL PROTECTED]> wrote: From: David Hart <[EMAIL PROTECTED]> Subject: Re: [agi] How Would You Design a Play Machine? To: [email protected] Date: Tuesday, August 26, 2008, 8:20 AM On 8/26/08, Mike Tintner <[EMAIL PROTECTED]> wrote: Is anyone trying to design a self-exploring robot or computer? Does this principle have a name? Interestingly, some views on AI advocate specifically prohibiting self-awareness and self-exploration as a precaution against the development of unfriendly AI. In my opinion, these views erroneously transfer familiar human motives onto 'alien' AGI cognitive architectures - there's a history of discussing this topic on SL4 and other places. I believe however that most approaches to designing AGI (those that do not specifically prohibit self-aware and self-explortative behaviors) take for granted, and indeed intentionally promote, self-awareness and self-exploration at most stages of AGI development. In other words, efficient and effective recursive self-improvement (RSI) requires self-awareness and self-exploration. If any term exists to describe a 'self-exploring robot or computer', that term is RSI. Coining a lesser term for 'self-exploring AI' may be useful in some proto-AGI contexts, but I suspect that 'RSI' is ultimately a more useful and meaningful term. -dave agi | Archives | Modify Your Subscription ------------------------------------------- agi Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: https://www.listbox.com/member/?member_id=8660244&id_secret=111637683-c8fa51 Powered by Listbox: http://www.listbox.com
