Stathis Papaioannou wrote:
> On 05/06/07, [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>* 
>     Self-improvement requires more than just extra hardware.  It also
>     requires the ability to integrate new knowledge with an existing
>     knowledge base in order to create truly orginal (novel) knowledge.
>     But this appears to be precisely the definition of reflective
>     intelligence!  Thus, it seems that a system missing reflective
>     intelligence simply cannot improve itself in an ordered way.  To
>     improve, a current goal structure has to be 'extrapolated' into a new
>     novel goal structure which none-the-less does not conflict with the
>     spirit of the old goal structure.  But nothing but a *reflective*
>     intelligence can possibly make an accurate assessment of whether a new
>     goal structure is compatible with the old version!  This stems from
>     the fact that comparison of goal structures requires a *subjective*
>     value judgement and it appears that only a *sentient* system can make
>     this judgement (since as far as we know, ethics/morality is not
>     objective).  This proves that only a *sentient* system (a *reflective
>     intelligence*) can possibly maintain a stable goal structure under
>     recursive self-improvement. 
> Why would you need to change the goal structure  in order to improve 
> yourself? 

Even more problematic: How would you know the change was an improvement?  An 
improvement relative to which goals, the old or the new?

Brent Meeker

You received this message because you are subscribed to the Google Groups 
"Everything List" group.
To post to this group, send email to [EMAIL PROTECTED]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at

Reply via email to