On Fri, Feb 28, 2014 at 11:06 AM, Ben Goertzel <[email protected]> wrote: > See > > http://hplusmagazine.com/2014/02/28/saving-the-world-with-analytical-philosophy/
And my blog comment. I read the book and reviewed it on Amazon. http://www.amazon.com/review/R1BC9TNLJSULFU/ref=cm_cr_rdp_perm?ie=UTF8&ASIN=B00IB4N4KU&linkCode=&nodeID=&tag= I still find it annoying that MIRI models AI as a powerful goal-directed optimization process, when in fact this has nothing to do with how AI is developed in practice. We don't build general purpose reinforcement learners and let them figure out how to satisfy some complex goal like winning at chess or Jeopardy. It makes no sense from an information theoretic point of view to transmit a large number of bits from the human brain to a goal system through slow channels like speech or writing, and then transmit that information one bit at a time to an AI through a reward signal. The approach we use in practice is to update the AI through explicit instruction to meet the goals in the developer's mind. The AI learns faster when it doesn't have a goal. This is a problem of information theory. It won't be solved by recursive self improvement because an AI rewriting its code adds no bits at all. So MIRI needs to explain why this will change in the future if it believes that unfriendly AI is a threat. If faster hardware will make reinforcement learning practical, then keep in mind that human evolution required 10^48 DNA base copy operations, which is higher by a factor of 10^22 than the number of equivalent neural network operations performed by a human brain over a lifetime. If the threat is from self-replicating agents (which I think is real), then the book should mention it. -- -- Matt Mahoney, [email protected] ------------------------------------------- AGI Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/21088071-f452e424 Modify Your Subscription: https://www.listbox.com/member/?member_id=21088071&id_secret=21088071-58d57657 Powered by Listbox: http://www.listbox.com
