So why wouldn't we design a system that attempts to attain a nice simple goal like "make people happy" and build in the awareness that in order to define that goal in all its complexity, it needs to *ask* us what we want. Then the system iteratively refines that goal as new information comes in at the measly rate of "1 to 5 bits per second through speech, writing, or typing", as time is available and the need arises, making do with a less individualized but still highly effective definition of the general goal in the meantime. People recognize the value of information vs. the time it takes to communicate it, and will point out the most inconvenient misunderstandings first, so the system can rely on the users to selectively identify and convey the information it needs to know in order to meet their needs. In other words, if you want the system to be individualized to your preferences, you pay the cost of gathering & transmitting a description of your preferences. This is the current model for all those apps you mention: you go to the preferences page and check the boxes according to what you prefer. In the future, it will be communicated via natural language, but it will be the same principle at work.
On Fri, Aug 24, 2012 at 11:47 AM, Matt Mahoney <[email protected]>wrote: > On Fri, Aug 24, 2012 at 12:00 PM, Aaron Hosford <[email protected]> > wrote: > > > > Humans have a built in animal drive system (emotion & the pleasure/pain > > dichotomy), which works in tandem with the goal-less observation system > tha > > constitutes our intelligence. Without drive to give direction and > precedence > > to choices of behavior, I don't imagine the intelligence we exhibit would > > actually do anything. We would be difficult to control in the way a large > > boulder is difficult to control -- we would be inert. How does the AGI > > machine you propose decide what to do with the regularities it finds in > the > > incoming sensory data? Or is it also inert? > > How does the internet decide what to do? We tell it what to do. > > It is convenient to define intelligence as expected reward in > arbitrary environments. Convenient but not realistic. AI is complex > because the human brain is complex. We prefer simple theories. A > reward seeking optimization process is one of those simple theories of > human intelligence that doesn't work. > > We would really like to describe AGI as a really powerful optimization > process with nice simple goals like "make people happy". But human > goals are almost as complex as the system itself. You can't write them > all down. We have a legal system consisting of many millions of pages > which attempts to describe how people ought to behave. To do this for > AGI, you would have to make these rules precise to the level of bit > operations, with no human judges to resolve ambiguities in the law. Do > you really think this is a practical approach? > > The internet has billions of independent, competing agents that act at > the behest of their human owners. Specifying the behavior of these > agents is an expensive and time consuming process consisting of making > copies of agents (apps) that guess what the average person wants, and > then fine tuning them to individual preferences, one person at a time. > We can use information theory to estimate what this costs. Human long > term memory capacity is 10^9 bits according to Landauer's recall tests > using words, pictures, and sounds. We may assume that 99% of what you > know is written down somewhere or known to at least one other person. > The other 1% (10^7 bits) is what makes you unique. The U.S. Dept of > Labor estimate that it costs 1% of lifetime earnings ($15K) to replace > an employee. One way or another, you need to collect this 10^7 bits > from each of the 10^10 people on the planet to build an AGI that acts > according the individual preferences of its users. Humans can only > communicate at the rate of about 1 to 5 bits per second through > speech, writing, or typing. Human time (global per capita income) is > worth about $5 per hour (maybe more for you, but 70% of the world > population has no internet access yet). Collecting these preferences > will cost on the order of US$100 trillion. Any alternative method of > collecting this information (such as brain scanning or surveillance) > has to cost less than $10K to be competitive. > > > -- Matt Mahoney, [email protected] > > > ------------------------------------------- > AGI > Archives: https://www.listbox.com/member/archive/303/=now > RSS Feed: > https://www.listbox.com/member/archive/rss/303/23050605-bcb45fb4 > Modify Your Subscription: > https://www.listbox.com/member/?&; > Powered by Listbox: http://www.listbox.com > ------------------------------------------- AGI Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/21088071-c97d2393 Modify Your Subscription: https://www.listbox.com/member/?member_id=21088071&id_secret=21088071-2484a968 Powered by Listbox: http://www.listbox.com
