Ben, The test you described (Easter Egg Hunt) is a perfectly good example of the type of test I was looking for. When you run the experiment you will no doubt repeat it many times, adjusting various parameters. Then you will evaluate by how many eggs are found, how fast, and the extent to which it helps the system learns to play Hide and Seek (also a measurable quantity).
Two other good qualities are that the test is easy to describe and obviously relevant to intelligence. For text compression, the relevance is not so obvious. I look forward to seeing a paper on the outcome of the tests. -- Matt Mahoney, [EMAIL PROTECTED] ----- Original Message ---- From: Ben Goertzel <[EMAIL PROTECTED]> To: agi@v2.listbox.com Sent: Friday, November 3, 2006 10:51:16 PM Subject: Re: Re: Re: Re: [agi] Natural versus formal AI interface languages > I am happy enough with the long-term goal of independent scientific > and mathematical discovery... > > And, in the short term, I am happy enough with the goals of carrying > out the (AGISim versions of) the standard tasks used by development > psychologists to study childrens' cognitive behavior... > > I don't see a real value to precisely quantifying these goals, though... To give an example of the kind of short-term goal that I think is useful, though, consider the following. We are in early 2007 (if all goes according to plan) going to teach Novamente to carry out a game called "iterated Easter Egg hunt" -- basically, to carry out an Easter Egg hunt in a room full of other agents ... and then do so over and over again, modeling what the other agents do and adjusting its behavior accordingly. Now, this task has a bit in common with the game Hide-and-Seek. So, you'd expect that a Novamente instance that had been taught iterated Easter Egg Hunt, would also be good at hide-and-seek. So, we want to see that the time required for an NM system to learn hide-and-seek will be less if the NM system has previously learned to play iterated Easter Egg hunt... This sort of goal is, I feel, good for infant-stage AGI education.... However, I wouldn't want to try to turn it into an "objective IQ test." Our goal is not to make the best possible system for playing Easter Egg hunt or hide and seek or fetch or whatever.... And, in terms of language learning, our initial goal will not be to make the best possible system for conversing in baby-talk... Rather, our goal will be to make a system that can adequately fulfill these early-stage tasks, but in a way that we feel will be indefinitely generalizable to more complex tasks. This, I'm afraid, highlights a general issue with formal quantitative intelligence measures as applied to immature AGI systems/minds. Often the best way to achieve some early-developmental-stage task is going to be an overfitted, narrow-AI type of algorithm, which is not easily extendable to address more complex tasks. This is similar to my complaint about the Hutter Prize. Yah, a superhuman AGI will be an awesome text compressor. But this doesn't mean that the best way to achieve slightly better text compression than current methods is going to be **at all** extensible in the direction of AGI. Matt, you have yet to convince me that seeking to optimize interim quantitative milestones is a meaningful path to AGI. I think it is probably just a path to creating milestone-task-overfit narrow-AI systems without any real AGI-related expansion potential... -- Ben ----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303 ----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
