Matt, On Fri, Mar 6, 2015 at 7:32 PM, Matt Mahoney via AGI <[email protected]> wrote:
> On Fri, Mar 6, 2015 at 8:26 PM, Steve Richfield > <[email protected]> wrote: > > The objective of the competition is to uncover presently hidden > challenges that lie ahead, e.g. is it even possible to explain to people > that sometimes it is what they value the most that is the very PROBLEM they > wish to overcome. I suspect that AGIs will be SEVERELY limited by the very > limited mentalities (us) that they will have to work with. > > You mean the problem that it is impossible to convince someone that > you are smarter than them. Or conversely, it is impossible to > recognize when someone is smarter than you. This is a well known > problem. It's not arrogance. In order to give an IQ test, you have to > know the answers. > Your comments above are made with the presumption that no CONVERSATION is involved. To illustrate, when I was in junior high I was called into the principal's office because I had taken an IQ test and tested average!!! This test involved sequences of letters or numbers and multiple choices as to what the next logical letter or number should be, when I could easily see more than one answer. Perhaps you recall my prior posting where I had been looking into algebraic formulas to predict the next element in hybrid series, which I was doing at the time I took this test. I had asked about this during the test, and was told to select the simplest answer, but even that was difficult, because there was usually more than one answer that was pretty simple. In short, the "intelligence" test was more a display of the lack of intelligence of its designers than it was a test of my own intelligence. The principal accepted my explanation, the test result was stricken from my record, and my place as the school's smart guy remained intact. > > It is also why there won't be a singularity. We can't produce smarter > than human intelligence if we can't recognize it. > This is what I call heidenbugs - where the a program produces the correct answer, which appears to us mere humans to be obviously incorrect. However, this can also be flushed out in conversation. It is conversation that makes this exercise possible - you must subsequently SELL the correct answers to the judge, rather than simply providing correct answers for the judge to figure out. After the 10-page answers, the judge will inquire regarding "obvious weak points" whereupon the conversation will enter what salesmen call "the objection elimination phase". It will be in this phase that winners will emerge, and losers will go down in flames. To illustrate, an AGI might answer something like the following in response to a challenge over whether a particular action must be taken: "On page 4 paragraph 2 I used the term "must" to indicate that there would surely be eventual severe consequences for not taking this action, which far exceeds any possible benefit of not taking this action. Of course you could choose not to take this action, but in so doing you would be dooming your grandchildren to an early death." Note here the AGI pointing out how its answer was correct, when the judge failed to see this. I doubt there would be much use of "oops" in winning replies. Steve ------------------------------------------- AGI Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/21088071-f452e424 Modify Your Subscription: https://www.listbox.com/member/?member_id=21088071&id_secret=21088071-58d57657 Powered by Listbox: http://www.listbox.com
