Ben, On Fri, Dec 28, 2012 at 11:35 AM, Ben Goertzel <[email protected]> wrote:
> > We have plenty of unit tests in OpenCog, of course... > > And we will develop various "IQ tests" in the virtual world environment, > during the next year... > Right now, things are simple enough to debug with end-to-end testing. Obviously, this day will pass as soon as you get things working a little better. > > I think that developing better tools for probing, visualizing and studying > what the OpenCog system is doing internally, is going to be important as we > proceed... > That was my point. If that is where the future lies, than capture it. > > I'm not opposed to testing systems and their components, of course. It's > important.... I'm just skeptical of the quantitative-metric-focused, > "bake-off" mentality one sees e.g. in the machine learning community... > The "bake off" is what future benchmarking standards look like, before they become robust enough for general application. I remember similar words being applied to early CPU benchmarks, some of which are now universally accepted standards. Standards are led by the technology leaders. Presuming that you see OpenCog and its successors as leading the technology, then you should be leading the bake off, probably by putting up some test cases that no one else sees any other way of dealing with, rather than responding to the present bake off. Steve ================= > On Fri, Dec 28, 2012 at 2:29 PM, Steve Richfield < > [email protected]> wrote: > >> Ben, >> >> Hey, you may have just posted the essential kernel needed to see the true >> challenge here. Regarding your comment of: >> >> >> But remember this: if OpenCog **never** produces any compelling >>> demonstration, this will not disprove the underlying design or ideas >>> whatsoever. The failure to produce a highly intelligent OpenCog system, >>> does not disambiguate between the options >>> 1) The OpenCog design is not workable >>> 2) Ben failed to gather enough human or financial resources to get the >>> OpenCog system sufficiently implemented and tested and taught, to make it >>> do highly intelligent things. >>> >> >> Exactly the same can be said about ANY piece software where there is no >> ability to debug it. Rather than wasting my breath about what might be >> missing in the secret sauce, we should BOTH be looking at the debugability >> issue here. Without any unit test strategy, benchmark goals, simple test >> cases, an effective "control panel", or ANY of the usual trappings of the >> sorts of advanced debugging systems that would be needed to make such >> things work, OF COURSE it can never (by human hands) be made work. C'mon >> now, you are no newbie. You have been writing and debugging complex >> software long enough to see this. Right? >> >> By "debugging" I don't mean simply making the individual statements do >> what the programmer intended, but rather "teasing" the subsystems and >> collections of subsystems into doing what was hoped for. Apparently, you >> have been hoping to jump from basic debugging to system test without >> anything in between. Right? >> >> The toughest task that an AGI could EVER be called on to perform is the >> debugging of another AGI without adequate tools. Here, you have stated >> things in *clear* chicken-or-egg terms, that as things now stand, you >> need an AGI to ever debug an AGI. Can you see this? >> >> It would seem that the next obvious do-it-or-throw-in-the-towel step >> would be to put a new sheet of butcher paper on your wall, and start >> sketching out what a screen shot of a control panel might look like, what a >> frame in a test set might look like, etc., sufficient to support the most >> advanced approaches to debugging as applied to OpenCog and its successors. >> After all, your whole approach seems to be to write SOMETHING, and then >> debug it into operation, so why not actually go in that direction? >> >> BTW, perhaps you remember my call for some sort of universal "functional >> unit" interface a couple of years ago, so that relatively straightforward >> pieces could be "plugged" together in various ways? Something like that >> would go a LONG way to being able to attach an advanced debugging tool, as >> it would constrain any intended or unintended "magic" to individual >> functional units. >> >> Sure you might be able to write an AGI without any sort of standard >> functional unit interface, but I don't see ANY way short of creating >> countless advanced tools, one for each module, to ever debug such a thing. >> Right? >> >> ... and, if you can figure out an approach to do this, THAT may be more >> valuable IP than anything now in OpenCog. If you are right that it is >> possible to in effect debug an AGI into operation, then the debugging tool >> to make this happen **IS** the most important part. >> >> I'll look forward to seeing your patent. >> >> Steve >> >> *AGI* | Archives <https://www.listbox.com/member/archive/303/=now> >> <https://www.listbox.com/member/archive/rss/303/212726-deec6279> | >> Modify<https://www.listbox.com/member/?&;>Your Subscription >> <http://www.listbox.com> >> > > > > -- > Ben Goertzel, PhD > http://goertzel.org > > "My humanity is a constant self-overcoming" -- Friedrich Nietzsche > *AGI* | Archives <https://www.listbox.com/member/archive/303/=now> > <https://www.listbox.com/member/archive/rss/303/10443978-6f4c28ac> | > Modify<https://www.listbox.com/member/?&;>Your Subscription > <http://www.listbox.com> > -- Full employment can be had with the stoke of a pen. Simply institute a six hour workday. That will easily create enough new jobs to bring back full employment. ------------------------------------------- AGI Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/21088071-f452e424 Modify Your Subscription: https://www.listbox.com/member/?member_id=21088071&id_secret=21088071-58d57657 Powered by Listbox: http://www.listbox.com
