Robin, While I was at Cycorp, a concerted effort was made to address Vaughan Pratt's test questions. I recall that most of them required the addition of facts and rules into the Cyc KB so that they would answer. I believe that a substantial portion are included in the Cyc query regression test used to maintain the KB quality. This regression test is proprietary to Cycorp, and has not been released even in ResearchCyc that I know of.
Also, Cycorp has been working on using an extract of the Cyc KB as a resource for evaluating theorem provers, described here. Cheers, -Steve Stephen L. Reed Artificial Intelligence Researcher http://texai.org/blog http://texai.org 3008 Oak Crest Ave. Austin, Texas, USA 78704 512.791.7860 ________________________________ From: Robin Hanson <[EMAIL PROTECTED]> To: [email protected] Sent: Sunday, November 30, 2008 7:43:38 AM Subject: Re: [agi] Seeking CYC critiques Hi Stephen, nice to meet you. When I search the web for critiques of CYC, I can only find stuff from '90-95. If no one has written critiques of CYC since then, perhaps you could comment on how applicable those early critiques would be to the current system. For example, would CYC today at least better answer Vaughan Pratt's test questions from http://boole.stanford.edu/cyc.html? Has there been more progress toward developing a neutral source of questions to use to evaluate how performance improves with time and with implementation variations? At 01:57 AM 11/30/2008, Stephen Reed wrote: Hi Robin, There are no Cyc critiques that I know of in the last few years. I was employed seven years at Cycorp until August 2006 and my non-compete agreement expired a year later. An interesting competition was held by Project Halo in which Cycorp participated along with two other research groups to demonstrate human-level competency answering chemistry questions. Results are here. Although Cycorp performed principled deductive inference giving detailed justifications, it was judged to have performed inferior due to the complexity of its justifications and due to its long running times. The other competitors used special purpose problem solving modules whereas Cycorp used its general purpose inference engine, extended for chemistry equations as needed. My own interest is in natural language dialog systems for rapid knowledge formation. I was Cycorp's first project manager for its participation in the the DARPA Rapid Knowledge Formation project where it performed to DARPA's satisfaction, but subsequently its RKF tools never lived up to Cycorp's expectations that subject matter experts could rapidly extend the Cyc KB without Cycorp ontological engineers having to intervene. A Cycorp paper describing its KRAKEN system is here. I would be glad to answer questions about Cycorp and Cyc technology to the best of my knowledge, which is growing somewhat stale at this point. >What are the best available critiques of CYC as it exists now (vs. soon after >project started)? Robin Hanson [EMAIL PROTECTED] http://hanson.gmu.edu Research Associate, Future of Humanity Institute at Oxford University Associate Professor of Economics, George Mason University MSN 1D3, Carow Hall, Fairfax VA 22030-4444 703-993-2326 FAX: 703-993-2323 ________________________________ agi | Archives | Modify Your Subscription ------------------------------------------- agi Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: https://www.listbox.com/member/?member_id=8660244&id_secret=120640061-aded06 Powered by Listbox: http://www.listbox.com
