On Thu, Dec 27, 2012 at 11:12 AM, Ben Goertzel <[email protected]> wrote: > I think we will need to have a handful of expert humans rate the > results on a small test corpus.
What about preparing a set of questions and answers in advance? I am thinking about eliminating a source of bias, namely "I didn't think of that answer, but it is close enough". Of course this introduces a second bias, namely tuning the system to pass the test. For that you would need to withhold part of the test questions until the end. The problem I really want to avoid is building something according to plan that ends up not doing anything useful. This way we will know if it is useful or not before we build it. -- -- Matt Mahoney, [email protected] ------------------------------------------- AGI Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/21088071-f452e424 Modify Your Subscription: https://www.listbox.com/member/?member_id=21088071&id_secret=21088071-58d57657 Powered by Listbox: http://www.listbox.com
