I have some ideas. When I learned go, I saw some small go books with many test suits for Life and Death, Connect etc. And each test have 4 candidate moves, each move has a score, best is 10 2nd is 6 3nd is 3 and the worst one is 0. After I finish a test, I will go a score. And a range of score is mapped to a level like 1d or 3d etc. Maybe we can gather these test suits from existed go books, and run tests on our program , then we can know how strong it is.
Another idea is more automatical one. Every computer go program has its own regression test suit. We can gather them togather, and let different existed programs to run the tests. And we can use our own program to run the test, and compare our results to exsited programs results, then we can know if our program is stronger or weaker than the existed programs. Zhiheng Zheng 2009/4/8 terry mcintyre <[email protected]> > Amen to that. When using positions to judge the strength of a program, one > would need to test not just one "pro move", but a sequence of plays -- > including some which don't appear in pro games. A pro knows how to deal > decisively not only with the optimal plays of other pros, but also with > suboptimal plays from the rest of us. Programs are often even stranger than > human players. > If I were designing a test set, I'd ask pros to defeat the program, and > would convert the blunders into a test set. To improve, the program would > have to generalize the lessons learned from those test cases. > > Terry McIntyre <[email protected]> > > -- People never lie so much as after a hunt, during a war or before an > election. - > Otto von Bismarck > > ------------------------------ > *From:* steve uurtamo <[email protected]> > *To:* computer-go <[email protected]> > *Sent:* Tuesday, April 7, 2009 5:12:27 PM > *Subject:* Re: [computer-go] Fast ways to evaluate program strength. > > otherwise pair-go wouldn't be as funny to watch. > > s. > > On Tue, Apr 7, 2009 at 8:05 PM, Michael Williams > <[email protected]> wrote: > > Łukasz Lew wrote: > >> > >> I would like to rephrase my question: > >> Let's measure prediction of pro moves of a whole engine while > >> modifying heavy playouts / MCTS in the engine. > >> How well might it work? > > > > Probably not well. Because what matters is not how often you play strong > > moves, but how often you avoid blunders. > > > > _______________________________________________ > > computer-go mailing list > > [email protected] > > http://www.computer-go.org/mailman/listinfo/computer-go/ > > > _______________________________________________ > computer-go mailing list > [email protected] > http://www.computer-go.org/mailman/listinfo/computer-go/ > > > _______________________________________________ > computer-go mailing list > [email protected] > http://www.computer-go.org/mailman/listinfo/computer-go/ >
_______________________________________________ computer-go mailing list [email protected] http://www.computer-go.org/mailman/listinfo/computer-go/
