I think most of test are designed by people who is stronger than best computer go program. So if MC program fail to pass a test, it is most likely MC is wrong. MC program is strong in some aspect, but week in other aspect. And the test suit is too focus on special aspect. We might split the test into different category, like opening, end game, L&D etc, and for each category we can set different weight. And when program pass all test, we can calculate final score adjusted by weight.
ZZ 2009/4/8 Alain Baeckeroot <alain.baecker...@laposte.net> > Le 08/04/2009 à 07:28, Petri Pitkanen a écrit : > > > > This is nice idea and this is to a degree what GnuGo regression test > > does. > afaik, gnugo testsuite (based on a previous one) is not totally suitable > for MC programs, as some position are dead lost / clear win but shows > gg misbehavior. > Some part should be re-usable, at least as a validation of simple > known L&D position for example. > Some times ago i tested a new (so weak) program on gg tests, it passed > less than 25%, when gg is above 65%. > > It would nice to know the results of various programs on this suite, > and see what happens. > > Probably this would lead to filter out many tests, and help gnugo team > to improve relevancy of the test suite ? > > > But as there is more than one way to skin the cat, it will not > > capture true strength of the programs. If you put Mogo to solve the > > tests from a book for example 501 opening problems, it will probably > > fail minimum of 75% of them because it has completely different style > > from what is expected from humans. This will also lead to different > > solution in connecting and capturing. MC program may reach the > > conclusion that it best option to ignore the whole connection issue. > > They often do exactly that. > > Thats why its a big task to set up a test suite (and we should start > now). Strong programs should have better results than weak ones ;-) > > Alain > > _______________________________________________ > computer-go mailing list > computer-go@computer-go.org > http://www.computer-go.org/mailman/listinfo/computer-go/ >
_______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
