Hi Aja, Thanks for this interesting test. This is Crazy Stone's output for seki_moves: http://www.grappa.univ-lille3.fr/~coulom/seki-128k.html Many correct answers are probably a bit lucky, because the evaluation is rarely correct.
It is not easy for me to implement sg_compare_float. But I'll try if I find time. gnugo rules :-) Rémi On 15 mai 2012, at 04:39, Aja Huang wrote: > Dear all, > > Martin Mueller and I are writing a paper about exploring some limitations of > current MCTS programs in Go. For this purpose we have carefully designed a > regression test set which consists of 20 seki and 15 two-safe-groups cases on > 9x9 board. If you are interested, it is available at > > http://webdocs.cs.ualberta.ca/~mmueller/ps/seki-and-two-safe-groups-regression-test.zip > > We will appreciate if you would like to run your program over our regression > test and send us the results for our publication. > > It's easy to run your program through these positions (.sgf). Mainly, the > script run.sh under /utility is able to run a given program for a given > regression test file (.tst) and produce the result in a related html file. > For example, for the seki test you can simply type > > ./run.sh -p PATH_TO_PROGRAM -t g_seki_moves.tst > > Some notes: > 1. Your program must support the command sg_compare_float for the > two-safe-groups test. If it doesn't support reg_genmove then the test file > g_seki_moves.tst is good to use which performs genmove instead. > > 2. On Windows platform, you will be able to execute 'run.sh' directly at the > command prompt after cygwin is installed. > > 3. If your program doesn't support the GTP command 'loadsgf', gogui-adapter > is able to translate 'loadsgf' into a sequence of 'play' commands. The file > gogui-adapter.jar under /utility is good to use because Markus has fixed some > bugs for us, see > > https://sourceforge.net/tracker/?func=detail&aid=3522401&group_id=59117&atid=489964 > https://sourceforge.net/tracker/?func=detail&aid=3519829&group_id=59117&atid=489964 > > Under /experimental results, there are results of several programs such as > Fuego (tilburg version), pachi, ManyFaces and GnuGo. We thank David for > providing us the valuable results of ManyFaces. The test set is really not > easy because these programs all failed in many cases. > > Questions are very welcome. If you find any error in the test set please > inform us. Thanks. > > Best regards, > Aja > _______________________________________________ > Computer-go mailing list > Computer-go@dvandva.org > http://dvandva.org/cgi-bin/mailman/listinfo/computer-go _______________________________________________ Computer-go mailing list Computer-go@dvandva.org http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
