If you run tests twice, you get nearly the same results ? Aja: you'll publish results with varying numbers of simulations for MC bots ? Olivier
2012/5/17 Rémi Coulom <remi.cou...@free.fr> > Hi Aja, > > Thanks for this interesting test. This is Crazy Stone's output for > seki_moves: > http://www.grappa.univ-lille3.fr/~coulom/seki-128k.html > Many correct answers are probably a bit lucky, because the evaluation is > rarely correct. > > It is not easy for me to implement sg_compare_float. But I'll try if I > find time. > > gnugo rules :-) > > Rémi > > On 15 mai 2012, at 04:39, Aja Huang wrote: > > > Dear all, > > > > Martin Mueller and I are writing a paper about exploring some > limitations of current MCTS programs in Go. For this purpose we have > carefully designed a regression test set which consists of 20 seki and 15 > two-safe-groups cases on 9x9 board. If you are interested, it is available > at > > > > > http://webdocs.cs.ualberta.ca/~mmueller/ps/seki-and-two-safe-groups-regression-test.zip > > > > We will appreciate if you would like to run your program over our > regression test and send us the results for our publication. > > > > It's easy to run your program through these positions (.sgf). Mainly, > the script run.sh under /utility is able to run a given program for a given > regression test file (.tst) and produce the result in a related html file. > For example, for the seki test you can simply type > > > > ./run.sh -p PATH_TO_PROGRAM -t g_seki_moves.tst > > > > Some notes: > > 1. Your program must support the command sg_compare_float for the > two-safe-groups test. If it doesn't support reg_genmove then the test file > g_seki_moves.tst is good to use which performs genmove instead. > > > > 2. On Windows platform, you will be able to execute 'run.sh' directly at > the command prompt after cygwin is installed. > > > > 3. If your program doesn't support the GTP command 'loadsgf', > gogui-adapter is able to translate 'loadsgf' into a sequence of 'play' > commands. The file gogui-adapter.jar under /utility is good to use because > Markus has fixed some bugs for us, see > > > > > https://sourceforge.net/tracker/?func=detail&aid=3522401&group_id=59117&atid=489964 > > > https://sourceforge.net/tracker/?func=detail&aid=3519829&group_id=59117&atid=489964 > > > > Under /experimental results, there are results of several programs such > as Fuego (tilburg version), pachi, ManyFaces and GnuGo. We thank David for > providing us the valuable results of ManyFaces. The test set is really not > easy because these programs all failed in many cases. > > > > Questions are very welcome. If you find any error in the test set please > inform us. Thanks. > > > > Best regards, > > Aja > > _______________________________________________ > > Computer-go mailing list > > Computer-go@dvandva.org > > http://dvandva.org/cgi-bin/mailman/listinfo/computer-go > > _______________________________________________ > Computer-go mailing list > Computer-go@dvandva.org > http://dvandva.org/cgi-bin/mailman/listinfo/computer-go > -- ========================================================= Olivier Teytaud -- olivier.teyt...@inria.fr TAO, LRI, UMR 8623(CNRS - Universite Paris-Sud), bat 490 Universite Paris-Sud F-91405 Orsay Cedex France http://0z.fr/EJm0g (one of the 56.5 % of french who did not vote for Sarkozy in 2007)
_______________________________________________ Computer-go mailing list Computer-go@dvandva.org http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
