Hello, In the spare time of the cluster I ran some experiments, and here are the results which may be of some interest.
Experimental settings: 19x19 Go, gnugo 3.7.10 level 8, MoGo playing with "sudden death" setting in max number of simulations (so that it is independent of the processor and of the implementation). The tests are done with 10^6, 2*10^6, 4*10^6 and 8*10^6 simulations per game. (if you have a core 2 duo, it corresponds roughly to 3 minutes, 6 min, 12 min and 24 min games). The 10^6 is chosen as level "0" as with this number of simulations, the winning rate against gnugo is 50%. The total number of games are not very high (between 100 and 200 games), so results are not precise, but that is the best I have :-). The first column is results playing with black, second column playing with white (often 0/0 because handicap games are always played as black), the third is the percentage over the total number of games, and between parentheses an estimation of the empirical std of the results. Very short summary of results (limited to those settings): - H1 and H2 often give the same results. - doubling simulations gives about 100 ELO points. - doubling simulations gives a little less than one handicap stone. BTW these results are quite surprisingly not consistent with the rating of bots on KGS. First experiment, MoGo against itself at 10^6 simulations per game changing the handicap: H1 117/180 0/0, 65.00%/180 (+-3.56%) H2 112/180 0/0, 62.22%/180 (+-3.61%) H3 139/180 0/0, 77.22%/180 (+-3.13%) H4 155/180 0/0, 86.11%/180 (+-2.58%) H5 212/240 0/0, 88.33%/240 (+-2.07%) Following experiments are MoGo against GnuGo, changing the total number of simulations and the handicaps (reminder: 10^6 H0 gives 50%). Between parenthesis, an approximation of what would be the time settings on a core2 duo. 10^6 (3min): MOGO black: H1 55/80 0/0, 68.75%/80 (+-5.18%) H2 66/100 0/0, 66.00%/100 (+-4.74%) H3 66/100 0/0, 66.00%/100 (+-4.74%) H4 82/100 0/0, 82.00%/100 (+-3.84%) H5 75/100 0/0, 75.00%/100 (+-4.33%) GNUGO black: H1 56/100 0/0, 56.00%/100 (+-4.96%) H2 57/100 0/0, 57.00%/100 (+-4.95%) H3 68/100 0/0, 68.00%/100 (+-4.66%) H4 79/100 0/0, 79.00%/100 (+-4.07%) H5 119/138 0/0, 86.23%/138 (+-2.93%) 2*10^6 (6min) GNUGO black: H0, komi 7.5 23/62 24/58, 39.17%/120 (+-4.46%) H1 39/105 0/0, 37.14%/105 (+-4.72%) H2 40/100 0/0, 40.00%/100 (+-4.90%) H3 56/96 0/0, 58.33%/96 (+-5.03%) H4 64/97 0/0, 65.98%/97 (+-4.81%) H5 68/104 0/0, 65.38%/104 (+-4.67%) 4*10^6 (12min) GNUGO black: H0 --komi 7.5 43/200 40/200, 20.75%/400 (+-2.03%) H1 37/180 0/0, 20.56%/180 (+-3.01%) H2 73/180 0/0, 40.56%/180 (+-3.66%) H3 87/180 0/0, 48.33%/180 (+-3.72%) H4 113/180 0/0, 62.78%/180 (+-3.60%) H5 118/180 0/0, 65.56%/180 (+-3.54%) 8*10^6 (24min) GNUGO black: H0 --komi 7.5 38/195 24/191, 16.06%/386 (+-1.87%) H1 28/180 0/0, 15.56%/180 (+-2.70%) H2 48/180 0/0, 26.67%/180 (+-3.30%) H3 63/180 0/0, 35.00%/180 (+-3.56%) H4 98/180 0/0, 54.44%/180 (+-3.71%) H5 117/200 0/0, 58.50%/200 (+-3.48%) Sylvain
_______________________________________________ computer-go mailing list [email protected] http://www.computer-go.org/mailman/listinfo/computer-go/
