On Tue, 2008-10-28 at 12:48 -0200, Mark Boon wrote: > What do those number mean exactly? Are those winning percentages > against your ref-bot with 1,000 playouts?
No, it is a massive tournament where each player is scheduled in a manner similar to CGOS. The number of playouts is identified by the name of the program, i.e. refbot-08192 is 8192 playouts. So what I see in the table is that mwRefbot at 4096 playouts is stronger than refbot-08192. mwRefbot is the mark williams enhanced bot. refbot-00000 has ZERO playouts - it plays a move randomly. refbot-00001 has 1 playout and of course is almost but not quite random. I define 00000 or random play to be ELO 0 - Don > > I'm not getting the same results as Michael does. I see some > improvement, but not nearly as much. > > I had to make a few changes however, as I was adding up the results > in a different way than your ref-bot. So what I did was implement > tracking the AMAF statistics exactly the same way as in your ref-bot. > Next I ran a few hundred games, comparing the win/hit ratio of the > newly implemented method with my old one to make sure I got the same > result. When that checked out I made the modifications as described > by Michael. It seems rather straightforward, but after 200 games I > only get a win-rate of ~55% with 2,000 playouts instead of the 63% he > has. That's a considerable difference. > > Possibly I introduced a little bug somewhere... Or I need a larger > test-sample. > > I do agree with Michael that this idea is more logical and simpler. I > had implemented a different way to check which side moved at which > location first. I believe it's more efficient but it doesn't lend > itself for Michael's method. However, your way of checking seems > rather costly as it's a loop of 1/4 N^2 iterations, where N is the > number of moves (which we know is 111 in the beginning). With > Michael's method this is reduced to 1/2N iterations. > > My ref-bot on CGOS is still going. It seems to be always within a few > ELO points of your Jref and Cref bots. So I'm going to stop it as I > don't see much point in populating CGOS with identical bots. It has > played over 500 games, which I think is enough empirical evidence > that the implementation is 'good enough'. > > Mark > > > On 28-okt-08, at 10:54, Don Dailey wrote: > > > I have empirical evidence that mwRefbot level 4096 (the Michael > > Williams > > enhanced reference bot at 4096 playouts) is stronger than the > > reference > > bot at the same (and even higher levels.) > > > > > > Rank Name Elo + - games score oppo. draws > > 1 mwRefbot-004096 2684 23 22 1529 78% 2140 0% > > 2 refbot-008192 2670 21 21 1532 73% 2274 0% > > 3 refbot-004096 2645 21 21 1528 71% 2251 0% > > 4 refbot-002048 2555 21 21 1530 63% 2237 0% > > 5 refbot-001024 2431 23 23 1529 54% 2178 0% > > 6 refbot-000512 2253 27 28 1531 47% 2066 0% > > 7 refbot-000256 2000 34 35 1529 42% 1932 0% > > 8 refbot-000128 1646 42 42 1529 45% 1655 0% > > 9 refbot-000064 1290 40 40 1529 50% 1321 0% > > 10 refbot-000032 970 40 40 1531 52% 1041 0% > > 11 refbot-000016 621 33 32 1529 55% 780 0% > > 12 refbot-000008 370 25 25 1529 47% 649 0% > > 13 refbot-000004 250 23 22 1529 43% 551 0% > > 14 refbot-000002 145 22 22 1529 35% 500 0% > > 15 refbot-000001 5 22 23 1528 22% 475 0% > > 16 refbot-000000 0 22 23 1531 22% 484 0% > > > > - Don > > > > > > _______________________________________________ > > computer-go mailing list > > [email protected] > > http://www.computer-go.org/mailman/listinfo/computer-go/ >
signature.asc
Description: This is a digitally signed message part
_______________________________________________ computer-go mailing list [email protected] http://www.computer-go.org/mailman/listinfo/computer-go/
