Christian Nilsson wrote:
Hello all,
I've recently unleashed a few versions of my program "Haste" on CGOS.
Seeing their ratings sparked a bit of curiosity about the relative
ratings they achieved. In self-play ( thousands of games ) they differ
by more than twice as many ELO-points as on the server.
I don't test with self play, but I am not very surprised by the
difference you observe.
I know it's bad to use self-play as a measure of improvement... but
I'd still like to know if anyone else is seeing this rather large
difference in self-play vs. CGOS. After all, most programs on CGOS
appear to be using MonteCarlo/UCT with either light, medium or heavy
playouts. Making them similar in many ways.
The presence of a few GNU Go's on CGOS moderates this phenomenon. For
instance, you can notice that CrazyStone-Fast has a score of 23/23
against Fatman (1800) and 9/17 against GNU Go 3.7.9 (1814). The reverse
is true for weaker bots. For instance Haste-10k-pure has 0/21 against
FatMan and 4/21 against GNU. These number may not be extremely
significant, but I believe they indicate a strong tendency.
That makes GNU Go a very good sparring partner for MC program: whatever
you do, there is little risk of overfitting GNU, because it works in
such a different way.
Rémi
_______________________________________________
computer-go mailing list
[email protected]
http://www.computer-go.org/mailman/listinfo/computer-go/
