On 4/19/07, Rémi Coulom <[EMAIL PROTECTED]> wrote:
Christian Nilsson wrote:
> Hello all,
>
> I've recently unleashed a few versions of my program "Haste" on CGOS.
> Seeing their ratings sparked a bit of curiosity about the relative
> ratings they achieved. In self-play ( thousands of games ) they differ
> by more than twice as many ELO-points as on the server.
I don't test with self play, but I am not very surprised by the
difference you observe.
>
> I know it's bad to use self-play as a measure of improvement... but
> I'd still like to know if anyone else is seeing this rather large
> difference in self-play vs. CGOS. After all, most programs on CGOS
> appear to be using MonteCarlo/UCT with either light, medium or heavy
> playouts. Making them similar in many ways.
The presence of a few GNU Go's on CGOS moderates this phenomenon. For
instance, you can notice that CrazyStone-Fast has a score of 23/23
against Fatman (1800) and 9/17 against GNU Go 3.7.9 (1814). The reverse
is true for weaker bots. For instance Haste-10k-pure has 0/21 against
FatMan and 4/21 against GNU. These number may not be extremely
significant, but I believe they indicate a strong tendency.
That makes GNU Go a very good sparring partner for MC program: whatever
you do, there is little risk of overfitting GNU, because it works in
such a different way.
After running some tests against GNU Go I'm getting quite different
results than in self-play. It seems I've been focusing on features
that gave self-play a boost, but not general strength. Time to start
over. :)
Rémi
_______________________________________________
computer-go mailing list
[email protected]
http://www.computer-go.org/mailman/listinfo/computer-go/
_______________________________________________
computer-go mailing list
[email protected]
http://www.computer-go.org/mailman/listinfo/computer-go/
