I think I've seen some discussion a while ago about a set of tests
used to test playing engines. I suppose it would make sense to
compile such a set for the reference bots. Matching a win-rate and
game-length after 1,000,000 playouts is a quick first milestone for
verification that can be done quickly. A winning rate of something
very close to 50% after thousands of games takes hours if not a whole
day. A set of situations with their expected result can provide
something in between. Maybe the original set posted by Yamato can be
used as a starting point. But I'd also include some very simple
situations that check whether the engines properly implement things
like ko, super-ko, seki, end of game etc...
So far I've been using unit-tests for things like this. But that's
not very easy to transfer to other engine implementations.
I haven't used the GoGui regression-test application yet but I'm
planning to look into it. The one thing I did notice is that the
problem diagram and the problem's solution are separated. Wouldn't it
have been easier to include the latter into the SGF? A simple format
like "#c? [result-set]" where c is the color to play, similar as the
format already used. Put that as a comment in the first or last SGF
node should be easy enough to parse. Or you could even define a
special SGF property for this information.
Maybe there's a well thought out reason why it is the way it is?
Mark
_______________________________________________
computer-go mailing list
computer-go@computer-go.org
http://www.computer-go.org/mailman/listinfo/computer-go/
