Another alternatives for testing the performance would be 1. replay a game and count the number of right/near guesses. This could be pro games, games againt fairly good players, or games played by the bot before. The benefit of this is that you might be able to estimate the strength on every few moves rather than an entire game. 2. play against earlier versions of the same bot. 3. keep bots yourself. 4. improve benchmark/regression scores instead of playing strength, http://www.cs.ualberta.ca/~games/go/cgtc for instance.
See "twogtp" on how to have bots on your own computer play each other. This might also be of interest: http://www.andromeda.com/people/ddyer/go/shape-library.html On 9/20/07, Jason House <[EMAIL PROTECTED]> wrote: > Christoph Birk wrote: > >> // Loop to do #1 above > >> while (p != singletonSimplePass){ > >> if (numMoves < keepMax) > >> moves[numMoves] = p; > >> workingCopy.play(c,p); > >> c = c.enemyColor(); > >> p = randomLegalMove(c, workingCopy, twister); > >> numMoves++; > >> } > >> > > > > Do you really stop the simulation after a single pass, ie. when > > one side has no more move to play but the other does? > > I believe that this would end many games before they are (really) over > > and that might lead to false results in the simulations. > > > > Christoph > > > > Actually, that's the only difference between housebot-621-amaf and > hb-amaf-alt. alt is playing games all the way to the end like you > suggest. Looking at the win rate against ego110_allfirst, it looks like > it may be doing a bit worse (but more samples are needed). It's > unfortunate that ranks that low vary so much based on which bots are on > CGOS. > > In the future, I'll probably offer the option to do either method. My > logic behind stopping at the first pass is that it's highly unlikely to > form life in the void from captured stones. Since capturing the stones > would increase the length of the game and isn't very likely to change > the outcome of the game, I figured it'd be a good compromise. > _______________________________________________ > computer-go mailing list > [email protected] > http://www.computer-go.org/mailman/listinfo/computer-go/ > -- Cenny Wenner _______________________________________________ computer-go mailing list [email protected] http://www.computer-go.org/mailman/listinfo/computer-go/
