On Mon, 2008-12-15 at 17:30 -0500, Weston Markham wrote: > Out of 3637 matches using 5k playouts, jrefgo won (i.e., was ahead > after 10 moves, as estimated by gnugo) 1688 of them. (46.4%) > Out of 2949 matches using 100k playouts, jrefgo won 785. (26.6%) > > It appears clear to me that increasing the number of playouts from 5k > to 100k certainly degrades the performance of jrefgo. Below, I am > including the commands that I used to run the tests and tally the > results.
Sometimes, it's possible for a bot to make a good move for the wrong reason. In other words, it doesn't understand the position. It's also possible to go the other way, a better program will play a weaker move in a particular situation. I remember once being showed a chess position where a master claimed that weaker players are more likely to play the correct move - but it went against a lot of the knowledge and patterns that strong players have. A monte carlo bot like refbot, in most positions is going to converge on some specific move. I think in the starting position it "wants" to play e5 and it is going to play e5 with an infinite number of playouts, whether than is the best move or not. There will be many situations where the move it "wants" to play is not the best, and so you can surmise that it's more likely to play a good move with fewer playouts. However, that by no means implies that it will play better with fewer playouts. It may play the worst move on the board too - the chances of that happening increases as the number of playouts drops. So this cuts both ways. With only 10 moves into the game, it's a possibility that a common trap, sequence or pattern is being played and together the three programs (refbot, mogo and gnugo) have conspired to make your results happen. It could be that refbot really does play a better move, but the better move is something that mogo is particularly good at handling. This is just speculation - but I think you have produced something that is like a bad pseudo random number generator that has obvious patterns and glitches. Another way to look at this is that "playing better" may not correlate well with increasing your chances of playing the best move - it's more about increasing your chances of playing a "good" move, or in my own personal opinion it's about avoiding bad moves. - Don _______________________________________________ computer-go mailing list [email protected] http://www.computer-go.org/mailman/listinfo/computer-go/
