Hi Gunnar, Gunnar Farnebäck: <[EMAIL PROTECTED]>: >Heikki Levanto wrote: > > On Mon, Dec 10, 2007 at 04:08:48PM -0500, Don Dailey wrote: > >> Would you rather be 95% confident of a win or 90% confident? There is > >> only 1 correct answer to that question. > > > > Yes, if you can offer me reliable confidence numbers. We all (should) >know > > that MC evaluations suffer from systematic problems that can not just be > > averaged away statistically. > > > > Compare these two positions: > > > > playout_benchmark 10000 > > = Initial board: > > komi 7.5 > > A B C D E F G H J > > 9 . . . . . O O O O 9 > > 8 O O O O O O O O O 8 > > 7 O O O O O O O O O 7 > > 6 O O O O O O O O O 6 > > 5 # # # # # # # # # 5 > > 4 O O O # # # # # # 4 > > 3 O O O O . # # # # 3 > > 2 . O O O . # # # . 2 > > 1 # . O O . # # . # 1 > > A B C D E F G H J > > Performance: > > 10000 playouts > > 0.032002 seconds > > 312.481 kpps > > Black wins = 1937 > > White wins = 8063 > > P(black win) = 0.1937 > > > > > > playout_benchmark 10000 > > = Initial board: > > komi 7.5 > > A B C D E F G H J > > 9 . # . . . O O O O 9 > > 8 O O O O O O O O O 8 > > 7 O O O O O O O O O 7 > > 6 O O O O O O # # # 6 > > 5 # # # # # # # # # 5 > > 4 O O O # # # # # # 4 > > 3 O O O O . # # # # 3 > > 2 . O O O . # # # . 2 > > 1 . . O O . # # . # 1 > > A B C D E F G H J > > Performance: > > 10000 playouts > > 0.084006 seconds > > 119.039 kpps > > Black wins = 7746 > > White wins = 2254 > > P(black win) = 0.7746 > > > > > > Which one is better, 77% or 19%? > >This reminds me of the first testcase I wrote when I started with >MonteGNU. Black to play, no komi. > > A B C D E F G H J > 9 . . O O X . X . X 9 > 8 . . . O X . X O X 8 > 7 O . O O X X O O X 7 > 6 O O O . X . X O O 6 > 5 X X X X X O O O . 5 > 4 . . X . O O . O . 4 > 3 X X O X O . + O . 3 > 2 X X O X O . . O . 2 > 1 . O O O O . . . . 1 > A B C D E F G H J > >Naturally B has to play B8, or white plays there and wins big. This is >trivial to find for a classic program and easy enough for a Monte >Carlo program. What's interesting is that it takes some work to make >black think that it has better than even winning chances after B8. The >Monte Carlo code in GNU Go CVS version gets 0.079 with 10k, 0.387 with >100k, and 0.475 with 1M simulations. I suspect that stronger programs >tend to be more optimistic about winning chances here. So please fill >in this table if you have an MC program:
I've tested by my latest version of GGMC Go (ggmc-x86-v2RaveQ on CGOS, rated 2039 ELO now, which is about 100 ELO weaker than MonteGNU). > 10k 100k 1M >-------------------------------- >GNU Go CVS 0.079 0.387 0.475 GGMC v2 0.212 0.244 0.342 BTW, will GNU Go CVS be new version of GNU Go with MC? -Hideki >The sgf file is attached, load it before the first move. The positions >before move 3 and 5 are also relevant tests. > >/Gunnar >---- inline file >_______________________________________________ >computer-go mailing list >[email protected] >http://www.computer-go.org/mailman/listinfo/computer-go/ -- [EMAIL PROTECTED] (Kato) _______________________________________________ computer-go mailing list [email protected] http://www.computer-go.org/mailman/listinfo/computer-go/
