> Martin, how close were the 2nd and 3rd choices?
The exact numbers vary each time you run it. But all three moves kept getting a
substantial number of simulations throughout as I was watching it. For a more
precise answer, you'd have to run it maybe 100 times and trace the number of
simulations for each move over time. One of the MoGo papers has such an
experiment.
>
> Looking at the bigger picture: is it yet possible for programs to do
> automatic
> post-game reviews of losing games - possibly spending a few hours or days on
> a
> single game - and locate weak spots in their own behavior?
In a few cases that will work, e.g. for one move blunders. But often, there is
a systematic misevaluation which means the program is off by 10 or 20 points
for a long period.
>
> Another question: at what point did Fuego realize that it had blundered? (
> when
> did the "winning percentage" show a sharp decline? ) Did it "know" that it
> was
> ahead prior to that weak move which arguably cost it the game?
Again the answer is "it depends" - as above.
>
> How close do such estimates correlate with the estimate of strong players?
> When
> they differ significantly, is it possible to say whether the program or the
> players are more correct?
Sufficiently strong players are always correct :)
Fuego, and I suspect other MCTS programs as well, suffer from the fact that
random simulations often get the tactical details wrong. On the small board
that is often hidden, because there are not many different fights, and the tree
is large enough to resolve many such tactics. On the big board it is often
fatal - you typically have five or ten settled local fights on the board by the
late middle game, but none of them may be 100% settled in the playouts, and a
few of them may be evaluated quite badly. If several get misevaluated in the
same direction, it adds up to a lot of bias. The effect is similar to thinking
that one of your groups is alive when in reality it is dead.
Martin
_______________________________________________
Computer-go mailing list
[email protected]
http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
