On Tue, Oct 5, 2010 at 18:40, <[email protected]> wrote: > No information is thrown away with maximizing win rate.
That is not true. :) If you look for robustness median and quantile statistics are a good choice. But that is not necessary because playout results almost follwow Bernoulli distribution. Just look at the histogram. I say almost because it is not a sum of iid variables. Result is a sum of random components of not equall sizes. Also there is artefact of killing a whole board. (especially on 9x9) > > :-) > > This is because in go it is only the sign of the final board count that > matter. > > Yes I know I am very stubborn and narrow minded on this issue. > > It is a weird thing to fear throwing away "information" and then estimate > the expected score which will throw away the actual distributions of > outcomes. And so will any other statistical measure do. > > Also I do measure the score in at least two ways in Valkyria. One is simply > taking the average of score at the root and the other is to estimate > territory directly by looking at black/white membership of individual points > of the board. The latter is much more stable and the average is often off > several points also late in the endgame. > > It could also be that different programs have very different kind of > playouts. I know that the playouts of Valkyria contains really weird stuff. > Sometimes black wins a playout with +100 points because a lot of perfectly > safe white groups died because of some really unlikely combination of bugs, > omissions and randomness in the playout. Such a score does not give me > information with any value. Killing several groups due to bugs etc is not an issue since killing them is independent. > > I do agree that maybe expected score could be useful in the opening, because > the opening has no systematic bias yet. In the endgame however almost every > group will have a small probability of flipping state from alive to dead or > dead or alive, which has a little to do with actually theoretical score of > perfect play. But these probabilities will add up to something that on > average close to the true value but mostly it will be very wrong. > > So in short, I think win rates is the most robust thing to evaluate > positions using MC playouts. And someone else have to prove me wrong! I wont > do it. (But if someone do prove me wrong I will of course steal the idea and > implement it). > > -Magnus > > > > > > Quoting Łukasz Lew <[email protected]>: > >> On Tue, Oct 5, 2010 at 15:34, <[email protected]> wrote: >>> >>> Quoting Łukasz Lew <[email protected]>: >>> >>>> Your dynamic komi results are very convincing.. >>>> But shouldn't we just concentrate on maximizing score instead of >>>> winning rate in the beginning of the game? >>> >>> Maximizing winning rate means that the probability of having a score > 0 >>> at >>> the end of the game is maximum. >> >> I want to maximize expected score of a playout. >> I think that if we set komi so that around half of the playouts have >> score > 0 >> AND if the noise is large (the game is in the beginning) then >> maximixing winning rate >> is almost the same as maximizing score. >> >> The drawbacks of maximizing winning rate are: >> - we need to adhoc adjust komi to tell engine to concentrate on >> maximizing score >> - we throw away information. >> >> Of course maximizing winning rate is the right way in low-noise >> conditions (endgame). >> >>> >>> Exactly what do you mean with "maximizing score in the beginning of the >>> game"? >>> It is hard to estimate the score. Also in a game of go territory is not >>> everything. Aji and influence is also important. Win rate is as far as i >>> know the best way of capturing all these things in one measure that >>> guides >>> search. >>> >>> Magnus >>> >>> >>> _______________________________________________ >>> Computer-go mailing list >>> [email protected] >>> http://dvandva.org/cgi-bin/mailman/listinfo/computer-go >> >> >> >> -- >> Łukasz >> _______________________________________________ >> Computer-go mailing list >> [email protected] >> http://dvandva.org/cgi-bin/mailman/listinfo/computer-go > > > _______________________________________________ > Computer-go mailing list > [email protected] > http://dvandva.org/cgi-bin/mailman/listinfo/computer-go -- Łukasz _______________________________________________ Computer-go mailing list [email protected] http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
