One could use a curve to map the MC winning rate to an actual winning probability. It would take only thousands of games to learn such a curve (as opposed to the 30 million games used to train the value network in AlphaGo).
Álvaro. On Wed, Aug 31, 2016 at 8:24 PM, Dan Schmidt <d...@dfan.org> wrote: > Hi Andrew, I enjoy the AGA broadcasts. I am mostly just an observer on > this list rather than a go programming expert, but I do have a couple of > things to say on this topic: > > 1. This is done all the time on chess broadcasts these days, and I hate > it. I want to know what a strong player thinks is going on, not a computer. > This is a bigger "problem" with chess than go because computers are now > hugely stronger than the pro commentators, so the commentators become > largely reduced to interpreting the computer output rather than getting > across their own human ideas. Of course, a) this is less of an issue right > now as go programs (except for AlphaGo, which is private) are not quite pro > strength yet, b) top-level go games have less volatile swings than chess > games, and perhaps most importantly c) my opinion may be in the minority. > > 2. Win rate is tough. Many programs can produce the win rate of their > Monte Carlo Tree Search, but because that involves looking at lots of > suboptimal moves on both sides, the reported win rate is closer to 50% than > it should be. A reported win rate of 60% (in the futures explored by MCTS) > probably means at least 80% in reality, assuming high-level play on both > sides. > > Crazy Stone reports an evaluation (like B+3.5) and confidence though it > cautions against taking it too seriously. My assumption is that it's > something like the median result, and standard deviation, of all the MCTS > playouts. I find this more useful than the win rate it provides. > > Dan > > > On Wed, Aug 31, 2016 at 6:41 PM, Jackson, Andrew <andrew.jack...@usgo.org> > wrote: > >> Hello all! Long time lurker, first time poster: I'm Andrew Jackson, i >> volunteer w/ the AGA in a number of capacities. >> >> Most recently, i've been putting together live broadcasts of games -- >> professional games from CJK, and also games from AGA events [1] >> >> These broadcasts feature professional commentary to help amateurs >> understand what's going on, but the number one question we get in the chat >> during the broadcasts is invariably "who's winning?" I was hoping >> computer-go could provide some suggestions on how we might run the >> game-in-progress through an engine to get an 'estimated winrate' -- or >> better yet, a graph -- that could be overlaid in the corner of the screen. >> >> Which brings me to the computer-go mailing list :) It seems like someone >> on this mailing list would probably have some good ideas about how this >> might be accomplished :) What do you think? >> >> >> Here are some more details: >> >> The game sgf is kept mirrored on KGS by a human transcriber w/ permission >> from the KBA. >> >> The overlay for these broadcasts is already rendered as a local webserver >> hosting a simple html shell holding the graphics; my thought was that this >> server could easily make requests to another service somewhere. That >> service would be tracking the game and able to send back some json >> describing the engine's view of the game, its confidence, etc. We could >> then plot with d3.js right on the overlay. >> >> The service would need to run somewhere else, as the computer streaming >> the game is already using all the CPU for transcoding & pushing the video. >> >> Given general overview of the goal & constraints, I ask you: >> >> - Does this sound plausible? >> - Any ideas for an engine that would be easy to hook this into? >> - Are bots capable of spectating games on KGS? >> >> Thanks in advance for any help you might provide! >> -Andrew >> >> >> _______________________________________________ >> Computer-go mailing list >> Computer-go@computer-go.org >> http://computer-go.org/mailman/listinfo/computer-go >> > > > _______________________________________________ > Computer-go mailing list > Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go >
_______________________________________________ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
