Training on Stockfish games is guaranteed to produce a blunder-fest, because there are no blunders in the training set and therefore the policy network never learns how to refute blunders.
This is not a flaw in MCTS, but rather in the policy network. MCTS will eventually search every move infinitely often, producing asymptotically optimal play. But if the policy network does not provide the guidance necessary to rapidly refute the blunders that occur in the search, then convergence of MCTS to optimal play will be very slow. It is necessary for the network to train on self-play games using MCTS. For instance, the AGZ approach samples next states during training games by sampling from the distribution of visits in the search. Specifically: not by choosing the most-visited play! You see how this policy trains both search and evaluation to be internally consistent? The policy head is trained to refute the bad moves that will come up in search, and the value head is trained to the value observed by the full tree. From: Computer-go [mailto:computer-go-boun...@computer-go.org] On Behalf Of Dan Sent: Monday, March 5, 2018 4:55 AM To: computer-go@computer-go.org Subject: Re: [Computer-go] 9x9 is last frontier? Actually prior to this it was trained with hundreds of thousands of stockfish games and didn’t do well on tactics (the games were actually a blunder fest). I believe this is a problem of the MCTS used and not due to for lack of training. Go is a strategic game so that is different from chess that is full of traps. I m not surprised Lela zero did well in go. On Mon, Mar 5, 2018 at 2:16 AM Gian-Carlo Pascutto <g...@sjeng.org <mailto:g...@sjeng.org> > wrote: On 02-03-18 17:07, Dan wrote: > Leela-chess is not performing well enough I don't understand how one can say that given that they started with the random network last week only and a few clients. Of course it's bad! That doesn't say anything about the approach. Leela Zero has gotten strong but it has been learning for *months* with ~400 people. It also took a while to get to 30 kyu. -- GCP _______________________________________________ Computer-go mailing list Computer-go@computer-go.org <mailto:Computer-go@computer-go.org> http://computer-go.org/mailman/listinfo/computer-go
_______________________________________________ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
