On Wed, Sep 23, 2020 at 06:22:28PM +0200, Øystein Schønning-Johansen wrote:
> I strongly believe that this is a pretty simple position to play For a neural net this is not so clear. With 7 men stacked on the 7 point this is a position that will never happen in real play and is unlikely in training. Neither will its immediate continuations. A plausible adversarial example indeed. If you look at the temperature map you will see how badly gnubg plays many rolls on the first turn. Using a one-sided bearoff database going to the 7 point instead of the 6 helps (and going to the 11 point should totally eliminate the issue), but another issue appears : the transition from race net evaluation to bearoff database evaluation. If the former overvalues the position and its close continuations, X will tend not to move the checker from the 11 point further than the 8 point to stay with the more generous evaluator. > I think that X (on roll) will lose about 15% gammon. It should not be > hard to roll this out. This is much less than that. Theodore in another follow-up got a quick estimate of 2%. Since O needs doublets but X needs not getting them, one can replace his 1/6 factor by 5/36 and get a slightly more accurate number of about 1%, showing that your 2 ply rollout is much better than 0 ply but still far from perfect. This is arguably a bug, but not a simple coding error : one (of undoubtly many) weak spot in evaluations.
