Thanks, I think you are spot on, Philippe. (And so i Theo) Easy for me does not imply that it is easy for the neural network.
-Øystein On Wed, Sep 23, 2020 at 10:27 PM Philippe Michel <philippe.mich...@free.fr> wrote: > On Wed, Sep 23, 2020 at 06:22:28PM +0200, Øystein Schønning-Johansen wrote: > > > I strongly believe that this is a pretty simple position to play > > For a neural net this is not so clear. With 7 men stacked on the 7 point > this is a position that will never happen in real play and is unlikely > in training. Neither will its immediate continuations. A plausible > adversarial example indeed. > > If you look at the temperature map you will see how badly gnubg plays > many rolls on the first turn. > > Using a one-sided bearoff database going to the 7 point instead of the 6 > helps (and going to the 11 point should totally eliminate the issue), > but another issue appears : the transition from race net evaluation to > bearoff database evaluation. If the former overvalues the position and > its close continuations, X will tend not to move the checker from the 11 > point further than the 8 point to stay with the more generous evaluator. > > > I think that X (on roll) will lose about 15% gammon. It should not be > > hard to roll this out. > > This is much less than that. Theodore in another follow-up got a quick > estimate of 2%. Since O needs doublets but X needs not getting them, one > can replace his 1/6 factor by 5/36 and get a slightly more accurate > number of about 1%, showing that your 2 ply rollout is much better than > 0 ply but still far from perfect. > > This is arguably a bug, but not a simple coding error : one (of > undoubtly many) weak spot in evaluations. >
