Leela has surprisingly large tactical holes. Right now it is throwing a good number of games against me in completely won endgames by fumbling away entirely alive groups.
As an example I attached one game of myself (3d), even vs Leela10 @7d. But this really isn't a onetime occurence. If you look around move 150, the game is completely over by human standards as well as Leelas evaluation (Leela will give itself >80% here) But then Leela starts doing weird things. 186 is a minor mistake, but itself does not yet throw the game. But it is the start of series of bad turns. 236 then is a non-threat in a Ko fight, and checking Leelas evaluation, Leela doesn't even consider the possibility of it being ignored. This is btw a common topic with Leela in ko fights - it does not look at all at what happens if the Ko threat is ignored. 238 follows up the "ko threat", but this move isn't doing anything either! So Leela passed twice now. Suddenly there is some Ko appearing at the top right. Leela plays this Ko fight in some suboptimal way, not fully utilizing local ko threats, but this is a concept rather difficult to grasp for AIs afaik. I can not 100% judge whether ignoring the black threat of 253 is correct for Leela, I have some doubts on this one too. With 253 ignored, the game is now heavily swinging, but to my judgement, playing the hane instead of 256 would still keep it rather close and I'm not 100% sure who would win it now. But Leela decides to completely bury itself here with 256, while giving itself still 70% to win. As slowly realization of the real game state kicks in, the rest of the game is then the usual MC-throw away style we have known for years. Still... in this game you can see how a series of massive tactical blunders leads to throwing a completely won game. And this is just one of many examples. And it can not be all pinned on Ko's. I have seen a fair number of games where Leela does similar mistakes without Ko involved, even though Ko's drastically increase Leelas fumble chance. At the same time, Leela is completely and utterly outplaying me on a strategical level and whenever it manages to not make screwups like the ones shown I stand no chance at all. Even 3 stones is a serious challenge for me then. But those mistakes are common enough to keep me around even. 2017-05-22 17:47 GMT+02:00 Erik van der Werf <erikvanderw...@gmail.com>: > On Mon, May 22, 2017 at 3:56 PM, Gian-Carlo Pascutto <g...@sjeng.org> > wrote: > >> On 22-05-17 11:27, Erik van der Werf wrote: >> > On Mon, May 22, 2017 at 10:08 AM, Gian-Carlo Pascutto <g...@sjeng.org >> > <mailto:g...@sjeng.org>> wrote: >> > >> > ... This heavy pruning >> > by the policy network OTOH seems to be an issue for me. My program >> has >> > big tactical holes. >> > >> > >> > Do you do any hard pruning? My engines (Steenvreter,Magog) always had a >> > move predictor (a.k.a. policy net), but I never saw the need to do hard >> > pruning. Steenvreter uses the predictions to set priors, and it is very >> > selective, but with infinite simulations eventually all potentially >> > relevant moves will get sampled. >> >> With infinite simulations everything is easy :-) >> >> In practice moves with, say, a prior below 0.1% aren't going to get >> searched, and I still regularly see positions where they're the winning >> move, especially with tactics on the board. >> >> Enforcing the search to be wider without losing playing strength appears >> to be hard. >> >> > Well, I think that's fundamental; you can't be wide and deep at the same > time, but at least you can chose an algorithm that (eventually) explores > all directions. > > BTW I'm a bit surprised that you are still able to find 'big tactical > holes' with Leela now playing as 8d KGS > > Best, > Erik > > > > _______________________________________________ > Computer-go mailing list > Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go >
leela10screwup.sgf
Description: application/go-sgf
_______________________________________________ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
