On Wed, 27 Nov 2019 at 13:25, Ian Dunstan <[email protected]> wrote: > > Hi Joseph, > > " > That is why I, personally, would like to see proof it really matters. like: > > -- Take a weak player - with the "weakest" MET file and the > "strongest" - and show a significant difference. > > " > > I understand your viewpoint. When Gergely and I can get the scripts and any > bug-fixes sorted we will first do some preliminary testing as you suggest. > > I think 0-ply v 0-ply using the Woolsey MET vs the Kaz-XG2 MET might be the > way to start. > > Then 2-ply v 2-ply using the Woolsey MET vs the Kaz-XG2 MET is probably > needed as well for comparison. > > Let's see what comes from these sorts of tests and move on from there. > > I think I probably need to ask Xavier how he arrived at his supposedly > 'significant difference' of 0.4 ELO between the ExtremGammon MET and the > Roc-Kaz MET. >
GNUBG ply-0 might not be weak enough :) I might be wrong here, but I "wanted" a weak player in the belief it will be easier to establish statistical significance. But I might be wrong and it is the other way. To get a weaker player you probably need to use a "weaker/lesser trained net". -Joseph > Cheers, Ian. > On Wednesday, 27 November 2019, 05:59:09 am AEDT, Joseph Heled > <[email protected]> wrote: > > > Hi Ian and bg fans, > > I would like to clarify my own view RE your post and the other replies. > > Any BG playing bot is a complex entity. GNUBG uses two-sided bearoff > tables, one-sided bearoff tables, three different types of neural > nets, not to mention my "pruning nets" -- And all this just to guess > the cubeless probabilities. For match play it uses the MET file and > *an algorithm* to covert cubeless to cubeful. GNUBG uses the > code+formulas by Joern Thyssen, which I never understood fully, and > was always slightly skeptical of. My own cube code is based on linear > interpolating, based on the ideas here > (https://bkgm.com/articles/met.html) which I think is the right > approach (danalyze.cc in gnubg-nn). > > Yes, those are details, but they matter. The playing abilities of a > bot are a complex emergent property of the system. The MET table is > just one part, and focusing on it is <very-personal-opinion>enormously > premature</very-personal-opinion>. My own experience indicates the > system as a whole is not that sensitive to the MET. That is why I, > personally, would like to see proof it really matters. like: > > -- Take a weak player - with the "weakest" MET file and the > "strongest" - and show a significant difference. > > Then we can start debating if it is worth while to "improve" the MET. > We have to use BOTS to develop this even if it is intended for human > use. Humans can't generate enough test data. > > I have more thoughts, but that is enough on this topic :) > > -Joseph > > On Tue, 26 Nov 2019 at 22:44, Ian Dunstan <[email protected]> wrote: > > > > Hi team, > > > > I have tried several subscription attempts to the email list directly, I > > think, without success. After finding the archives I see that there have > > been a number of replies to my first post. I didn't read them until very > > recently I wasn't ignoring your feedback, I just had not viewed it. Thank > > you, all, for what I received. Also, I apologise that this post almost > > certainly starts a new thread, though I considered it more important to > > make a response now. > >
