Speaking of bot to bot comparisons... Anyone knows anything about the "raw" strength of the XG variants? By raw I mean a the 0-ply equivalent, i.e. without any lookup.
I guess what I am asking is: how much of XG strength comes from quality of implementation, and how much from the basic net evaluation? -Joseph On 20 January 2012 12:03, Nikos Papahristou <[email protected]> wrote: > Interesting. > For comparison reasons with Palamedes and TD-Gammon, backgammons should > count as gammons. This changes the second performance metric to 0.6046ppg. > > Nikos > > On 20/1/2012 0:20 πμ, Joseph Heled wrote: >> Thanks for all of you who chipped in! >> >> Playing a 1 point match at 0 ply, gnubg won 7193 of 10000 matches, >> which is 0.4386 ppg in my book. >> Playing 10000 money sessions (gammons count, no cube) gnubg got 0.6296 >> ppg (the breakdown is 4504, 2298, 285 2202, 676, 35 ) >> >> -Joseph >> >> On 20 January 2012 06:16, Nikos Papahristou <[email protected]> wrote: >>> My evaluations of a starting roll 3-1 are exactly the same with Mark. >>> Joseph, my implementation of pubeval doesn't agree with the moves >>> selection in the games you sent. >>> In the first game, the first roll of pubeval (4-5) is played as 24/15, >>> and in the second game the first roll (52) is played as 13/8, 13/11. >>> >>> Nikos >>> >>> On 19/1/2012 17:14 μμ, Mark Higgins wrote: >>>> I get something different for a 3-1 off the starting board. Sorry for >>>> the non-standard format but hopefully it's intelligible. I'm printing >>>> out the layout and the pubeval value, ordered by descending value >>>> (based on my pubeval implementation). 8-5 6-5 is the best move. >>>> >>>> *|| | | | | ||| | | | | ||* >>>> *|o *| * o|* >>>> *|o *| * o|* >>>> *| *| * o|* >>>> *| *| o|* >>>> *| *| o|* >>>> *| | |* >>>> *| | |* >>>> *| | |* >>>> *| | *|* >>>> *| o| *|* >>>> *| o| *|* >>>> *|* o o| o *|* >>>> *|* o o| o *|* >>>> *|| | | | | ||| | | | | ||* >>>> *Value 10.3431* >>>> >>>> *|| | | | | ||| | | | | ||* >>>> *|o o *| * o|* >>>> *| *| * o|* >>>> *| *| * o|* >>>> *| *| o|* >>>> *| *| o|* >>>> *| | |* >>>> *| | |* >>>> *| | |* >>>> *| o| *|* >>>> *| o| *|* >>>> *| o| o *|* >>>> *|* o| o *|* >>>> *|* o| o *|* >>>> *|| | | | | ||| | | | | ||* >>>> *Value 7.22287* >>>> >>>> *|| | | | | ||| | | | | ||* >>>> *|o *| * o|* >>>> *|o *| * o|* >>>> *| *| * o|* >>>> *| *| o|* >>>> *| *| |* >>>> *| | |* >>>> *| | |* >>>> *| | |* >>>> *| o| *|* >>>> *| o| *|* >>>> *| o| o *|* >>>> *|* o| o *|* >>>> *|* o| o o *|* >>>> *|| | | | | ||| | | | | ||* >>>> *Value 6.73507* >>>> >>>> *|| | | | | ||| | | | | ||* >>>> *| o o *| * o|* >>>> *| *| * o|* >>>> *| *| * o|* >>>> *| *| o|* >>>> *| *| o|* >>>> *| | |* >>>> *| | |* >>>> *| | |* >>>> *| o| *|* >>>> *| o| *|* >>>> *| o| o *|* >>>> *|* o| o *|* >>>> *|* o| o *|* >>>> *|| | | | | ||| | | | | ||* >>>> *Value 6.64798* >>>> >>>> *|| | | | | ||| | | | | ||* >>>> *|o o *| * o|* >>>> *| *| * o|* >>>> *| *| * o|* >>>> *| *| o|* >>>> *| *| |* >>>> *| | |* >>>> *| | |* >>>> *| | |* >>>> *| o| *|* >>>> *| o| *|* >>>> *| o| o *|* >>>> *|* o| o *|* >>>> *|* o| o o *|* >>>> *|| | | | | ||| | | | | ||* >>>> *Value 6.61369* >>>> >>>> *|| | | | | ||| | | | | ||* >>>> *|o o *| * o|* >>>> *| *| * o|* >>>> *| *| * o|* >>>> *| *| o|* >>>> *| *| o|* >>>> *| | |* >>>> *| | |* >>>> *| | |* >>>> *| | *|* >>>> *| o| *|* >>>> *| o| o *|* >>>> *|* o| o *|* >>>> *|* o o| o *|* >>>> *|| | | | | ||| | | | | ||* >>>> *Value 6.44695* >>>> >>>> *|| | | | | ||| | | | | ||* >>>> *|o o *| * o|* >>>> *| *| * o|* >>>> *| *| * o|* >>>> *| *| o|* >>>> *| *| o|* >>>> *| | |* >>>> *| | |* >>>> *| | |* >>>> *| o| *|* >>>> *| o| *|* >>>> *| o| *|* >>>> *|* o| o *|* >>>> *|* o|o o *|* >>>> *|| | | | | ||| | | | | ||* >>>> *Value 6.17616* >>>> >>>> *|| | | | | ||| | | | | ||* >>>> *|o *| * o|* >>>> *|o *| * o|* >>>> *| *| * o|* >>>> *| *| o|* >>>> *| *| |* >>>> *| | |* >>>> *| | |* >>>> *| | |* >>>> *| | *|* >>>> *| o| *|* >>>> *| o| o *|* >>>> *|* o| o *|* >>>> *|* o o| o o *|* >>>> *|| | | | | ||| | | | | ||* >>>> *Value 6.09006* >>>> >>>> *|| | | | | ||| | | | | ||* >>>> *|o *| * o|* >>>> *|o *| * o|* >>>> *| *| * o|* >>>> *| *| o|* >>>> *| *| |* >>>> *| | |* >>>> *| | |* >>>> *| | |* >>>> *| o| *|* >>>> *| o| *|* >>>> *| o| *|* >>>> *|* o| o *|* >>>> *|* o|o o o *|* >>>> *|| | | | | ||| | | | | ||* >>>> *Value 5.81926* >>>> >>>> *|| | | | | ||| | | | | ||* >>>> *|o o *| * o|* >>>> *| *| * o|* >>>> *| *| * o|* >>>> *| *| o|* >>>> *| *| o|* >>>> *| | |* >>>> *| | |* >>>> *| | |* >>>> *| o| *|* >>>> *| o| *|* >>>> *| o| *|* >>>> *|* o| o *|* >>>> *|* o o| o *|* >>>> *|| | | | | ||| | | | | ||* >>>> *Value 5.76592* >>>> >>>> *|| | | | | ||| | | | | ||* >>>> *|o *| * o|* >>>> *|o *| * o|* >>>> *| *| * o|* >>>> *| *| o|* >>>> *| *| o|* >>>> *| | |* >>>> *| | |* >>>> *| | |* >>>> *| | *|* >>>> *| | *|* >>>> *| o| o *|* >>>> *|* o| o *|* >>>> *|* o o o| o *|* >>>> *|| | | | | ||| | | | | ||* >>>> *Value 5.65459* >>>> >>>> *|| | | | | ||| | | | | ||* >>>> *|o *| * o|* >>>> *|o *| * o|* >>>> *| *| * o|* >>>> *| *| o|* >>>> *| *| o|* >>>> *| | |* >>>> *| | |* >>>> *| | |* >>>> *| o| *|* >>>> *| o| *|* >>>> *| o| *|* >>>> *|* o| o *|* >>>> *|* o o| o *|* >>>> *|| | | | | ||| | | | | ||* >>>> *Value 5.63356* >>>> >>>> *|| | | | | ||| | | | | ||* >>>> *|o o *| * o|* >>>> *| *| * o|* >>>> *| *| * o|* >>>> *| *| o|* >>>> *| *| o|* >>>> *| | |* >>>> *| | |* >>>> *| | |* >>>> *| | *|* >>>> *| o| *|* >>>> *| o| o *|* >>>> *|* o| o *|* >>>> *|* o o| o *|* >>>> *|| | | | | ||| | | | | ||* >>>> *Value 5.55805* >>>> >>>> *|| | | | | ||| | | | | ||* >>>> *|o *| * o|* >>>> *|o *| * o|* >>>> *| *| * o|* >>>> *| *| o|* >>>> *| *| o|* >>>> *| | |* >>>> *| | |* >>>> *| | |* >>>> *| | *|* >>>> *| o| *|* >>>> *| o| o *|* >>>> *|* o| o *|* >>>> *|* o o| o *|* >>>> *|| | | | | ||| | | | | ||* >>>> *Value 5.44036* >>>> >>>> *|| | | | | ||| | | | | ||* >>>> *|o *| * o|* >>>> *|o *| * o|* >>>> *| *| * o|* >>>> *| *| o|* >>>> *| *| o|* >>>> *| | |* >>>> *| | |* >>>> *| | |* >>>> *| | *|* >>>> *| o| *|* >>>> *| o| *|* >>>> *|* o| o *|* >>>> *|* o o|o o *|* >>>> *|| | | | | ||| | | | | ||* >>>> *Value 4.76362* >>>> >>>> *|| | | | | ||| | | | | ||* >>>> *|o *| * o|* >>>> *|o *| * o|* >>>> *| *| * o|* >>>> *| *| o|* >>>> *| *| o|* >>>> *| | |* >>>> *| | |* >>>> *| | |* >>>> *| o| *|* >>>> *| o| *|* >>>> *| o| *|* >>>> *|* o| *|* >>>> *|* o o|o o *|* >>>> *|| | | | | ||| | | | | ||* >>>> *Value 3.78692* >>>> >>>> >>>> On Jan 19, 2012, at 9:57 AM, Joseph Heled wrote: >>>> >>>>> Here are all the possibilities for a 3,1 from the initial position. I >>>>> am worried that 8-5 6-5 sneaks at number 6 >>>>> >>>>> score gnubg id position >>>>> 8.00384996831 4HPwATDgc/ABIQ (0, 1, 0, 0, 0, 1, -5, 0, -3, 0, 0, 0, 5, >>>>> -5, 0, 0, 0, 3, 0, 5, 0, 0, 0, 0, -2, 0) >>>>> 7.83777993917 4HPwATDEc/ABKA (0, 1, 1, 0, 0, 0, -5, 0, -3, 0, 0, 0, 5, >>>>> -5, 0, 0, 0, 3, 0, 4, 0, 0, 1, 0, -2, 0) >>>>> 7.81063996255 4HPwATDgc/ABEg (0, 0, 1, 0, 1, 0, -5, 0, -3, 0, 0, 0, 5, >>>>> -5, 0, 0, 0, 3, 0, 5, 0, 0, 0, 0, -2, 0) >>>>> 7.75730997324 4HPwATDgc+IBKA (0, 1, 1, 0, 0, 0, -5, 0, -3, 0, 0, 0, 4, >>>>> -5, 0, 1, 0, 3, 0, 5, 0, 0, 0, 0, -2, 0) >>>>> 7.43968993425 4HPwATDQZ/ABKA (0, 1, 1, 0, 0, 0, -5, 0, -3, 0, 0, 0, 5, >>>>> -5, 0, 0, 0, 2, 0, 5, 1, 0, 0, 0, -2, 0) >>>>> 6.84054994583 4HPwATCwZ/ABMA (0, 2, 0, 0, 0, 0, -5, 0, -3, 0, 0, 0, 5, >>>>> -5, 0, 0, 0, 2, 0, 4, 2, 0, 0, 0, -2, 0) >>>>> 6.81688992679 4HPwATDga/ABIg (0, 1, 0, 0, 1, 0, -5, 0, -3, 0, 0, 0, 5, >>>>> -5, 0, 0, 0, 2, 1, 5, 0, 0, 0, 0, -2, 0) >>>>> 6.67967991531 4HPwATDQc/ABIg (0, 1, 0, 0, 1, 0, -5, 0, -3, 0, 0, 0, 5, >>>>> -5, 0, 0, 0, 3, 0, 4, 1, 0, 0, 0, -2, 0) >>>>> 6.67962995172 4HPwATDCc/ABMA (0, 2, 0, 0, 0, 0, -5, 0, -3, 0, 0, 0, 5, >>>>> -5, 0, 0, 0, 3, 0, 4, 0, 0, 0, 1, -2, 0) >>>>> 6.24834996462 4HPwATDIZ/ABMA (0, 2, 0, 0, 0, 0, -5, 0, -3, 0, 0, 0, 5, >>>>> -5, 0, 0, 0, 2, 0, 5, 0, 1, 0, 0, -2, 0) >>>>> 6.10885995626 4HPwATDgc+EBMA (0, 2, 0, 0, 0, 0, -5, 0, -3, 0, 0, 0, 4, >>>>> -5, 0, 0, 1, 3, 0, 5, 0, 0, 0, 0, -2, 0) >>>>> 4.3719098568 4HPwATDEa/ABMA (0, 2, 0, 0, 0, 0, -5, 0, -3, 0, 0, 0, 5, >>>>> -5, 0, 0, 0, 2, 1, 4, 0, 0, 1, 0, -2, 0) >>>>> 4.29143989086 4HPwATDga+IBMA (0, 2, 0, 0, 0, 0, -5, 0, -3, 0, 0, 0, 4, >>>>> -5, 0, 1, 0, 2, 1, 5, 0, 0, 0, 0, -2, 0) >>>>> 4.23469984531 4HPwATCkc/ABMA (0, 2, 0, 0, 0, 0, -5, 0, -3, 0, 0, 0, 5, >>>>> -5, 0, 0, 0, 3, 0, 3, 1, 0, 1, 0, -2, 0) >>>>> 4.15422987938 4HPwATDQc+IBMA (0, 2, 0, 0, 0, 0, -5, 0, -3, 0, 0, 0, 4, >>>>> -5, 0, 1, 0, 3, 0, 4, 1, 0, 0, 0, -2, 0) >>>>> 3.22565984726 4HPwATDQV/ABMA (0, 2, 0, 0, 0, 0, -5, 0, -3, 0, 0, 0, 5, >>>>> -5, 0, 0, 0, 1, 1, 5, 1, 0, 0, 0, -2, 0) >>>>> >>>>> >>>>> On 20 January 2012 03:45, Mark Higgins <[email protected] >>>>> <mailto:[email protected]>> wrote: >>>>>> :) I'll give it a try and check the couple games you sent. >>>>>> >>>>>> >>>>>> On Jan 19, 2012, at 9:39 AM, Joseph Heled wrote: >>>>>> >>>>>>> I was asking for help, not for more work. >>>>>>> >>>>>>> For example, anyone who implemented play against pubeval can check a >>>>>>> few of the moves I sent. >>>>>>> >>>>>>> -Joseph >>>>>>> >>>>>>> On 20 January 2012 03:35, Mark Higgins <[email protected] >>>>>>> <mailto:[email protected]>> wrote: >>>>>>>> I used the following post to define pubeval; is this the proper >>>>>>>> definition still: >>>>>>>> >>>>>>>> http://www.bkgm.com/rgb/rgb.cgi?view+610 >>>>>>>> >>>>>>>> ? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Jan 19, 2012, at 8:31 AM, Joseph Heled wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I just run 10000 1 point match games between gnubg and pubeval. I >>>>>>>>> think this is the most fair, since there are no gammons of cubes >>>>>>>>> involved. >>>>>>>>> >>>>>>>>> However, gnubg won 9840 of those, which makes me think something >>>>>>>>> is wrong. >>>>>>>>> >>>>>>>>> Can someone help me verify that pubeval makes the "right" moves? >>>>>>>>> >>>>>>>>> I can send a few games in oldmoves format, or can check specific >>>>>>>>> moves. >>>>>>>>> >>>>>>>>> I am attaching 2 games as examples. >>>>>>>>> >>>>>>>>> -Joseph >>>>>>>>> <game1.fibs><game2.fibs> > _______________________________________________ Bug-gnubg mailing list [email protected] https://lists.gnu.org/mailman/listinfo/bug-gnubg
