Handicap stones. - Dave Hillis -----Original Message----- From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Cc: computer-go@computer-go.org Sent: Sat, 27 Jan 2007 8:07 AM Subject: Re: [computer-go] early results
On Fri, 2007-01-26 at 20:18 -0800, David Doshay wrote: > I would highly recommend that you do your testing against > a different Go engine. Self-play is a weak indicator. > > Cheers, > David I agree that there is a pretty good amount of in-transitivity with self-play. However there is not a practical way to test with another program that I can see. You don't get reliable ELO ratings with programs significantly stronger or weaker. For instance if you play 1000 games, there is huge rating difference between beating an opponent once, or twice in these thousand games - and if that is your expectancy, you would have to play many hundreds of thousands of games to get a very precise measurement. Because UCT is a full tree search, I am not as concerned as I would be with SlugGo for instance, (although I agree it's still a problem) because Slugo uses gnugo directly to choose it's moves there is a highly incestuous relationship. - Don > > > On 26, Jan 2007, at 5:39 PM, Don Dailey wrote: > > > Here are some early results on the scalability study. > > > > Basically, level 2 beats level 1 83.6 percent of the time. > > level 4 beats level 2 90.0 percent of the time. > > > > Where a level is number of play-outs divided by 1024 > > > > Approximately 300 ELO between levels. I fixed level 1 to have > > an ELO of zero. > > > > Of course these numbers are still rough as there has only be > > a few games played between players so far. > > > > - Don > > > > > > > > games win% score Match Up > > ------ ------ ----- ------------------------------------------ > > 55 16.4% 159.0 0001 0002 > > > > 55 83.6% 202.0 0002 0001 > > 30 10.0% 153.2 0002 0004 > > > > 30 90.0% 207.8 0004 0002 > > > > > > > > Rating Win perc Tot Gms Ave Time Player > > ------- -------- ------- -------- ------ > > 589.1 90.000 30 396.5 0004 > > 269.1 57.647 85 225.3 0002 > > 0.0 16.364 55 120.7 0001 > > > > Black wins: 37 43.5 % > > White wins: 48 56.5 % > > > > > > _______________________________________________ > > computer-go mailing list > > computer-go@computer-go.org > > http://www.computer-go.org/mailman/listinfo/computer-go/ > _______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ________________________________________________________________________ Check Out the new free AIM(R) Mail -- 2 GB of storage and industry-leading spam and email virus protection.
_______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
