What's the point of getting a stable ranking which is more or less meaningless?
If someone want to?debug or test their programs, they could easily use the publicaly available version of MoGo or GnuGo. The purpose of the 19x19 CGOS is to compare strength between serious programs in a meaningful way. By meaningful, I refer to the human play.? DL -----Original Message----- From: David Fotland <[EMAIL PROTECTED]> To: 'computer-go' <computer-go@computer-go.org> Sent: Tue, 8 Jan 2008 2:23 pm Subject: RE: [computer-go] How to get more participation in 19x19 CGOS? I expect the faster time control will make my program look worse. It will certainly do worse vs Gnugo with less time. I expect it will also do worse against crazystone, since the crazystone version currently running is much faster than the time control, so it won't change strength at all. The problem with the current time limits is that it takes forever to get a stable rank, so people don't leave their programs on long enough. David > -----Original Message----- > From: [EMAIL PROTECTED] [mailto:computer-go- > [EMAIL PROTECTED] On Behalf Of Don Dailey > Sent: Tuesday, January 08, 2008 12:08 PM > To: computer-go > Subject: Re: [computer-go] How to get more participation in 19x19 CGOS? > > Hi David, > > Putting some weaker programs on CGOS is a good idea - I would like to > see it seeded with players of many different strengths. > > Olivier is in charge of cgos 19x19 so whatever time he wants to set is > up to him. I would suggest to him to set a 1 second time gift if he > chooses to play at a faster rate since network will be a real issue > with > any who have a lousy connection. > > If the time control is cut in half, it will probably change the > relative strengths of the programs, because some programs will not be > crippled by the time change and other will. Here is what I would > predict: > > 1. Gnugo would increase in ranking. > 2. Your program will increase in rank - but not as much as GnuGo. > 3. Crazy Stone will drop in rank. > 4. Every program will drop in ELO except GnuGo or other non- > scalable > programs. > > I'm assuming that your program is scalable, that Crazy Stone is more > scalable and GnuGo is not scalable (in other words it would run just > the > same as it does now.) > > So if we decrease the time control, it will make you look better and > you may get lazy about improving your program. However, it might > make GnuGo competitive with your program. I'm of course making > certain scalability assumptions that might not be true. > > My suggest to Olivier is to compromise and set time control 20 minutes > per side and give 1 second gift. > > - Don > > > > > David Fotland wrote: > > I think there are two reasons there are not more programs on 19x19 > CGOS: > > > > 1) The anchor, Gnugo, is quite strong, Many Faces 12 is stronger, and > > CrazyStone is much stronger. Since the programs playing are so > strong, it > > is demoralizing for a new program to lose so often. Without weaker > > competition, it is hard to get accurate ratings for new, weaker > programs. > > > > 2) The rounds take almost an hour, so it takes much too long to get > enough > > games to see how your program is doing. In my local testing, I use > 10 or 15 > > minutes per side. I like to see 50 to 100 test games get any > confidence > > that a new version is stronger. I prefer to get a test run complete > in > > under a day. > > > > I propose the following changes: > > > > I'll put up 3 weaker versions of Many Faces 12, so there is > competition at > > lower ratings. These versions are quite fast, using only a few > seconds for > > a full game. This will provide some stable opponents for the weaker > > programs. > > > > I think the time limits for CGOS19x19 should be reduced to 10 or 15 > minutes > > per side. This is enough to test programs, and it's still a > reasonable time > > limit for games against people. Since programs that search scale > with > > additional time, the relative ratings of these programs should be > similar at > > 10 minutes per game and 30 minutes per game. > > > > -David > > > > > > _______________________________________________ > > computer-go mailing list > > computer-go@computer-go.org > > http://www.computer-go.org/mailman/listinfo/computer-go/ > > > > > _______________________________________________ > computer-go mailing list > computer-go@computer-go.org > http://www.computer-go.org/mailman/listinfo/computer-go/ _______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ________________________________________________________________________ More new features than ever. Check out the new AOL Mail ! - http://webmail.aol.com
_______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
