One must be very careful about proclaiming wild transitivity issues. I'm not saying it's not an issue, there is some going on with every program on CGOS, but with less than 500 games between any two players you are going to get error margins of +/- 30-50 ELO or something like that.
And CGOS ratings are always going to have high error margins because they are incrementally rated. You should go by the bayeselo rating to get something more stable and you really need to have over 1000 games to begin trusting the ratings within 20 or 30 ELO. - Don On Thu, Oct 8, 2009 at 4:50 PM, Petr Baudis <pa...@ucw.cz> wrote: > On Thu, Oct 08, 2009 at 01:48:05PM -0600, Brian Sheppard wrote: > > About two weeks ago I took Pebbles offline for an extensive overhaul of > its > > board representation. At that time Valkyria 3.3.4 had a 9x9 CGOS rating > of > > roughly 2475. > > > > When I looked today, I saw Valkyria 3.3.4 rated at roughly 2334, so I > > wondered what was going on. > > > > I found a contributing factor: Valkyria has massively different results > > against Pachi than against Pebbles. It happens that Pachi started playing > a > > day or two after Pebbles went offline. > > > > Pebbles and Pachi are both rated around 2200, but Valkyria shreds Pebbles > a > > lot more often than Pachi: > > > > Pachi: 185 / 273 = 67.8% > > Pebbles: 429 / 503 = 85.3% > > > > There are a lot of lessons here... > > So, I'm curious how well Pachi will do against Pebbles. ;-) I'm hoping > that I'm nearing another major improvement in strength soon, so the > current version may not stay on CGOS much longer. > > (Curiously, the two identical Pachi instances were even after many games > very far apart in ELO points (about 100 ELO), somehow one of the > instances was winning against the other in about 80% of games; it > corrected itself after few more days without me doing anything, though. > Stochastic environments are funny.) > > -- > Petr "Pasky" Baudis > A lot of people have my books on their bookshelves. > That's the problem, they need to read them. -- Don Knuth > _______________________________________________ > computer-go mailing list > computer-go@computer-go.org > http://www.computer-go.org/mailman/listinfo/computer-go/ >
_______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
