RE: [computer-go] Rank on servers at 9x9
Many Faces has the same issue. The pruning and tuning that is required for 19x19 doesn't help 9x9. It seems that now the programs are strong enough that 9x9 requires a good opening book, and I'd rather spend my time making 19x19 stronger. David Zen's algorithm is getting heavier and heavier. It works well on 19x19 but does not so on 9x9. -- Yamato ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Rank on servers at 9x9
The difference between ManyFaces blitz and slow is also a lovely gem of information. A quick check shows both accounts are playing a lot of I have a data about this. AyaMC4 and AyaMC play on same condition, but it is not for human. AyaMC4 is blitz. Human is about 1 rank weaker in blitz. Hi Hiroshi, That is interesting; can I just check I've not misunderstood? You are saying that humans play about 1 rank (more precisely, 0.7 ranks) weaker when playing blitz against a fixed strength opponent? Going back to the Many Faces blitz (1.8d) and slow (0.6d), suggests that ManyFaces playing blitz against a human playing at slow time levels would be 1.8d - 0.7 = 1.1d. I.e. ManyFaces *loses* half a rank by thinking more :-) That is unlikely to be the case, implying people lose more strength playing blitz? In the Aya experiment I wonder if people playing slowly against a fast opponent tend to speed up and not use all their thinking time? Darren P.S. Using your last year's data, the difference is 1.2 ranks, which exactly matches the ManyFaces data, but still suggests ManyFaces gets no benefit from thinking more. AyaMC4 1k (0.1k) 10sec/mov (8cores) 1minute + 15sec byoyomi (x10) AyaMC 1k (0.8k) 10sec/mov (8cores) 10minutes + 30sec byoyomi (x5) AyaMC2 3k (2.6k) 1 playouts 10minutes + 30sec byoyomi (x5) AyaBot2 4k (3.6k) 5000 playouts 10minutes + 30sec byoyomi (x5) AyaBot4 5k (4.6k) 5000 playouts 10minutes + 30sec byoyomi (x5) These are last year's data, AyaMC4 2k (1.4k) 10sec/mov (8cores) 1minute + 15sec byoyomi (x10) AyaMC 3k (2.6k) 10sec/mov (8cores) 10minutes + 30sec byoyomi (x5) (about 16 playouts/mov) -- Darren Cook, Software Researcher/Developer Specializing in intelligent search (in multiple languages), discovery of context, aiding communication, and basically helping people find and make good use of their data. http://dcook.org/gobet/ (Shodan Go Bet - who will win?) http://dcook.org/mlsn/ (Multilingual open source semantic network) http://dcook.org/work/ (About me and my work) http://dcook.org/blogs.html (My blogs and articles) ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Rank on servers at 9x9
I think we can guess something from CGOS rating. I made a table about CGOS and KGS bot rating. At least in 19x19, CGOS and KGS rank is similar. CGOS KGS? CGOS 9x9 CGOS 19x19 KGS rank 19x19 1800 6k gnugo3.7.10 gnugo3.7.106k GnuGo (postneo, etc) 1900 5k 2000 4k 2100 3k Aya693c_1c 3k AyaMC2 (10k playouts) 2200 2k Aya693a_10k 2k pachi2 2300 1k pachi-c919f1k ManyFaces (30minutes) 2400 1d mfgo12-610-2c 1d ManyFaces1 (10sec blitz) 2500 2d Aya693_1c Zen-4.9-1c 2d Zen19 Zen (15sec/move) 2600 3d Fuego-1095-1c 2700 4d Zen-4.9-1c 2800 5d Zengg9-4x4c 2900 6d Hiroshi Yamashita - Original Message - From: Darren Cook dar...@dcook.org To: computer-go computer-go@computer-go.org Sent: Friday, February 12, 2010 4:43 PM Subject: [computer-go] Rank on servers at 9x9 Do any of the strongest MCTS programs have a rank at 9x9 on any major server? I found the fuego9 account on KGS but it appears to be unranked and only playing free games (*). The ManyFaces account appears to play only 19x19. I know the programs are stronger at 9x9 than 19x19, but I'm trying to get a figure of just exactly how strong they are against humans of known rank. Ideally in a statistically meaningful way (not just a short series of games against one player), and on non-supercomputer hardware. Thanks in advance, Darren ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Rank on servers at 9x9
Thanks for all the replies! It is much appreciated. Hiroshi Yamashita wrote: I think we can guess something from CGOS rating. I made a table about CGOS and KGS bot rating. At least in 19x19, CGOS and KGS rank is similar. That is a very interesting table. It is curious that Zen is only 2 ranks higher on 9x9. However Aya is 4 ranks higher and Fuego [1] is about 2kyu on KGS, so (if KGS and CGOS 19x19 correspond for it too, then) also 4 ranks higher. The difference between ManyFaces blitz and slow is also a lovely gem of information. A quick check shows both accounts are playing a lot of games, so this should be statistically significant; the ranking charts are hard to read, but it looks like 1 to 1.5 ranks difference. (Is there a way on KGS to get the underlying rating number?) Darren CGOS KGS? CGOS 9x9 CGOS 19x19 KGS rank 19x19 1800 6k gnugo3.7.10 gnugo3.7.106k GnuGo (postneo, etc) 1900 5k 2000 4k 2100 3k Aya693c_1c 3k AyaMC2 (10k playouts) 2200 2k Aya693a_10k 2k pachi2 2300 1k pachi-c919f1k ManyFaces (30minutes) 2400 1d mfgo12-610-2c 1d ManyFaces1 (10sec blitz) 2500 2d Aya693_1c Zen-4.9-1c 2d Zen19 Zen (15sec/move) 2600 3d Fuego-1095-1c 2700 4d Zen-4.9-1c 2800 5d Zengg9-4x4c 2900 6d Hiroshi Yamashita [1]: http://www.gokgs.com/graphPage.jsp?user=Fuego -- Darren Cook, Software Researcher/Developer Specializing in intelligent search (in multiple languages), discovery of context, aiding communication, and basically helping people find and make good use of their data. http://dcook.org/gobet/ (Shodan Go Bet - who will win?) http://dcook.org/mlsn/ (Multilingual open source semantic network) http://dcook.org/work/ (About me and my work) http://dcook.org/blogs.html (My blogs and articles) ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Rank on servers at 9x9
On Fri, Feb 12, 2010 at 10:08:07PM +0900, Darren Cook wrote: Thanks for all the replies! It is much appreciated. Hiroshi Yamashita wrote: I think we can guess something from CGOS rating. I made a table about CGOS and KGS bot rating. At least in 19x19, CGOS and KGS rank is similar. That is a very interesting table. It is curious that Zen is only 2 ranks higher on 9x9. However Aya is 4 ranks higher and Fuego [1] is about 2kyu on KGS, so (if KGS and CGOS 19x19 correspond for it too, then) also 4 ranks higher. I guess that this is because the higher your rating is, the more difficult it is to get even stronger. ;-) The difference between ManyFaces blitz and slow is also a lovely gem of information. 2300 1k pachi-c919f1k ManyFaces (30minutes) 2400 1d mfgo12-610-2c 1d ManyFaces1 (10sec blitz) My guess is that this is caused by huge amount of domain-knowledge from classical ManyFaces used to prune the tree initially - that gives big constant boost against MCTS bots with equal thinking time, but with additional time this does not contribute anything anymore and MFGo scales similarly to other MCTS bots. At least that's my interpretation, I wonder what David Fotland thinks about it. (Another factor is that 10sec blitz is much more prone for the human opponents to lose on time.) A quick check shows both accounts are playing a lot of games, so this should be statistically significant; the ranking charts are hard to read, but it looks like 1 to 1.5 ranks difference. (Is there a way on KGS to get the underlying rating number?) In your graphical editor from the rank graph, perhaps. ;-) -- Petr Pasky Baudis A great many people think they are thinking when they are merely rearranging their prejudices. -- William James ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Rank on servers at 9x9
The difference between ManyFaces blitz and slow is also a lovely gem of information. 2300 1k pachi-c919f1k ManyFaces (30minutes) 2400 1d mfgo12-610-2c 1d ManyFaces1 (10sec blitz) My guess is that this is caused by huge amount of domain-knowledge from classical ManyFaces used to prune the tree initially - that gives big constant boost against MCTS bots with equal thinking time, but with additional time this does not contribute anything anymore and MFGo scales similarly to other MCTS bots. The above stat was from KGS, not CGOS. So, my thought was: MCTS doesn't scale with extra thinking time as well as humans do. Which conveniently matches my hypothesis that MCTS doesn't use extra CPU cycles very efficiently. Though, as you say, those foolish humans foolishly managing their blitz time foolishly does distort things a bit. Darren -- Darren Cook, Software Researcher/Developer Specializing in intelligent search (in multiple languages), discovery of context, aiding communication, and basically helping people find and make good use of their data. http://dcook.org/gobet/ (Shodan Go Bet - who will win?) http://dcook.org/mlsn/ (Multilingual open source semantic network) http://dcook.org/work/ (About me and my work) http://dcook.org/blogs.html (My blogs and articles) ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Rank on servers at 9x9
Darren Cook wrote: Hiroshi Yamashita wrote: I think we can guess something from CGOS rating. I made a table about CGOS and KGS bot rating. At least in 19x19, CGOS and KGS rank is similar. That is a very interesting table. It is curious that Zen is only 2 ranks higher on 9x9. However Aya is 4 ranks higher and Fuego [1] is about 2kyu on KGS, so (if KGS and CGOS 19x19 correspond for it too, then) also 4 ranks higher. Zen's algorithm is getting heavier and heavier. It works well on 19x19 but does not so on 9x9. Another problem is 7.5 komi. I guess it gives very big advantage to white for high-dan level players. 7.0 komi would give better results. -- Yamato ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Rank on servers at 9x9
Darren Cook: 4b7552b7.9060...@dcook.org: Thanks for all the replies! It is much appreciated. Hiroshi Yamashita wrote: I think we can guess something from CGOS rating. I made a table about CGOS and KGS bot rating. At least in 19x19, CGOS and KGS rank is similar. That is a very interesting table. It is curious that Zen is only 2 ranks higher on 9x9. However Aya is 4 ranks higher and Fuego [1] is about 2kyu on KGS, so (if KGS and CGOS 19x19 correspond for it too, then) also 4 ranks higher. Please don't compare the ratings of different versions of programs on different hardware. The rating of Zen (at least version 4.2) on CGOS 9x9 is about 3 ranks higher than KGS, although (:-) only 1 rank higher than CGOS 19x19. Followings are the rating and rank of Zengg-4x4c (CGOS) and Zengg19 (KGS) respectively, though the games for Zengg19 aren't many enough. Zengg-4x4c and Zengg19 share the same code based on Zen-4.2 and hardware (a cluster of 4 quad-core PCs). CGOS 9x9 bayeselo 2919+75-64 CGOS 19x19 bayeselo 2830+80-71 KGS 19x19 3d I don't know why Hirosi didn't use bayeselo for the ratings on CGOS, which should give more reliable rating numbers. Below is another table using bayeselo. http://cgos.boardspace.net/9x9/bayes.html http://cgos.boardspace.net/19x19/bayes.html CGOS 9x9CGOS 19x19 KGS 19x19 GNU Go v3.7.10 lvl101805+7-61800 (ref) 6k Aya693_1c 2493+-742175+-443k AyaMC2 (10k po) Fuego svn1095 1c2629+-22 Fuego svn1095 2c2150+31-30 Fuego svn1089 8c2759+86-71 2353+-141 2k Fuego? Zen v4.9 1c 2795+74-68 2716+210-146 Zen v4.9 8c 2d Zen19? (15sec/move) Zengg (v4.2) 4x4c 2919+75-64 2830+80-71 3d Zengg19 (20:00+5x0:30) Note that Zen19 (KGS) runs on an 8-core MacPro while Zen-4.9-1c ran on just one core. I also wonder if Zen19 and Fuego on KGS are v4.9 and svn1089, respectively. Watching the ratings of GNU Go and Zengg on CGOS 19x19 and KGS in above table, they don't scale the same, say, 6k to 3d should be plus 800 Elo but is 1000 Elo on CGOS. Hideki The difference between ManyFaces blitz and slow is also a lovely gem of information. A quick check shows both accounts are playing a lot of games, so this should be statistically significant; the ranking charts are hard to read, but it looks like 1 to 1.5 ranks difference. (Is there a way on KGS to get the underlying rating number?) Darren CGOS KGS? CGOS 9x9 CGOS 19x19 KGS rank 19x19 1800 6k gnugo3.7.10 gnugo3.7.106k GnuGo (postneo, etc) 1900 5k 2000 4k 2100 3k Aya693c_1c 3k AyaMC2 (10k playouts) 2200 2k Aya693a_10k 2k pachi2 2300 1k pachi-c919f1k ManyFaces (30minutes) 2400 1d mfgo12-610-2c 1d ManyFaces1 (10sec blitz) 2500 2d Aya693_1c Zen-4.9-1c 2d Zen19 Zen (15sec/move) 2600 3d Fuego-1095-1c 2700 4d Zen-4.9-1c 2800 5d Zengg9-4x4c 2900 6d Hiroshi Yamashita [1]: http://www.gokgs.com/graphPage.jsp?user=Fuego -- Darren Cook, Software Researcher/Developer Specializing in intelligent search (in multiple languages), discovery of context, aiding communication, and basically helping people find and make good use of their data. http://dcook.org/gobet/ (Shodan Go Bet - who will win?) http://dcook.org/mlsn/ (Multilingual open source semantic network) http://dcook.org/work/ (About me and my work) http://dcook.org/blogs.html (My blogs and articles) ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ -- g...@nue.ci.i.u-tokyo.ac.jp (Kato) ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
