[computer-go] Re: Rank on servers at 9x9
Most recent results of Zen9 versus a female Korean 1p Nam Chi-hyeuong at an event on Feb 6 in Japan can be seen at KGS (IDs are Zen9 and daishodai1). There are 24 game records with Zen's 21 wins (I guess all but last one are practice). Zen9 ran on an eight core MacPro. Also see http://www.computer-go.info/h-c/index.html. So we might say Zen is stronger than 1p on 9x9. I think, however, we have to count that almost all human players are not so good on 9x9 as 19x19 where Nam Chi-hyeuong is 1p. This is the opposit of computer programs. Hideki Darren Cook: 4b7506ad.6080...@dcook.org: Do any of the strongest MCTS programs have a rank at 9x9 on any major server? I found the fuego9 account on KGS but it appears to be unranked and only playing free games (*). The ManyFaces account appears to play only 19x19. I know the programs are stronger at 9x9 than 19x19, but I'm trying to get a figure of just exactly how strong they are against humans of known rank. Ideally in a statistically meaningful way (not just a short series of games against one player), and on non-supercomputer hardware. Thanks in advance, Darren ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ -- g...@nue.ci.i.u-tokyo.ac.jp (Kato) ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Rank on servers at 9x9
I think we can guess something from CGOS rating. I made a table about CGOS and KGS bot rating. At least in 19x19, CGOS and KGS rank is similar. CGOS KGS? CGOS 9x9 CGOS 19x19 KGS rank 19x19 1800 6k gnugo3.7.10 gnugo3.7.106k GnuGo (postneo, etc) 1900 5k 2000 4k 2100 3k Aya693c_1c 3k AyaMC2 (10k playouts) 2200 2k Aya693a_10k 2k pachi2 2300 1k pachi-c919f1k ManyFaces (30minutes) 2400 1d mfgo12-610-2c 1d ManyFaces1 (10sec blitz) 2500 2d Aya693_1c Zen-4.9-1c 2d Zen19 Zen (15sec/move) 2600 3d Fuego-1095-1c 2700 4d Zen-4.9-1c 2800 5d Zengg9-4x4c 2900 6d Hiroshi Yamashita - Original Message - From: Darren Cook dar...@dcook.org To: computer-go computer-go@computer-go.org Sent: Friday, February 12, 2010 4:43 PM Subject: [computer-go] Rank on servers at 9x9 Do any of the strongest MCTS programs have a rank at 9x9 on any major server? I found the fuego9 account on KGS but it appears to be unranked and only playing free games (*). The ManyFaces account appears to play only 19x19. I know the programs are stronger at 9x9 than 19x19, but I'm trying to get a figure of just exactly how strong they are against humans of known rank. Ideally in a statistically meaningful way (not just a short series of games against one player), and on non-supercomputer hardware. Thanks in advance, Darren ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Rank on servers at 9x9
Thanks for all the replies! It is much appreciated. Hiroshi Yamashita wrote: I think we can guess something from CGOS rating. I made a table about CGOS and KGS bot rating. At least in 19x19, CGOS and KGS rank is similar. That is a very interesting table. It is curious that Zen is only 2 ranks higher on 9x9. However Aya is 4 ranks higher and Fuego [1] is about 2kyu on KGS, so (if KGS and CGOS 19x19 correspond for it too, then) also 4 ranks higher. The difference between ManyFaces blitz and slow is also a lovely gem of information. A quick check shows both accounts are playing a lot of games, so this should be statistically significant; the ranking charts are hard to read, but it looks like 1 to 1.5 ranks difference. (Is there a way on KGS to get the underlying rating number?) Darren CGOS KGS? CGOS 9x9 CGOS 19x19 KGS rank 19x19 1800 6k gnugo3.7.10 gnugo3.7.106k GnuGo (postneo, etc) 1900 5k 2000 4k 2100 3k Aya693c_1c 3k AyaMC2 (10k playouts) 2200 2k Aya693a_10k 2k pachi2 2300 1k pachi-c919f1k ManyFaces (30minutes) 2400 1d mfgo12-610-2c 1d ManyFaces1 (10sec blitz) 2500 2d Aya693_1c Zen-4.9-1c 2d Zen19 Zen (15sec/move) 2600 3d Fuego-1095-1c 2700 4d Zen-4.9-1c 2800 5d Zengg9-4x4c 2900 6d Hiroshi Yamashita [1]: http://www.gokgs.com/graphPage.jsp?user=Fuego -- Darren Cook, Software Researcher/Developer Specializing in intelligent search (in multiple languages), discovery of context, aiding communication, and basically helping people find and make good use of their data. http://dcook.org/gobet/ (Shodan Go Bet - who will win?) http://dcook.org/mlsn/ (Multilingual open source semantic network) http://dcook.org/work/ (About me and my work) http://dcook.org/blogs.html (My blogs and articles) ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Rank on servers at 9x9
On Fri, Feb 12, 2010 at 10:08:07PM +0900, Darren Cook wrote: Thanks for all the replies! It is much appreciated. Hiroshi Yamashita wrote: I think we can guess something from CGOS rating. I made a table about CGOS and KGS bot rating. At least in 19x19, CGOS and KGS rank is similar. That is a very interesting table. It is curious that Zen is only 2 ranks higher on 9x9. However Aya is 4 ranks higher and Fuego [1] is about 2kyu on KGS, so (if KGS and CGOS 19x19 correspond for it too, then) also 4 ranks higher. I guess that this is because the higher your rating is, the more difficult it is to get even stronger. ;-) The difference between ManyFaces blitz and slow is also a lovely gem of information. 2300 1k pachi-c919f1k ManyFaces (30minutes) 2400 1d mfgo12-610-2c 1d ManyFaces1 (10sec blitz) My guess is that this is caused by huge amount of domain-knowledge from classical ManyFaces used to prune the tree initially - that gives big constant boost against MCTS bots with equal thinking time, but with additional time this does not contribute anything anymore and MFGo scales similarly to other MCTS bots. At least that's my interpretation, I wonder what David Fotland thinks about it. (Another factor is that 10sec blitz is much more prone for the human opponents to lose on time.) A quick check shows both accounts are playing a lot of games, so this should be statistically significant; the ranking charts are hard to read, but it looks like 1 to 1.5 ranks difference. (Is there a way on KGS to get the underlying rating number?) In your graphical editor from the rank graph, perhaps. ;-) -- Petr Pasky Baudis A great many people think they are thinking when they are merely rearranging their prejudices. -- William James ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Rank on servers at 9x9
The difference between ManyFaces blitz and slow is also a lovely gem of information. 2300 1k pachi-c919f1k ManyFaces (30minutes) 2400 1d mfgo12-610-2c 1d ManyFaces1 (10sec blitz) My guess is that this is caused by huge amount of domain-knowledge from classical ManyFaces used to prune the tree initially - that gives big constant boost against MCTS bots with equal thinking time, but with additional time this does not contribute anything anymore and MFGo scales similarly to other MCTS bots. The above stat was from KGS, not CGOS. So, my thought was: MCTS doesn't scale with extra thinking time as well as humans do. Which conveniently matches my hypothesis that MCTS doesn't use extra CPU cycles very efficiently. Though, as you say, those foolish humans foolishly managing their blitz time foolishly does distort things a bit. Darren -- Darren Cook, Software Researcher/Developer Specializing in intelligent search (in multiple languages), discovery of context, aiding communication, and basically helping people find and make good use of their data. http://dcook.org/gobet/ (Shodan Go Bet - who will win?) http://dcook.org/mlsn/ (Multilingual open source semantic network) http://dcook.org/work/ (About me and my work) http://dcook.org/blogs.html (My blogs and articles) ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
[computer-go] ADMIN: Migration
Hi, I may need to bring everything down before the new site is ready. (Disappearing consultant and family illnesses have claimed needed time.) Timeline is uncertain. veg ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Rank on servers at 9x9
Darren Cook wrote: Hiroshi Yamashita wrote: I think we can guess something from CGOS rating. I made a table about CGOS and KGS bot rating. At least in 19x19, CGOS and KGS rank is similar. That is a very interesting table. It is curious that Zen is only 2 ranks higher on 9x9. However Aya is 4 ranks higher and Fuego [1] is about 2kyu on KGS, so (if KGS and CGOS 19x19 correspond for it too, then) also 4 ranks higher. Zen's algorithm is getting heavier and heavier. It works well on 19x19 but does not so on 9x9. Another problem is 7.5 komi. I guess it gives very big advantage to white for high-dan level players. 7.0 komi would give better results. -- Yamato ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Dynamic Komi's basics
On Thu, Feb 11, 2010 at 11:24:25PM +0900, Hiroshi Yamashita wrote: I also get a bit good result to change komi inside. My code is like this. Interesting, thank you! I already started experimenting with something quite similar this week too (no commit pushed out in pachi codebase yet), I call that online komi for myself. winning rate (wins/all) 0.544 (336/617) komi chage by previous thinking result. You mean adjusting komi after each simulation or after each move? (I think the former?) -- Petr Pasky Baudis A great many people think they are thinking when they are merely rearranging their prejudices. -- William James ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Dynamic Komi's basics
You mean adjusting komi after each simulation or after each move? (I think the former?) Yes, each simulation, like this, play w e5 genmove b AdjustingKomi(latest_komi); move = getBestMoveUCT(); latest_komi = CalculateAdjustingKomi(); =c3 play w d4 genmove b AdjustingKomi(latest_komi); move = getBestMoveUCT(); latest_komi = CalculateAdjustingKomi(); =e3 Undo or load position are a bit difficult... Hiroshi Yamashita ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Rank on servers at 9x9
Darren Cook: 4b7552b7.9060...@dcook.org: Thanks for all the replies! It is much appreciated. Hiroshi Yamashita wrote: I think we can guess something from CGOS rating. I made a table about CGOS and KGS bot rating. At least in 19x19, CGOS and KGS rank is similar. That is a very interesting table. It is curious that Zen is only 2 ranks higher on 9x9. However Aya is 4 ranks higher and Fuego [1] is about 2kyu on KGS, so (if KGS and CGOS 19x19 correspond for it too, then) also 4 ranks higher. Please don't compare the ratings of different versions of programs on different hardware. The rating of Zen (at least version 4.2) on CGOS 9x9 is about 3 ranks higher than KGS, although (:-) only 1 rank higher than CGOS 19x19. Followings are the rating and rank of Zengg-4x4c (CGOS) and Zengg19 (KGS) respectively, though the games for Zengg19 aren't many enough. Zengg-4x4c and Zengg19 share the same code based on Zen-4.2 and hardware (a cluster of 4 quad-core PCs). CGOS 9x9 bayeselo 2919+75-64 CGOS 19x19 bayeselo 2830+80-71 KGS 19x19 3d I don't know why Hirosi didn't use bayeselo for the ratings on CGOS, which should give more reliable rating numbers. Below is another table using bayeselo. http://cgos.boardspace.net/9x9/bayes.html http://cgos.boardspace.net/19x19/bayes.html CGOS 9x9CGOS 19x19 KGS 19x19 GNU Go v3.7.10 lvl101805+7-61800 (ref) 6k Aya693_1c 2493+-742175+-443k AyaMC2 (10k po) Fuego svn1095 1c2629+-22 Fuego svn1095 2c2150+31-30 Fuego svn1089 8c2759+86-71 2353+-141 2k Fuego? Zen v4.9 1c 2795+74-68 2716+210-146 Zen v4.9 8c 2d Zen19? (15sec/move) Zengg (v4.2) 4x4c 2919+75-64 2830+80-71 3d Zengg19 (20:00+5x0:30) Note that Zen19 (KGS) runs on an 8-core MacPro while Zen-4.9-1c ran on just one core. I also wonder if Zen19 and Fuego on KGS are v4.9 and svn1089, respectively. Watching the ratings of GNU Go and Zengg on CGOS 19x19 and KGS in above table, they don't scale the same, say, 6k to 3d should be plus 800 Elo but is 1000 Elo on CGOS. Hideki The difference between ManyFaces blitz and slow is also a lovely gem of information. A quick check shows both accounts are playing a lot of games, so this should be statistically significant; the ranking charts are hard to read, but it looks like 1 to 1.5 ranks difference. (Is there a way on KGS to get the underlying rating number?) Darren CGOS KGS? CGOS 9x9 CGOS 19x19 KGS rank 19x19 1800 6k gnugo3.7.10 gnugo3.7.106k GnuGo (postneo, etc) 1900 5k 2000 4k 2100 3k Aya693c_1c 3k AyaMC2 (10k playouts) 2200 2k Aya693a_10k 2k pachi2 2300 1k pachi-c919f1k ManyFaces (30minutes) 2400 1d mfgo12-610-2c 1d ManyFaces1 (10sec blitz) 2500 2d Aya693_1c Zen-4.9-1c 2d Zen19 Zen (15sec/move) 2600 3d Fuego-1095-1c 2700 4d Zen-4.9-1c 2800 5d Zengg9-4x4c 2900 6d Hiroshi Yamashita [1]: http://www.gokgs.com/graphPage.jsp?user=Fuego -- Darren Cook, Software Researcher/Developer Specializing in intelligent search (in multiple languages), discovery of context, aiding communication, and basically helping people find and make good use of their data. http://dcook.org/gobet/ (Shodan Go Bet - who will win?) http://dcook.org/mlsn/ (Multilingual open source semantic network) http://dcook.org/work/ (About me and my work) http://dcook.org/blogs.html (My blogs and articles) ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ -- g...@nue.ci.i.u-tokyo.ac.jp (Kato) ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
