Re: [Computer-go] level of MCTS programs depending on the number of sims in 9x9 (unlimited time setting for humans)
Hi Brian; thanks for sharing these results. I'll try to post here results in Kyu / Dan. For the first results I've seen it's a bit surprising to me; in 9x9 very small numbers of simulations have unexpectedly good results... but I'll wait for more stats for sharing as it's really surprising to me :-) (experiences with humans in the loop take so much time...) Incidentally, if you want to contribute, here a version of MoGo which adapts itself to your level; from the tables of results it generates you can see your level in MC simulations: http://teytaud.over-blog.com/article-a-linux-program-for-evaluating-your-level-in-go-91946330.html (sorry, only for Linux...) Best regards, Olivier 2011/11/19 Brian Sheppard sheppar...@aol.com I have been running Fuego 0.4.1 and Mogo3 on CGOS 9x9 for many thousands of games. Here is the data: ** ** Fuego0.4.1-100K,p2657 Fuego0.4.1-30K,p2520 Fuego0.4.1-10K,p2376 Fuego0.4.1-3K,p2205 Fuego0.4.1-1K,p2024 Fuego0.4.1-300,p1756 ** ** Mogo3MC90K,p2428 Mogo3MC30K,p2304 Mogo3MC10K,p2169 Mogo3MC3K,p 2008 Mogo3MC1K,p1823 ** ** *From:* computer-go-boun...@dvandva.org [mailto: computer-go-boun...@dvandva.org] *On Behalf Of *Olivier Teytaud *Sent:* Thursday, November 17, 2011 11:11 PM *To:* computer-go *Subject:* [Computer-go] level of MCTS programs depending on the number of sims in 9x9 (unlimited time setting for humans) ** ** Hi; Does anyone roughly know the level of a MCTS program in 9x9 depending on the number of simulations per move ? I know there are plenty of drawbacks to any such correspondence, that it depends on your style, on the time settings (I here assume the human has very long time...) and so on, but any rough estimate would be better than nothing :-) I would roughly say: 500 simulations = KGS 10 kyu 3 000 simulations = 5 kyu 15 000 simulations = KGS 1 Dan 3 000 000 simulations = pro level ...but I have no clear idea of this :-) (please don't spend too much time on the 1000 reasons for which such a correspondance does not make too much sense, we all know it :-) ) Best regards, Olivier ___ Computer-go mailing list Computer-go@dvandva.org http://dvandva.org/cgi-bin/mailman/listinfo/computer-go -- = Olivier Teytaud -- olivier.teyt...@inria.fr TAO, LRI, UMR 8623(CNRS - Universite Paris-Sud), bat 490 Universite Paris-Sud F-91405 Orsay Cedex France http://0z.fr/EJm0g (one of the 56.5 % of french who did not vote for Sarkozy in 2007) ___ Computer-go mailing list Computer-go@dvandva.org http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
Re: [Computer-go] level of MCTS programs depending on the number of sims in 9x9 (unlimited time setting for humans)
Let me guess: 1000 simulations per move ~= 1 Dan ? Erik On Sat, Dec 10, 2011 at 2:19 PM, Olivier Teytaud teyt...@lri.fr wrote: Hi Brian; thanks for sharing these results. I'll try to post here results in Kyu / Dan. For the first results I've seen it's a bit surprising to me; in 9x9 very small numbers of simulations have unexpectedly good results... but I'll wait for more stats for sharing as it's really surprising to me :-) (experiences with humans in the loop take so much time...) Incidentally, if you want to contribute, here a version of MoGo which adapts itself to your level; from the tables of results it generates you can see your level in MC simulations: http://teytaud.over-blog.com/article-a-linux-program-for-evaluating-your-level-in-go-91946330.html (sorry, only for Linux...) Best regards, Olivier 2011/11/19 Brian Sheppard sheppar...@aol.com I have been running Fuego 0.4.1 and Mogo3 on CGOS 9x9 for many thousands of games. Here is the data: Fuego0.4.1-100K,p 2657 Fuego0.4.1-30K,p 2520 Fuego0.4.1-10K,p 2376 Fuego0.4.1-3K,p 2205 Fuego0.4.1-1K,p 2024 Fuego0.4.1-300,p 1756 Mogo3MC90K,p 2428 Mogo3MC30K,p 2304 Mogo3MC10K,p 2169 Mogo3MC3K,p 2008 Mogo3MC1K,p 1823 From: computer-go-boun...@dvandva.org [mailto:computer-go-boun...@dvandva.org] On Behalf Of Olivier Teytaud Sent: Thursday, November 17, 2011 11:11 PM To: computer-go Subject: [Computer-go] level of MCTS programs depending on the number of sims in 9x9 (unlimited time setting for humans) Hi; Does anyone roughly know the level of a MCTS program in 9x9 depending on the number of simulations per move ? I know there are plenty of drawbacks to any such correspondence, that it depends on your style, on the time settings (I here assume the human has very long time...) and so on, but any rough estimate would be better than nothing :-) I would roughly say: 500 simulations = KGS 10 kyu 3 000 simulations = 5 kyu 15 000 simulations = KGS 1 Dan 3 000 000 simulations = pro level ...but I have no clear idea of this :-) (please don't spend too much time on the 1000 reasons for which such a correspondance does not make too much sense, we all know it :-) ) Best regards, Olivier ___ Computer-go mailing list Computer-go@dvandva.org http://dvandva.org/cgi-bin/mailman/listinfo/computer-go -- = Olivier Teytaud -- olivier.teyt...@inria.fr TAO, LRI, UMR 8623(CNRS - Universite Paris-Sud), bat 490 Universite Paris-Sud F-91405 Orsay Cedex France http://0z.fr/EJm0g (one of the 56.5 % of french who did not vote for Sarkozy in 2007) ___ Computer-go mailing list Computer-go@dvandva.org http://dvandva.org/cgi-bin/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@dvandva.org http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
Re: [Computer-go] level of MCTS programs depending on the number of sims in 9x9 (unlimited time setting for humans)
Someone should make a web interface for this. Might run into CPU capacity issue, though. On Sat, Dec 10, 2011 at 8:48 AM, Erik van der Werf erikvanderw...@gmail.com wrote: Let me guess: 1000 simulations per move ~= 1 Dan ? Erik On Sat, Dec 10, 2011 at 2:19 PM, Olivier Teytaud teyt...@lri.fr wrote: Hi Brian; thanks for sharing these results. I'll try to post here results in Kyu / Dan. For the first results I've seen it's a bit surprising to me; in 9x9 very small numbers of simulations have unexpectedly good results... but I'll wait for more stats for sharing as it's really surprising to me :-) (experiences with humans in the loop take so much time...) Incidentally, if you want to contribute, here a version of MoGo which adapts itself to your level; from the tables of results it generates you can see your level in MC simulations: http://teytaud.over-blog.com/article-a-linux-program-for-evaluating-your-level-in-go-91946330.html (sorry, only for Linux...) Best regards, Olivier 2011/11/19 Brian Sheppard sheppar...@aol.com I have been running Fuego 0.4.1 and Mogo3 on CGOS 9x9 for many thousands of games. Here is the data: Fuego0.4.1-100K,p2657 Fuego0.4.1-30K,p2520 Fuego0.4.1-10K,p2376 Fuego0.4.1-3K,p2205 Fuego0.4.1-1K,p2024 Fuego0.4.1-300,p1756 Mogo3MC90K,p2428 Mogo3MC30K,p2304 Mogo3MC10K,p2169 Mogo3MC3K,p 2008 Mogo3MC1K,p1823 From: computer-go-boun...@dvandva.org [mailto:computer-go-boun...@dvandva.org] On Behalf Of Olivier Teytaud Sent: Thursday, November 17, 2011 11:11 PM To: computer-go Subject: [Computer-go] level of MCTS programs depending on the number of sims in 9x9 (unlimited time setting for humans) Hi; Does anyone roughly know the level of a MCTS program in 9x9 depending on the number of simulations per move ? I know there are plenty of drawbacks to any such correspondence, that it depends on your style, on the time settings (I here assume the human has very long time...) and so on, but any rough estimate would be better than nothing :-) I would roughly say: 500 simulations = KGS 10 kyu 3 000 simulations = 5 kyu 15 000 simulations = KGS 1 Dan 3 000 000 simulations = pro level ...but I have no clear idea of this :-) (please don't spend too much time on the 1000 reasons for which such a correspondance does not make too much sense, we all know it :-) ) Best regards, Olivier ___ Computer-go mailing list Computer-go@dvandva.org http://dvandva.org/cgi-bin/mailman/listinfo/computer-go -- = Olivier Teytaud -- olivier.teyt...@inria.fr TAO, LRI, UMR 8623(CNRS - Universite Paris-Sud), bat 490 Universite Paris-Sud F-91405 Orsay Cedex France http://0z.fr/EJm0g (one of the 56.5 % of french who did not vote for Sarkozy in 2007) ___ Computer-go mailing list Computer-go@dvandva.org http://dvandva.org/cgi-bin/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@dvandva.org http://dvandva.org/cgi-bin/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@dvandva.org http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
Re: [Computer-go] level of MCTS programs depending on the number of sims in 9x9 (unlimited time setting for humans)
Pebbles is in between Mogo and Fuego in terms of simulation count and CPU effort. E.g., Pebbles-800 is better than Mogo-1000 and worse than Fuego-1000. My impression is that Erica needs even fewer trials. From the Erica papers I recall that Erica-300 was a formidable player. E.g., better than GnuGo (1800-rated). It is clear that Valkyria also needs very few trials to hit high ratings on 9x9. Brian _ From: computer-go-boun...@dvandva.org [mailto:computer-go-boun...@dvandva.org] On Behalf Of Olivier Teytaud Sent: Saturday, December 10, 2011 8:19 AM To: computer-go@dvandva.org Subject: Re: [Computer-go] level of MCTS programs depending on the number of sims in 9x9 (unlimited time setting for humans) Hi Brian; thanks for sharing these results. I'll try to post here results in Kyu / Dan. For the first results I've seen it's a bit surprising to me; in 9x9 very small numbers of simulations have unexpectedly good results... but I'll wait for more stats for sharing as it's really surprising to me :-) (experiences with humans in the loop take so much time...) Incidentally, if you want to contribute, here a version of MoGo which adapts itself to your level; from the tables of results it generates you can see your level in MC simulations: http://teytaud.over-blog.com/article-a-linux-program-for-evaluating-your-lev el-in-go-91946330.html (sorry, only for Linux...) Best regards, Olivier 2011/11/19 Brian Sheppard sheppar...@aol.com I have been running Fuego 0.4.1 and Mogo3 on CGOS 9x9 for many thousands of games. Here is the data: Fuego0.4.1-100K,p2657 Fuego0.4.1-30K,p2520 Fuego0.4.1-10K,p2376 Fuego0.4.1-3K,p2205 Fuego0.4.1-1K,p2024 Fuego0.4.1-300,p1756 Mogo3MC90K,p2428 Mogo3MC30K,p2304 Mogo3MC10K,p2169 Mogo3MC3K,p 2008 Mogo3MC1K,p1823 From: computer-go-boun...@dvandva.org [mailto:computer-go-boun...@dvandva.org] On Behalf Of Olivier Teytaud Sent: Thursday, November 17, 2011 11:11 PM To: computer-go Subject: [Computer-go] level of MCTS programs depending on the number of sims in 9x9 (unlimited time setting for humans) Hi; Does anyone roughly know the level of a MCTS program in 9x9 depending on the number of simulations per move ? I know there are plenty of drawbacks to any such correspondence, that it depends on your style, on the time settings (I here assume the human has very long time...) and so on, but any rough estimate would be better than nothing :-) I would roughly say: 500 simulations = KGS 10 kyu 3 000 simulations = 5 kyu 15 000 simulations = KGS 1 Dan 3 000 000 simulations = pro level ...but I have no clear idea of this :-) (please don't spend too much time on the 1000 reasons for which such a correspondance does not make too much sense, we all know it :-) ) Best regards, Olivier ___ Computer-go mailing list Computer-go@dvandva.org http://dvandva.org/cgi-bin/mailman/listinfo/computer-go -- = Olivier Teytaud -- olivier.teyt...@inria.fr TAO, LRI, UMR 8623(CNRS - Universite Paris-Sud), bat 490 Universite Paris-Sud F-91405 Orsay Cedex France http://0z.fr/EJm0g (one of the 56.5 % of french who did not vote for Sarkozy in 2007) ___ Computer-go mailing list Computer-go@dvandva.org http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
Re: [Computer-go] level of MCTS programs depending on the number ofsims in 9x9 (unlimited time setting for humans)
I just gave a quick try. The current version of Erica scores 75.4%(±3.2) against GnuGo 3.8 Level 10 on 9x9, with 300 playouts/move. It can be even stronger since the parameters are well tuned for 19x19 but not 9x9. I think Zen and Crazy Stone can reach higher winning rate. Aja From: Brian Sheppard Sent: Saturday, December 10, 2011 2:47 PM To: computer-go@dvandva.org Subject: Re: [Computer-go] level of MCTS programs depending on the number ofsims in 9x9 (unlimited time setting for humans) Pebbles is in between Mogo and Fuego in terms of simulation count and CPU effort. E.g., Pebbles-800 is better than Mogo-1000 and worse than Fuego-1000. My impression is that Erica needs even fewer trials. From the Erica papers I recall that Erica-300 was a formidable player. E.g., better than GnuGo (1800-rated). It is clear that Valkyria also needs very few trials to hit high ratings on 9x9. Brian From: computer-go-boun...@dvandva.org [mailto:computer-go-boun...@dvandva.org] On Behalf Of Olivier Teytaud Sent: Saturday, December 10, 2011 8:19 AM To: computer-go@dvandva.org Subject: Re: [Computer-go] level of MCTS programs depending on the number of sims in 9x9 (unlimited time setting for humans) Hi Brian; thanks for sharing these results. I'll try to post here results in Kyu / Dan. For the first results I've seen it's a bit surprising to me; in 9x9 very small numbers of simulations have unexpectedly good results... but I'll wait for more stats for sharing as it's really surprising to me :-) (experiences with humans in the loop take so much time...) Incidentally, if you want to contribute, here a version of MoGo which adapts itself to your level; from the tables of results it generates you can see your level in MC simulations: http://teytaud.over-blog.com/article-a-linux-program-for-evaluating-your-level-in-go-91946330.html (sorry, only for Linux...) Best regards, Olivier 2011/11/19 Brian Sheppard sheppar...@aol.com I have been running Fuego 0.4.1 and Mogo3 on CGOS 9x9 for many thousands of games. Here is the data: Fuego0.4.1-100K,p2657 Fuego0.4.1-30K,p2520 Fuego0.4.1-10K,p2376 Fuego0.4.1-3K,p2205 Fuego0.4.1-1K,p2024 Fuego0.4.1-300,p1756 Mogo3MC90K,p2428 Mogo3MC30K,p2304 Mogo3MC10K,p2169 Mogo3MC3K,p 2008 Mogo3MC1K,p1823 From: computer-go-boun...@dvandva.org [mailto:computer-go-boun...@dvandva.org] On Behalf Of Olivier Teytaud Sent: Thursday, November 17, 2011 11:11 PM To: computer-go Subject: [Computer-go] level of MCTS programs depending on the number of sims in 9x9 (unlimited time setting for humans) Hi; Does anyone roughly know the level of a MCTS program in 9x9 depending on the number of simulations per move ? I know there are plenty of drawbacks to any such correspondence, that it depends on your style, on the time settings (I here assume the human has very long time...) and so on, but any rough estimate would be better than nothing :-) I would roughly say: 500 simulations = KGS 10 kyu 3 000 simulations = 5 kyu 15 000 simulations = KGS 1 Dan 3 000 000 simulations = pro level ...but I have no clear idea of this :-) (please don't spend too much time on the 1000 reasons for which such a correspondance does not make too much sense, we all know it :-) ) Best regards, Olivier ___ Computer-go mailing list Computer-go@dvandva.org http://dvandva.org/cgi-bin/mailman/listinfo/computer-go -- = Olivier Teytaud -- olivier.teyt...@inria.fr TAO, LRI, UMR 8623(CNRS - Universite Paris-Sud), bat 490 Universite Paris-Sud F-91405 Orsay Cedex France http://0z.fr/EJm0g (one of the 56.5 % of french who did not vote for Sarkozy in 2007) ___ Computer-go mailing list Computer-go@dvandva.org http://dvandva.org/cgi-bin/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@dvandva.org http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
Re: [Computer-go] level of MCTS programs depending on the number of sims in 9x9 (unlimited time setting for humans)
Dear Erik, yes, something around that; I did not expect so much with so few simulations. (without opening book - but with small thinking time I think our opening books do not help...). Best regards, Olivier 2011/12/10 Erik van der Werf erikvanderw...@gmail.com Let me guess: 1000 simulations per move ~= 1 Dan ? Erik On Sat, Dec 10, 2011 at 2:19 PM, Olivier Teytaud teyt...@lri.fr wrote: Hi Brian; thanks for sharing these results. I'll try to post here results in Kyu / Dan. For the first results I've seen it's a bit surprising to me; in 9x9 very small numbers of simulations have unexpectedly good results... but I'll wait for more stats for sharing as it's really surprising to me :-) (experiences with humans in the loop take so much time...) Incidentally, if you want to contribute, here a version of MoGo which adapts itself to your level; from the tables of results it generates you can see your level in MC simulations: http://teytaud.over-blog.com/article-a-linux-program-for-evaluating-your-level-in-go-91946330.html (sorry, only for Linux...) Best regards, Olivier 2011/11/19 Brian Sheppard sheppar...@aol.com I have been running Fuego 0.4.1 and Mogo3 on CGOS 9x9 for many thousands of games. Here is the data: Fuego0.4.1-100K,p2657 Fuego0.4.1-30K,p2520 Fuego0.4.1-10K,p2376 Fuego0.4.1-3K,p2205 Fuego0.4.1-1K,p2024 Fuego0.4.1-300,p1756 Mogo3MC90K,p2428 Mogo3MC30K,p2304 Mogo3MC10K,p2169 Mogo3MC3K,p 2008 Mogo3MC1K,p1823 From: computer-go-boun...@dvandva.org [mailto:computer-go-boun...@dvandva.org] On Behalf Of Olivier Teytaud Sent: Thursday, November 17, 2011 11:11 PM To: computer-go Subject: [Computer-go] level of MCTS programs depending on the number of sims in 9x9 (unlimited time setting for humans) Hi; Does anyone roughly know the level of a MCTS program in 9x9 depending on the number of simulations per move ? I know there are plenty of drawbacks to any such correspondence, that it depends on your style, on the time settings (I here assume the human has very long time...) and so on, but any rough estimate would be better than nothing :-) I would roughly say: 500 simulations = KGS 10 kyu 3 000 simulations = 5 kyu 15 000 simulations = KGS 1 Dan 3 000 000 simulations = pro level ...but I have no clear idea of this :-) (please don't spend too much time on the 1000 reasons for which such a correspondance does not make too much sense, we all know it :-) ) Best regards, Olivier ___ Computer-go mailing list Computer-go@dvandva.org http://dvandva.org/cgi-bin/mailman/listinfo/computer-go -- = Olivier Teytaud -- olivier.teyt...@inria.fr TAO, LRI, UMR 8623(CNRS - Universite Paris-Sud), bat 490 Universite Paris-Sud F-91405 Orsay Cedex France http://0z.fr/EJm0g (one of the 56.5 % of french who did not vote for Sarkozy in 2007) ___ Computer-go mailing list Computer-go@dvandva.org http://dvandva.org/cgi-bin/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@dvandva.org http://dvandva.org/cgi-bin/mailman/listinfo/computer-go -- = Olivier Teytaud -- olivier.teyt...@inria.fr TAO, LRI, UMR 8623(CNRS - Universite Paris-Sud), bat 490 Universite Paris-Sud F-91405 Orsay Cedex France http://0z.fr/EJm0g (one of the 56.5 % of french who did not vote for Sarkozy in 2007) ___ Computer-go mailing list Computer-go@dvandva.org http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
