Re: [computer-go] Scalability study request
Don Dailey wrote: Gian-Carlo, We could probably add this new version to the mix and extend the study.But what kind of data has your own testing produced? Do you have an indication that it is roughly as strong at the same basic time setting (because of it's being 3X faster or so?) It is definitely weaker at the same basic time setting, otherwhise there would have been no point to move to heavier playouts. I will run a test at 16k playouts to see the ELO difference. Even if it isn't, it would still be interesting to see if the line is parallel.It might indicate, for instance, that some simplified hardware implementation of play-outs could be competitive. Yes, exactly. It also means that much higher levels are probably more easily reached via playout improvements than via search improvements or hardware improvements. -- GCP ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Scalability study request
Doesn't the total number of playout simply relates to the search ply depth? DL -Original Message- From: Gian-Carlo Pascutto [EMAIL PROTECTED] To: computer-go computer-go@computer-go.org Sent: Fri, 11 Apr 2008 10:01 am Subject: [computer-go] Scalability study request Hi all,? ? the result of the scalability study at? ? http://cgos.boardspace.net/study/13/index.html? ? seems to look a lot like 2 parallel lines over the entire range, which I find very surprising, since I'd have expected at least some differences caused by different playout strategies.? ? I am now wondering if scalability could be unaffected by playouts (just adding a constant offset) and only depend on the UCT/search implementation. From the publications of the MoGo team it seems likely that the programs are very similar there.? ? I am able to provide a Leela version that is identical to the one currently used in the study, but with light (uniform random) playouts (and being about 3 times as fast).? ? I think it would be very interesting to see the behaviour of this. IMHO it would provide deeper understanding in how the combination of MC/UCT works. The critical question is if we would get another parallel line, indicating that the search is the key to further progress, or if we would get a line with a different steepness, indicating playouts are the key. Or maybe the two are more strongly linked and both contribute. Either way I believe more data would be usefull.? ? -- GCP? ___? computer-go mailing list? [EMAIL PROTECTED] http://www.computer-go.org/mailman/listinfo/computer-go/? ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Scalability study request
[EMAIL PROTECTED] wrote: Doesn't the total number of playout simply relates to the search ply depth? I have no idea what you mean or what the relevance is in the discussion. -- GCP ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Scalability study request
[EMAIL PROTECTED] wrote: Doesn't the total number of playout simply relates to the search ply depth? There is a relation, but it's not simple. The more playouts you do, the more lines you will explore to deeper depths. My suspicious is that with lighter playouts, you will tend to explore more lines less deep. - Don DL -Original Message- From: Gian-Carlo Pascutto [EMAIL PROTECTED] To: computer-go computer-go@computer-go.org Sent: Fri, 11 Apr 2008 10:01 am Subject: [computer-go] Scalability study request Hi all,? ? the result of the scalability study at? ? http://cgos.boardspace.net/study/13/index.html? ? seems to look a lot like 2 parallel lines over the entire range, which I find very surprising, since I'd have expected at least some differences caused by different playout strategies.? ? I am now wondering if scalability could be unaffected by playouts (just adding a constant offset) and only depend on the UCT/search implementation. From the publications of the MoGo team it seems likely that the programs are very similar there.? ? I am able to provide a Leela version that is identical to the one currently used in the study, but with light (uniform random) playouts (and being about 3 times as fast).? ? I think it would be very interesting to see the behaviour of this. IMHO it would provide deeper understanding in how the combination of MC/UCT works. The critical question is if we would get another parallel line, indicating that the search is the key to further progress, or if we would get a line with a different steepness, indicating playouts are the key. Or maybe the two are more strongly linked and both contribute. Either way I believe more data would be usefull.? ? -- GCP? ___? computer-go mailing list? [EMAIL PROTECTED] http://www.computer-go.org/mailman/listinfo/computer-go/? ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Scalability study request
I am now wondering if scalability could be unaffected by playouts (just adding a constant offset) and only depend on the UCT/search implementation. From the publications of the MoGo team it seems likely that the programs are very similar there. Leela and mogo are probably quite similar. On the other hand, we know that some playouts have different scalings than others: - the nakade patch on the playouts does not change anything for small numbers of simulations per move - it is very efficient for high number of simulations per move. (By the way, this implies that optimizing parameters on small numbers of simulations is perhaps not always a good idea.) ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Scalability study request
If we start up another scalability test, I'd be delighted to offer up a few computer cores. It would be real nice to not only have the light-playout variant of leela, but perhaps the nakade-patch version of mogo and maybe even some third program. ( if wishes were horses ... ) Terry McIntyre lt;[EMAIL PROTECTED]gt; Wherever is found what is called a paternal government, there is found state education. It has been discovered that the best way to insure implicit obedience is to commence tyranny in the nursery. Benjamin Disraeli, Speech in the House of Commons [June 15, 1874] __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Scalability study request
Olivier Teytaud wrote: I am now wondering if scalability could be unaffected by playouts (just adding a constant offset) and only depend on the UCT/search implementation. From the publications of the MoGo team it seems likely that the programs are very similar there. Leela and mogo are probably quite similar. On the other hand, we know that some playouts have different scalings than others: - the nakade patch on the playouts does not change anything for small numbers of simulations per move - it is very efficient for high number of simulations per move. (By the way, this implies that optimizing parameters on small numbers of simulations is perhaps not always a good idea.) Is the patch in some way parameterized by the number of simulations? -- GCP ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Scalability study request
simulations is perhaps not always a good idea.) Is the patch in some way parameterized by the number of simulations? No. Perhaps it should :-) ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Scalability study request
--- Olivier Teytaud [EMAIL PROTECTED] wrote: light-playout variant of leela, but perhaps the nakade-patch version of mogo and maybe even some third no problem for the nakade-patch version of mogo, but results are only known in 9x9, no idea for 13x13. Maybe it is better, maybe it is worse :-) Good to find out, no? Terry McIntyre lt;[EMAIL PROTECTED]gt; Wherever is found what is called a paternal government, there is found state education. It has been discovered that the best way to insure implicit obedience is to commence tyranny in the nursery. Benjamin Disraeli, Speech in the House of Commons [June 15, 1874] __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Scalability study request
light-playout variant of leela, but perhaps the nakade-patch version of mogo and maybe even some third no problem for the nakade-patch version of mogo, but results are only known in 9x9, no idea for 13x13. Maybe it is better, maybe it is worse :-) ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Scalability study request
Good to find out, no? we have validated that: - it is good in 9x9; - it is bad in 19x19 (unless perhaps for very large number of simulations). we did not have a look at 13x13. ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Scalability study request
Olivier Teytaud wrote: light-playout variant of leela, but perhaps the nakade-patch version of mogo and maybe even some third no problem for the nakade-patch version of mogo, but results are only known in 9x9, no idea for 13x13. Maybe it is better, maybe it is worse :-) At 9x9 you see a diminishing return in the previous study already, perhaps because you search so deep without the UCT exploration term. So in this case I'm not surprised that improving the playouts changes the steepness. It's not so clear to me what happens in the area where the search itself is still scaling so well (as is the case in the 13x13 study area). My playouts should be quite different from yours, so the parallel lines in the study were very surprising to me. -- GCP ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Scalability study request
If we can get some consensus on what to test, we can do more.Or we can add 1 program version to the current study. Any ideas?(Or we could do a 19x19 study.) - Don terry mcintyre wrote: If we start up another scalability test, I'd be delighted to offer up a few computer cores. It would be real nice to not only have the light-playout variant of leela, but perhaps the nakade-patch version of mogo and maybe even some third program. ( if wishes were horses ... ) Terry McIntyre lt;[EMAIL PROTECTED]gt; “Wherever is found what is called a paternal government, there is found state education. It has been discovered that the best way to insure implicit obedience is to commence tyranny in the nursery.” Benjamin Disraeli, Speech in the House of Commons [June 15, 1874] __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Scalability study request
Gian-Carlo, We could probably add this new version to the mix and extend the study.But what kind of data has your own testing produced? Do you have an indication that it is roughly as strong at the same basic time setting (because of it's being 3X faster or so?) Even if it isn't, it would still be interesting to see if the line is parallel.It might indicate, for instance, that some simplified hardware implementation of play-outs could be competitive. - Don Gian-Carlo Pascutto wrote: Hi all, the result of the scalability study at http://cgos.boardspace.net/study/13/index.html seems to look a lot like 2 parallel lines over the entire range, which I find very surprising, since I'd have expected at least some differences caused by different playout strategies. I am now wondering if scalability could be unaffected by playouts (just adding a constant offset) and only depend on the UCT/search implementation. From the publications of the MoGo team it seems likely that the programs are very similar there. I am able to provide a Leela version that is identical to the one currently used in the study, but with light (uniform random) playouts (and being about 3 times as fast). I think it would be very interesting to see the behaviour of this. IMHO it would provide deeper understanding in how the combination of MC/UCT works. The critical question is if we would get another parallel line, indicating that the search is the key to further progress, or if we would get a line with a different steepness, indicating playouts are the key. Or maybe the two are more strongly linked and both contribute. Either way I believe more data would be usefull. ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
