[computer-go] 13x13 study.
The study is running very well. We have 32 computers being used so far, some participants are providing 2 (or even more) computers. It would be great to get even more as we get into higher levels, as it will take a LOT of power to get a lot of games when each games takes 2 or 3 hours or more. http://cgos.boardspace.net/study/13/index.html - Don ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] 13x13 study.
Le jeudi 21 février 2008, Don Dailey a écrit : The study is running very well. We have 32 computers being used so far, some participants are providing 2 (or even more) computers. It would be great to get even more as we get into higher levels, as it will take a LOT of power to get a lot of games when each games takes 2 or 3 hours or more. Is it possible to have a indicative time and memory consumption at each level, for 9x9 and 13x13 study ? I guess this does not vary too much on recent hardware (factor 2 at max), and it is interesting to extrapolate how the scalability is (im)possible. Thanks a lot for this interesting and convincing study. Alain ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] 13x13 study.
It would be great to get even more as we get into higher levels, as it
will take a LOT of power to get a lot of games when each games takes 2
or 3 hours or more.
Is it possible to have a indicative time and memory consumption at each
level, for 9x9 and 13x13 study ?
I guess this does not vary too much on recent hardware (factor 2 at max),
and it is interesting to extrapolate how the scalability is (im)possible.
I can give you the data for my locally running instance of the study - I
have a core 2 duo system.
One interesting thing about mogo is that even when you set it to a fixed
level, it has heuristics to stop the search early - so you see that the
following table does not indicate that each level take 2X time, instead
it is something less.
It's interesting also that mogo is much stronger than gnugo at the same
basic time control - at least 300 ELO stronger.
PLAYERTIME/GME RATING GAMES WIN% Total games: 1028
--- - ---
Mogo_13_11 645.36 2493.2 5986.44 Mogo at 131072 play-outs
Mogo_13_10 367.31 2322.5 6063.33 Mogo at 65536 play-outs
Mogo_13_09 194.76 2231.114166.67 Mogo at 32768 play-outs
Mogo_13_08 109.11 2103.812249.18 Mogo at 16384 play-outs
Mogo_13_07 79.86 2010.112551.20 Mogo at 8192 play-outs
Mogo_13_06 39.13 1823.013034.62 Mogo at 4096 play-outs
Gnugo-3.7.11137.56 1800.029768.69 Gnugo level 8
Mogo_13_05 19.80 1741.711642.24 Mogo at 2048 play-outs
Mogo_13_04 10.86 1629.126638.72 Mogo at 1024 play-outs
Mogo_13_035.29 1398.423855.46 Mogo at 512 play-outs
Mogo_13_022.90 1198.927657.97 Mogo at 256 play-outs
Mogo_13_011.69909.922612.39 Mogo at 128 play-outs
By the way, the ELO ratings are not very meaningful from this report
because they are only rough estimates for pairing purposes and they are
designed to converge very slowly.
- Don
Thanks a lot for this interesting and convincing study.
Alain
___
computer-go mailing list
computer-go@computer-go.org
http://www.computer-go.org/mailman/listinfo/computer-go/
___
computer-go mailing list
computer-go@computer-go.org
http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] 13x13 study.
If you look at the table you will notice that going from level 4 to
level 11 (which is 7 doublings and should take 128X longer) only takes
59.43 X longer.
Some possible hypothesis:
1. Computer more loaded during early games?
2. Mogo resigns earlier at longer levels.
3. Mogo's stop early heuristic works better at longer levels.
PLAYERTIME/GME RATING GAMES WIN% Total games: 1028
--- - ---
Mogo_13_11 645.36 2493.2 5986.44 Mogo at 131072 play-outs
Mogo_13_10 367.31 2322.5 6063.33 Mogo at 65536 play-outs
Mogo_13_09 194.76 2231.114166.67 Mogo at 32768 play-outs
Mogo_13_08 109.11 2103.812249.18 Mogo at 16384 play-outs
Mogo_13_07 79.86 2010.112551.20 Mogo at 8192 play-outs
Mogo_13_06 39.13 1823.013034.62 Mogo at 4096 play-outs
Gnugo-3.7.11137.56 1800.029768.69 Gnugo level 8
Mogo_13_05 19.80 1741.711642.24 Mogo at 2048 play-outs
Mogo_13_04 10.86 1629.126638.72 Mogo at 1024 play-outs
Mogo_13_035.29 1398.423855.46 Mogo at 512 play-outs
Mogo_13_022.90 1198.927657.97 Mogo at 256 play-outs
Mogo_13_011.69909.922612.39 Mogo at 128 play-outs
- Don
___
computer-go mailing list
computer-go@computer-go.org
http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] 13x13 study.
Hi Don, 2008/2/21, Don Dailey [EMAIL PROTECTED]: If you look at the table you will notice that going from level 4 to level 11 (which is 7 doublings and should take 128X longer) only takes 59.43 X longer. Mogo's stop early heuristic works better at longer levels. That is actually very interesting, and may be a new hypothesis for the scalability limits we saw in 9x9. There are two kind of stop early heuristics - a safe one, in the following case: if we began to always simulate the second best move, it would not have more simulations that the first best move at the end of the time limit. As the chosen move is the one with the maximum number of simulations, there is no point to continue thinking. - a risky one, in the following case: if the first best move have more than x% of all simulations, and the ratio first best move/second best move (in number of simulations) is more than y, and the total number of simulations is greater than expected total of simulations / 2, then we stop. There is also a hard stop early in the following case: if the first best move have more than 1-(1-x%)/2 of all simulations, and the ratio first best move/second best move (in number of simulations) is more than 2 * y, and the total number of simulations is greater than expected total of simulations / 4, then we stop Maybe x and y are not adapted to long thinking time (stop too early in a loosing move). Or maybe they are, and it worth saving time :). Anyway, it is normal that we longer thinking time, even the first heuristic arrives much more often. Sylvain ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] 13x13 study.
On Thu, Feb 21, 2008 at 04:08:56PM -0500, Don Dailey wrote: It's also interesting how the graph up to level 11 seems to form 2 very straight lines, almost as if they were connected at an angle. This must be a by-product of how we started the test. We played only the first 4 levels as we were testing the system and that is where the bend point is.Then I added levels gradually.I cannot figure out why this would cause such strange behavior. I noticed the same angle in the 9x9 study, more on Fatman, but also on Mogo. To me it still seems to be an interesting coincidence (if that what it is) that it happens about the level where a MC program (without a tree search) levels off. On 9x9, it was around 1400 elo for Fatman and 1600 for Mogo. The same seems to apply here. It would be interesting to see a similar study with pure MC programs (no tree search of any kind). And/or to get a few 'in-between' data points near the corner. Or to make the MC simulations weaker/stronger, and see how that affects the performance of the UCT programs... If I had all the machine power, and nothing better to do... -H -- Heikki Levanto In Murphy We Turst heikki (at) lsd (dot) dk ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] 13x13 study.
--- Don Dailey [EMAIL PROTECTED] wrote: It's also interesting how the graph up to level 11 seems to form 2 very straight lines, almost as if they were connected at an angle. This must be a by-product of how we started the test. We played only the first 4 levels as we were testing the system and that is where the bend point is.Then I added levels gradually. I cannot figure out why this would cause such strange behavior. Is it possible to generate a cross-table? Level x plays opponents 1,2,3,4,5... with so many games each and such-and-such winning percentage? Perhaps that would help understand the results. The upper part of the graph looks very interesting, but there aren't many games there yet. Terry McIntyre lt;[EMAIL PROTECTED]gt; Wherever is found what is called a paternal government, there is found state education. It has been discovered that the best way to insure implicit obedience is to commence tyranny in the nursery. Benjamin Disraeli, Speech in the House of Commons [June 15, 1874] Looking for last minute shopping deals? Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
