[computer-go] Some performance number
I have some preliminary performance numbers for various programming languages with reference bots, I now have reference bots in 3 languages. I also used the same data structure, and pretty much the same techniques for each language.The Vala program is almost a direct port of the java version (the language are VERY similar.) My disclaimer: I am sure the performance numbers for any given platform could be easily improved, perhaps even with different compiler options or simple changes to code. I am no expert in any of these languages. Below is the data as I have it now. - Don platform: core 2 duo e6700 vendor_id : GenuineIntel cpu family: 6 model : 15 model name: Intel(R) Core(TM)2 CPU 6700 @ 2.66GHz stepping : 6 cpu MHz : 2667.000 cache size: 4096 KB boardsize 9 komi 0.5 Starting position PROGRAM SPEED TIME MOVETOTAL NODES AVE SCORE - - crefgo 1.00 40.2 E5 111,040,853 0.523997 valago 1.38 55.70 E5 111,060,969 0.523488 jrefgo.jar 1.69 68.16 E5 111,037,354 0.524622 jrefgo 1.75 70.5 E5 111,048,045 0.523744 crefgo --- gcc version 4.2.3 options: -O3 -march=native valago --- Vala version 0.3.5 compile: valac --Xcc=-O3 --disable-assert --disable-checking --disable-non-null -o vgo valaGo.vala jrefgo.jar version: ibm java 1.6.0 compile: javac -O jrefgo gcj version 4.2.3 (gcc native code compiler) compile: gcj -O3 signature.asc Description: This is a digitally signed message part ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Some performance number
If anyone wants to TEST their program against one of the reference bots, you should get close to 50% score when playing lots of games. If you play 1000 games, you should expect your score to be within the range 0.46838 - 0.53162 in order to be within two standard deviations, the amount usually considered to be within a reasonable amount of noise. If you play 10,000 games, you need to be within 49 - 51 percent. These ranges are such that you should expect the result to be outside that range less than 5% of the time (actually, about 4.5 percent) if the programs really are equal. So far the Vala bot is well within 2 STD after almost 2000 games. It has a minor implementation defect so I am looking to see if it gets picked up. When 2 moves have equal scores, it does not choose between them randomly, but chooses the one first in the list (the highest rank first, then the lowest file.) With 1000 simulations that should make very little difference perhaps not even easily measurable. It's not even clear if it would make it play 1 or 2 ELO stronger or weaker if it does make a difference. Note: I can make a test that catches this too. - Don On Wed, 2008-10-22 at 14:10 -0400, Don Dailey wrote: I have some preliminary performance numbers for various programming languages with reference bots, I now have reference bots in 3 languages. I also used the same data structure, and pretty much the same techniques for each language.The Vala program is almost a direct port of the java version (the language are VERY similar.) My disclaimer: I am sure the performance numbers for any given platform could be easily improved, perhaps even with different compiler options or simple changes to code. I am no expert in any of these languages. Below is the data as I have it now. - Don platform: core 2 duo e6700 vendor_id : GenuineIntel cpu family : 6 model : 15 model name : Intel(R) Core(TM)2 CPU 6700 @ 2.66GHz stepping: 6 cpu MHz : 2667.000 cache size : 4096 KB boardsize 9 komi 0.5 Starting position PROGRAM SPEED TIME MOVETOTAL NODES AVE SCORE - - crefgo 1.00 40.2 E5 111,040,853 0.523997 valago 1.38 55.70 E5 111,060,969 0.523488 jrefgo.jar 1.69 68.16 E5 111,037,354 0.524622 jrefgo 1.75 70.5 E5 111,048,045 0.523744 crefgo --- gcc version 4.2.3 options: -O3 -march=native valago --- Vala version 0.3.5 compile: valac --Xcc=-O3 --disable-assert --disable-checking --disable-non-null -o vgo valaGo.vala jrefgo.jar version: ibm java 1.6.0 compile: javac -O jrefgo gcj version 4.2.3 (gcc native code compiler) compile: gcj -O3 ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ signature.asc Description: This is a digitally signed message part ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Some performance number
FWIW, since I don't have exactly the same outcome yet, but my stats are Mac Pro 2.8Ghz 2 x Quad-Core Intel Xeon Memory is 4Gb 667Mhz DDR2 L2 cache per processor (of 4 cores) 12Mb Java version: Java HotSpot version 1.5.0.16 Run with java -server (javac optimization level of Eclipse is unknown to me.) Time for 1 million playouts: 42 secs. Java version: 64-bit Java HotSpot version 1.6.0_07 Time for 1 million playouts: 36 secs. Interestingly, Java 5 is faster for multiple processing, 12 secs. vs. 17 sec. for Java 6 running on 8 cores. But those numbers look way too high anyway, I used to get more like a factor 6-7 out of 8 processors. I spent some time optimizing the playouts, but I believe the UCT-tree building is still very inefficient as it takes about a third of the total time. I built it for clarity, reusabiility and scalability rather than speed. Maybe if I improve the tree-building it will be faster than crefgo on equal hardware. (That's just a wild guess of course.) Mark On 22-okt-08, at 16:10, Don Dailey wrote: I have some preliminary performance numbers for various programming languages with reference bots, I now have reference bots in 3 languages. I also used the same data structure, and pretty much the same techniques for each language.The Vala program is almost a direct port of the java version (the language are VERY similar.) My disclaimer: I am sure the performance numbers for any given platform could be easily improved, perhaps even with different compiler options or simple changes to code. I am no expert in any of these languages. Below is the data as I have it now. - Don platform: core 2 duo e6700 vendor_id : GenuineIntel cpu family: 6 model : 15 model name: Intel(R) Core(TM)2 CPU 6700 @ 2.66GHz stepping : 6 cpu MHz : 2667.000 cache size: 4096 KB boardsize 9 komi 0.5 Starting position PROGRAM SPEED TIME MOVETOTAL NODES AVE SCORE - - crefgo 1.00 40.2 E5 111,040,853 0.523997 valago 1.38 55.70 E5 111,060,969 0.523488 jrefgo.jar 1.69 68.16 E5 111,037,354 0.524622 jrefgo 1.75 70.5 E5 111,048,045 0.523744 crefgo --- gcc version 4.2.3 options: -O3 -march=native valago --- Vala version 0.3.5 compile: valac --Xcc=-O3 --disable-assert --disable-checking --disable-non-null -o vgo valaGo.vala jrefgo.jar version: ibm java 1.6.0 compile: javac -O jrefgo gcj version 4.2.3 (gcc native code compiler) compile: gcj -O3 ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
