Hi all, Before going to the Libre Software Meeting and LinuxTag <http://www.libresoftwaremeeting.org/> <http://www.linuxtag.org/2003/en/index.html> I wanted to know how the different free VMs were doing. So I did some micro benchmarks against them. I didn't have time to do more complete benchmarks and I couldn't get Orp and JRVM working (which is a shame since those are claimed to be the fasted VMs there are). So maybe someone with more time (or access to working Orp/JRVM VMs) can do some bigger tests.
Orp didn't compile since it has some hardcoded values for glibc 2.1 and glibc 2.2, but I am using glibc 2.3. And it needs several fixes to get it to compile with gcc 3.x. JRVM did compile, but needs a specific version of a proprietary VM to create the boot image. The resulting JikesRVM binary and image keep seg faulting however on my machine. The machine that ran the test had a Athlon 1600+ (1.4Ghz) and 768MB. It was lightly used and I don't claim to have done scientific benchmarking. All times were measured with /usr/bin/time -p and the best of three runs is given. (Note that some tests measure elapsed time themselves so the time -p values are only for reference.) The benchmarks were mentioned earlier on shudo.net. If someone has more suggestions please let me know. VMs used: kissme 0.0.30 (Jun 24 2003) SableVM version 1.0.8 kaffe 1.1.x-cvs (checkout today) ikvm (Latest ikvmbin snapshot plus Mono JIT compiler version 0.25) gij/gcj 3.3.1 20030626 (Debian prerelease) (For gcj tests were compiled from source using just -O2) Looking at the results (see below) there are clearly three groups: the plain interpreters (kissme, gij and sablevm), the jitters (kaffe and ikvm) and the ahead of time compiler (gcj). The jitters are clearly much faster then the interpreters, and the ahead of time compiler is clearly much faster then the jitters. Interesting is that ikvm has results comparable with kaffe, but often takes more time "preparing" the classes. This can be clearly seen with the Linpack benchmark where the ikvm score and kaffe score are almost identical, but when looking at the real time spend you see that ikvm consumes much more processor cycles. SableVM recently got a "portable" jit compiler, it would be interesting to see how it matches up to kaffe and ikvm. Since gcj compiled code seems to be very fast but the gij interpreter is not very fast it might be interesting to combine gcj with one of the jitters or make one of the jitters be able to use precompiled binaries produced by gcj (for example to precompile all core classes used by the VM). Let me know if you have other benchmark/speed test stories. Cheers, Mark The results: http://math.nist.gov/scimark2/ Kissme: Composite Score: 3.0156149099413163E174 FFT (1024): 1.5078077289324666E175 SOR (100x100): -1.1107352312578372E61 Monte Carlo : 1.6082285944663933E129 Sparse matmult (N=1000, nz=5000): 2.6108808678857117E25 LU (100x100): 3.5030103117925084 real 26.99 user 26.82 sys 0.05 gij: Composite Score: 5.145989276632694 FFT (1024): 2.648052903699278 SOR (100x100): 8.881613884448855 Monte Carlo : 1.387464107626074 Sparse matmult (N=1000, nz=5000): 5.551639995055836 LU (100x100): 7.261175492333425 real 31.30 user 31.19 sys 0.10 SableVM: Composite Score: 7.329552111085732 FFT (1024): 4.404292204203279 SOR (100x100): 13.062330882926906 Monte Carlo : 2.369663222573336 Sparse matmult (N=1000, nz=5000): 7.329992764243664 LU (100x100): 9.481481481481481 real 28.23 user 26.95 sys 0.11 Kaffe: Composite Score: 60.46979156068268 FFT (1024): 30.891199299334463 SOR (100x100): 148.2273083076923 Monte Carlo : 12.628691263262526 Sparse matmult (N=1000, nz=5000): 54.10832237609153 LU (100x100): 56.49343655703252 real 31.17 user 31.10 sys 0.00 ikvm: Composite Score: 100.9979186697824 FFT (1024): 59.79980402491375 SOR (100x100): 224.2731618120265 Monte Carlo : 34.02223834505183 Sparse matmult (N=1000, nz=5000): 83.7413726183124 LU (100x100): 103.1530165486078 real 35.60 user 35.42 sys 0.09 gcj: Composite Score: 217.51089337171615 FFT (1024): 225.23768378150965 SOR (100x100): 305.57485467536026 Monte Carlo : 15.335663953302584 Sparse matmult (N=1000, nz=5000): 235.82583613959164 LU (100x100): 305.5804283088167 real 28.49 user 28.44 sys 0.06 Sieve.java from TYA <http://sax.sax.de/~adlibit/> Kissme: 634 iterations in 10.008 seconds Sieve score = 63 real 10.10 user 9.98 sys 0.02 SableVM: 2355 iterations in 10.0 seconds Sieve score = 236 real 10.08 user 10.08 sys 0.01 gij: 2692 iterations in 10.0 seconds Sieve score = 269 real 10.10 user 10.05 sys 0.05 ikvm: 40937 iterations in 10.0 seconds Sieve score = 4094 real 11.32 user 11.24 sys 0.05 Kaffe: 48611 iterations in 10.0 seconds Sieve score = 4861 real 10.11 user 10.10 sys 0.01 gcj: 91621 iterations in 10.0 seconds Sieve score = 9162 real 10.03 user 9.95 sys 0.06 http://www.netlib.org/benchmark/linpackjava/ Kissme: Mflops/s: 2.289 Time: 36.62s Norm Res: 0.0 Precision: 2.220446049250313E-16 real 40.73 user 40.62 sys 0.10 gij: Mflops/s: 6.078 Time: 13.79s Norm Res: 5.81 Precision: 2.220446049250313E-16 real 14.74 user 14.59 sys 0.14 SableVM: Mflops/s: 8.45 Time: 9.92s Norm Res: 5.81 Precision: 2.220446049250313E-16 real 10.88 user 10.86 sys 0.02 ikvm: Mflops/s: 60.312 Time: 1.39s Norm Res: 5.68 Precision: 2.220446049250313E-16.0 real 5.03 user 4.92 sys 0.06 Kaffe: Mflops/s: 61.282 Time: 1.37s Norm Res: 4.28 Precision: 2.220446049250313E-16 real 1.92 user 1.89 sys 0.03 gcj: Mflops/s: 79.163 Time: 1.06s Norm Res: 4.28 Precision: 2.220446049250313E-16 real 1.41 user 1.37 sys 0.03 _______________________________________________ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath