For java fib and parse_int seem to benefit substantially from changing NITER from 5 to 20. After doing that the minimum numbers are in the same ballpark as Julia. Of course that raises a question of how long does a jit get to warm up and still be considered fair.
On Sun, Mar 30, 2014 at 1:25 PM, Stefan Karpinski < [email protected]> wrote: > Nathan Tippy wrote them. I just ran the setup.sh script after installing > OpenJDK 7 on the Ubuntu machine. I'm not sure how much tweaking is > reasonable to allow for Java. For example, it's unreasonable to build a > custom NumPy that uses a better BLAS - because most people just won't do > that. It's the responsibility of each system to have good defaults. > > On Mar 30, 2014, at 3:15 PM, Jake Bolewski <[email protected]> wrote: > > How are you running the benchmarks? These types of micro-benchmarks in > Java are really difficult to get right. > > On Sunday, March 30, 2014 1:25:15 PM UTC-4, Stefan Karpinski wrote: >> >> Ok, the Java mandel benchmark code cheats by manually inlining and >> strength reducing all the operations on complex numbers. This benchmark >> needs to use a complex number type like everyone else. >> >> >> On Sun, Mar 30, 2014 at 1:18 PM, Stefan Karpinski >> <[email protected]>wrote: >> >>> Ok, this may be more interesting than I thought, but not in a good way >>> for Java. Here are some preliminary results after getting this to run on >>> our benchmark machine: >>> >>> benchmarkC Javarelative 1fib 0.070812.792345 39.434331309137132parse_int >>> 0.231028 4.10451617.766314039856642 3 mandel0.416994 0.272333 >>> 0.6530861355319262 4 quicksort0.600815 1.8509083.080662100646622 5pi_sum >>> 55.112839 55.5331281.0076259725977825 6 rand_mat_stat16.684055 63.864488 >>> 3.827875657326711 7 rand_mat_mul106.070995 614.2645555.791069981006589 >>> 8printfd >>> 27.725935 145.0293545.230819231163891 >>> >>> Those are some rough numbers for Java. It's getting clobbered by C, >>> Fortran, Julia, Go, JavaScript and sometimes even Python. Brutal. We >>> definitely need some Java pros to take a look at the code and make sure >>> it's a fair comparison. The mandel result is also suspicious because it >>> doesn't seem reasonable that Java can be beating C and Fortran by that much. >>> >>> >>> On Sun, Mar 30, 2014 at 11:43 AM, Stefan Karpinski <[email protected] >>> > wrote: >>> >>>> I merged the Java benchmarks, but couldn't get them to run: >>>> https://github.com/JuliaLang/julia/issues/6317. If anyone is a Java >>>> pro and wants to take a crack at this, that would be most appreciated. >>>> >>>> >>>> On Sun, Mar 30, 2014 at 11:10 AM, Isaiah Norton <[email protected]>wrote: >>>> >>>>> No. http://docs.julialang.org/en/latest/manual/performance-tips/ >>>>> >>>>> >>>>> On Sun, Mar 30, 2014 at 11:06 AM, Freddy Chua <[email protected]>wrote: >>>>> >>>>>> I did some simple benchmark on for loop >>>>>> >>>>>> a=0 >>>>>> for i=1:1000000000 >>>>>> a+=1 >>>>>> end >>>>>> >>>>>> The C equivalent runs way faster... does that mean julia is slow on >>>>>> loops ? >>>>>> >>>>>> On Sunday, March 30, 2014 10:06:20 PM UTC+8, Isaiah wrote: >>>>>> >>>>>>> https://github.com/JuliaLang/julia/tree/master/test/perf >>>>>>> >>>>>>> > I also wonder why no tests were done with Java.. >>>>>>> >>>>>>> There is an open PR for Java, which you could check out and try: >>>>>>> >>>>>>> https://github.com/JuliaLang/julia/pull/5260 >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Sun, Mar 30, 2014 at 9:55 AM, Freddy Chua <[email protected]>wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I wonder where can I download the source code of these benchmarks, >>>>>>>> I want to try it on my own... I also wonder why no tests were done with >>>>>>>> Java.. >>>>>>>> >>>>>>>> >>>>>>>> Fortran JuliaPythonR MatlabOctaveMathe-matica JavaScriptGo gcc >>>>>>>> 4.8.10.22.7.3 3.0.2R2012a 3.6.48.0V8 3.7.12.22 go1 fib0.260.9130.37 >>>>>>>> 411.361992.003211.81 64.462.181.03parse_int 5.031.6013.9559.401463.16 >>>>>>>> 7109.8529.542.43 4.79quicksort1.111.14 31.98524.29101.841132.0435.74 >>>>>>>> 3.511.25mandel 0.860.8514.19106.97 64.58316.956.073.49 2.36pi_sum >>>>>>>> 0.801.00 16.3315.421.29237.41 1.320.841.41rand_mat_stat 0.641.66 >>>>>>>> 13.5210.84 6.6114.984.523.28 8.12rand_mat_mul0.961.01 3.413.981.10 >>>>>>>> 3.41 1.1614.608.51 >>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>> >>> >>
