I also ran the petstore benchmark a couple of times today with
different setup options. The petstore app as it is in SVN does some
heavy logging, so I decided not only to test JRuby 1.0.1 but also
the latest fixes-1_0 branch (with slow IO being fixed).
The setup procedure was like Alexey described in one of his previous
postings. I tested the ar-jdbc version only.
My results:
1.0.x
controller : 5.729000 0.000000 5.729000 ( 5.729000)
view : 21.572000 0.000000 21.572000 ( 21.572000)
full action: 42.043000 0.000000 42.043000 ( 42.043000)
1.0.x -J-Djruby.jit.enabled=false
controller : 5.337000 0.000000 5.337000 ( 5.337000)
view : 19.633000 0.000000 19.633000 ( 19.633000)
full action: 38.506000 0.000000 38.506000 ( 38.506000)
TRUNK
controller : 5.427000 0.000000 5.427000 ( 5.428000)
view : 18.653000 0.000000 18.653000 ( 18.653000)
full action: 50.560000 0.000000 50.560000 ( 50.560000)
TRUNK -J-Djruby.jit.enabled=false
controller : 6.671000 0.000000 6.671000 ( 6.671000)
view : 18.512000 0.000000 18.512000 ( 18.512000)
full action: 49.191000 0.000000 49.191000 ( 49.192000)
1.0.1
controller : 8.909000 0.000000 8.909000 ( 8.909000)
view : 25.409000 0.000000 25.409000 ( 25.409000)
full action: 52.643000 0.000000 52.643000 ( 52.643000)
1.0.1 -J-Djruby.jit.enabled=false
controller : 8.618000 0.000000 8.618000 ( 8.618000)
view : 23.495000 0.000000 23.495000 ( 23.495000)
full action: 50.521000 0.000000 50.521000 ( 50.521000)
and some benchmarks using server Hotspot:
TRUNK -J-server
controller : 5.428000 0.000000 5.428000 ( 5.428000)
view : 17.659000 0.000000 17.659000 ( 17.659000)
full action: 43.340000 0.000000 43.340000 ( 43.340000)
TRUNK -J-server -J-Djruby.jit.enabled=false
controller : 5.636000 0.000000 5.636000 ( 5.636000)
view : 13.643000 0.000000 13.643000 ( 13.643000)
full action: 36.770000 0.000000 36.770000 ( 36.770000)
1.0.x -J-server
controller : 5.415000 0.000000 5.415000 ( 5.415000)
view : 18.243000 0.000000 18.243000 ( 18.243000)
full action: 38.209000 0.000000 38.209000 ( 38.209000)
1.0.x -J-server -J-Djruby.jit.enabled=false
controller : 4.578000 0.000000 4.578000 ( 4.578000)
view : 15.037000 0.000000 15.037000 ( 15.037000)
full action: 32.002000 0.000000 32.002000 ( 32.002000)
MRI:
controller : 2.010000 0.100000 2.110000 ( 2.248452)
view : 9.420000 0.730000 10.150000 ( 10.183790)
full action: 17.250000 1.120000 18.370000 ( 19.341900)
=> my (partially weird?) results:
1. 1.0.x is faster than trunk
2. 1.0.1 is slower than trunk (probably due to slow file IO)
3. most tests run faster without JIT (only controller part seems
to be faster with JIT)!!
4. fastest JRuby setup (1.0.x / nojit / -server): about 50% of MRI
performance
Any idea? Trunk faster than 1.0.x and JIT causing a slowdown??
Is there a special issue with my hardware (maybe Hotspot / CPU related)?
My setup (Ubuntu this time):
Hardware: IBM Thinkpad R40, an older one (Intel(R) Pentium(R) M processor
1300MHz, 1024MB cache, 1199.90 bogomips)
OS: Ubuntu 7.04
Kernel: 2.6.20-16-generic #2 SMP Sun Sep 23 19:50:39 UTC 2007 i686 GNU/Linux
Ruby: ruby 1.8.5 (2006-08-25) [i486-linux]
JRuby: trunk rev 4384, fixes-1_0 rev 4385
Database: MySQL 5.0.38
Application: Petstore rev 177
Alexey Verkhovsky-2 wrote:
>
> Some new numbers.
>
> SUMMARY
>
> In Petstore script/benchmark_action:
> * trunk runs two times faster than 1.0.1
> * there seems to be a significant bottleneck somewhere in executing
> the Rails framework
> * when framework processing is excluded, JRuby is about 1.5 times
> slower than MRI
> * Unlike earlier test runs, AR-JDBC in the current trunk has a
> noticeable advantage over pure-Ruby MySQL driver
>
> DETAILS
>
> Ola and yours truly are on the next iteration of JRuby performance
> tuning for Rails. I wrote a couple of scripts to drill down into
> various parts of Rails, to decide where to focus.
>
> The scripts, essentially, measure the average roundtrip time of
> app.get('url'), controller.send(action) and controller.render(), by
> 1000 iterations of each block, 10 runs.
>
> JRuby JIT compiler is warmed up by 10000 iterations of each test prior
> to measurements. The numbers reported below are from the 10th run.
> Variation between last 5 runs in all of these tests is under 5%, so
> it's doesn't really matter that it's not an average.
>
> Note that there is no web server, and no session handling involved in
> these tests.
> Also, note that the relative amount of database / model work involved
> in CategoriesController#show is rather small.
>
> Here are the numbers:
>
> Runtime MRI Jruby Jruby 1.0.1
> DB driver native pure Ruby AR-JDBC pure Ruby AR-JDBC
>
> Controller 1.1 2.0 1.7 2.5 7.19
> View 4.5 4.4 6.7 6.6 17.08
> Framework 2.7 3.5 9.3 9.5 11.8
> Full action 8.3 9.9 17.7 18.6 36.03
>
> JRuby 1.0.1 / AR-JDBC:
> controller : 1.867000 0.000000 1.867000 ( 1.867000)
>
>
> TEST SETUP AND PROCEDURE
>
> Hardware: Dell D620
> OS: Ubuntu 7.04
> Kernel: 2.6.20-16-generic #2 SMP Fri Aug 31 00:55:27 UTC 2007 i686
> GNU/Linux, default from the distro.
> Ruby: ruby 1.8.5 (2007-03-13 patchlevel 35) [i686-linux]
> JRuby: trunk rev 4384
> Database: MySQL 5.0.38 (binary from the distro)
> Application: Petstore from
> http://tw-commons.rubyforge.org/svn/petstore/trunk/ (rev 177).
> Measuring:
> Ruby: echo 'load "script/benchmark_action"' | ruby script/console
> production
> JRuby: echo 'load "script/benchmark_action"' | jruby
> -J-Djruby.objectspace.enabled=false script/console production
> # replace production with ar_jdbc for running with AR:JDBC driver
>
> --
> Alexey Verkhovsky
> CruiseControl.rb [http://cruisecontrolrb.thoughtworks.com]
> RubyWorks [http://rubyworks.thoughtworks.com]
>
> ---------------------------------------------------------------------
> To unsubscribe from this list please visit:
>
> http://xircles.codehaus.org/manage_email
>
>
>
--
View this message in context:
http://www.nabble.com/JRuby-vs-MRI---Petstore-shootout-tf4289470.html#a12884516
Sent from the JRuby - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe from this list please visit:
http://xircles.codehaus.org/manage_email
