Don, Thanks for the enlightening post. jconsole is pretty nifty! I ran the clone script again while watching from jconsole. I don't see any obvious thrashing of any resources going on. Here's my stats from the last run.
heap size: 5k max heap size: 65k committed memory: 6k live threads: 12, peak: 14 daemon threads: 7, total started: 15 current classes loaded: 2,364 total physical memory: 4gb committed virtual memory: 1.2gb free physical memory: 1.1gb Records being cloned: 22k columns: 63. of these, roughly 15 are string columns, all 25 characters or less, 4 are date/time, 5 are floats, and the rest are integers. four indexes on the table. For whatever its worth, the same data can be cloned using Delphi compiled app on Windows XP in about 30 seconds. My primary goal is to write a wicked fast cloner in ruby that can really crank through a database. I'm hoping to scale by either pushing to many threads or multiple servers (a la Pentaho Kettle). Of course, my better option may very well be to use Kettle, but I thought I'd have a little fun with jruby before diving into Java (which isn't something I'm all that keen on doing). Michael On Fri, Apr 23, 2010 at 4:43 PM, Don Morrison <[email protected]> wrote: > I've been doing some work with JRuby and love it. I have found the > performance to be fine, but the deployment options are where the differences > really come in! > > That said, how many records are we talking here? 6 minutes seems like > quite a while, so it's a fair amount. When running your code under MRI, Ruby > will just take as many system resources as it can to get the work done, > however, under JRuby things are a little different. JRuby includes the > constraints of the JVM and my *guess* is that src_ds is a big list of files > and that may be pushing on the upper limits of the JVM heap and causing some > havoc there. I would start by checking the max heap size, if you haven't set > it explicitly you may need to dig a bit to find the default. Give your code > some headroom and things may look better. > > The last week or so I have been using jconsole to inspect my Rails > application running under JRuby(+ Tomcat). It has been eye-opening to say > the least. I see you're on a mac so just fire up 'jconsole' and point it at > the running ruby process. It will tell you all kinds of things about what is > going on including the JVM Summary where you can see the max heap size. > > Hope this helps somewhat. > Don > > On Apr 23, 2010, at 12:57 PM, Michael Lang wrote: > > > One of the things I've been hearing about is jruby well outperforming > ruby mri. > > > > Today, I got jruby + jdbc + sequel working on my mac and took things for > a twirl on connecting Sequel to MSDE 2000 via the JDBC drivers. > > > > If I use dataset.all, I see practically no difference with querying, > copying and inserting data from one database schema when comparing the two > platforms. Both take roughly 6 minutes +/- 20 secs. However, if I use > dataset.each, I see jruby taking 9 minutes and 8 seconds to do a clone while > Ruby MRI stays at 6 minutes 15 seconds, both +/- 20 secs on subsequent > benchmarks. That's about a 1:1.44 difference with Ruby MRI well > outperforming JRuby! The 20 seconds margin of error is simply me testing > during the day on production/live data where 30 or more users are actively > using the server (not ideal, I know, but good enough to see basic patterns). > > > > Hardware: Macbook Pro 2.33 GHz Intel Core 2 Duo 4GB RAM > > All installed from macports or latest gems > > > > Ruby MRI: > > ruby 1.8.7 (2009-06-12 patchlevel 174) [i686-darwin9] > > sequel 3.10.0 > > ODBC Driver: Actual Technologies - Actual ODBC Driver for SQL Server > version 2.9e > > > > JRuby: > > java version "1.5.0_22" > > Java(TM) 2 Runtime Environment, Standard Edition (build > 1.5.0_22-b03-333-9M3125) > > Java HotSpot(TM) Client VM (build 1.5.0_22-147, mixed mode, sharing) > > jruby 1.4.0 (ruby 1.8.7 patchlevel 174) (2010-04-22 6586) (Java > HotSpot(TM) Client VM 1.5.0_22) [i386-java] > > sequel 3.10.0 > > JDBC Driver: Microsoft sqljdbc_3.0.1301.101 > > > > The code to clone a table: http://gist.github.com/377053 > > > > The only thing outside the GIST above is using two DB connections (two > different servers) and making the calls to the clone_table method. > > > > Am I doing something horribly wrong in JRuby, or is this par for the > course? > > > > Michael > > -- > > http://codeconnoisseur.org > > > > -- > > You received this message because you are subscribed to the Google Groups > "sequel-talk" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > [email protected]<sequel-talk%[email protected]> > . > > For more options, visit this group at > http://groups.google.com/group/sequel-talk?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "sequel-talk" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<sequel-talk%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/sequel-talk?hl=en. > > -- http://codeconnoisseur.org -- You received this message because you are subscribed to the Google Groups "sequel-talk" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/sequel-talk?hl=en.
