Hi Charlie, It is great seeing you looking into this startup issue. :) I've been doing that a lot during the last couple of weeks, and so far, it seems that the real bottleneck is in parsing/loading of require'd files.
LoadService is inefficient, yeah, but it never played significant role in measurements I've taken. Yes, it does excessive filesystem calls, but they are not that bad, especially on subsequent invocations, when entire filesystem is in the memory already. What is killing us is the parsing. Especially cold parsing, where we are easily 10x-50x times slower. It doesn't matter whether this is rubygems or any other rb file. Rubygems is more noticeable since it used a lot and it is *huge* (which should be a task on its own to reduce that beast to something more simple/manageable). But even simple require of any other rb file at startup is slow, and takes at least 100ms or more. While MRI takes a couple of ms. No wonder that Rubygems were never optimized to load minimally, since loading in MRI is very cheap, if noticeable. For us, it is clearly noticeable, esp. cold load. But even hot loads are not very good in JRuby, I've created a small load benchmark that does 1000 loads for a rather big rb file (similar to fileutils.rb). And JRuby is 3x slower on it. That's on Windows, where MRI is not that fast at all. On Linux, JRuby's load perf esp. bad compared to much master MRI. I've filed a bug report for that: http://jira.codehaus.org/browse/JRUBY-4263 Another possibility I was hoping to get some good results was to use pre-compiled rb files into classes. But that turned out even worse performance (10x worse or so): http://jira.codehaus.org/browse/JRUBY-4273 And in these benchmarks we don't even encounter any LoadService issues, since the files are loaded directly, with no search. In short, if our parsing is not improved somehow, we'll always pay big for loading rb files, and especially at startup. Rubygems is even not *that* bad compared to rails, where there is just scary amount of various rb loading. Just take a look: http://gist.github.com/247086 Also, I saw your bug about setAccessible() calls, was hoping that it would speed things up somewhat, but in normal case, on PC, there is no measurable difference. Thanks, --Vladimir On Wed, Dec 2, 2009 at 9:44 AM, Charles Oliver Nutter <[email protected]> wrote: > Findings for startup > > * RubyGems init time is mostly a result of poor cold Ruby execution > perf; very difficult to improve. Speeding up parses can help somewhat, > but there's a limit to how much we can do to improve cold perf. > RubyGems just plain runs a lot of code on boot, and slow cold > execution perf = slow RubyGems boot > * Running an NG server does not improve Rails boot time very much, but > I have not figured out why. The strongest culprit is excessive > overhead/logic in LoadService. Reducing the number of suffixes or load > path entries searched seems to help, but we still never get close to > MRI. Given that our interpreter and core classes *should* be faster > than MRI after warmup, there's something not warm-uppable causing perf > to stay poor. > * Setting a higher -Xms seems to improve perf by 10-15% on require > 'config/environment.rb', reducing the number of incremental and full > GCs by half. The default size, when used for this loading process, > appears to grow several times before the booting is complete. > * RubyGems activation procedure for gems is to add all their loadable > paths to $LOAD_PATH. This causes load path to grow to include several > dozen entries when running Rails. As a result, the already inefficient > searches in LoadService are multiplied by the number of activated > gems, which with Rails can be a very high number. Add to this the fact > that we search .class, .rb, and .jar for every failing entry, and at > least .class and .rb for even successful entries, and it all adds up > very quickly. > * Failed requires of a file (not counting classloader searches) > constructs at least two java.io.File and two JRubyFile objects. File > objects normalize the paths on construction as well as query the > platform-specific FileSystem object for the file's prefix (for > absolute path logic). JRubyFile itself constructs at minimum one more > JavaSecuredFile and at maximum two more, all of which are java.io.File > descendants. Ultimately for a failed require we construct and > normalize from 6 to 8 File-related objects, and call isAbsolute at > least twice. Successful requires construct almost as many File > objects. > * LoadService is grossly inefficient, repeatedly constructing filename > + ext, repeatedly checking that name for absolutism and more. In order > to clean it up, it needs a major overhaul/redesign. > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > > --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email
