On Wed, Dec 2, 2009 at 3:51 AM, Vladimir Sizikov <[email protected]> wrote: > Hi Charlie, > > It is great seeing you looking into this startup issue. :) I've been > doing that a lot during the last couple of weeks, and so far, it seems > that the real bottleneck is in parsing/loading of require'd files. > > LoadService is inefficient, yeah, but it never played significant role > in measurements I've taken. Yes, it does excessive filesystem calls, > but they are not that bad, especially on subsequent invocations, when > entire filesystem is in the memory already. > > What is killing us is the parsing. Especially cold parsing, where we > are easily 10x-50x times slower. It doesn't matter whether this is > rubygems or any other rb file. Rubygems is more noticeable since it > used a lot and it is *huge* (which should be a task on its own to > reduce that beast to something more simple/manageable). > > But even simple require of any other rb file at startup is slow, and > takes at least 100ms or more. While MRI takes a couple of ms. No > wonder that Rubygems were never optimized to load minimally, since > loading in MRI is very cheap, if noticeable. For us, it is clearly > noticeable, esp. cold load. > > But even hot loads are not very good in JRuby, I've created a small > load benchmark that does 1000 loads for a rather big rb file (similar > to fileutils.rb). And JRuby is 3x slower on it. That's on Windows, > where MRI is not that fast at all. On Linux, JRuby's load perf esp. > bad compared to much master MRI.
I was running this yesterday and on my machine we are about 2x slower if you load 3000 loads, so warmup does not do a bad job, but other than rails it is difficult to make it to 3k requires. I do have two optimizations which will make a little difference (but both will end up being less than 10% speedup): 1. Comment parsing can by-pass lexer read() for direct inputstream read(). This will eliminate quite a bit of logic. 2. Indentifier parsng can do a similiar thing. I have some other more minor optimizations like removing the SCRIPT_LINES capture conditional from the hotpath (in theory it should get optimized out over time, but cold perf is our problem...so let's do it by hand!). > I've filed a bug report for that: http://jira.codehaus.org/browse/JRUBY-4263 > > Another possibility I was hoping to get some good results was to use > pre-compiled rb files into classes. But that turned out even worse > performance (10x worse or so): > > http://jira.codehaus.org/browse/JRUBY-4273 We used to have serialized AST files and they were about 5-10% faster than our old parser (which was 2-3 times slower than our current parser when warm). Serialization could be a good idea, but from some experimentation it does not make a big difference. > And in these benchmarks we don't even encounter any LoadService > issues, since the files are loaded directly, with no search. > > In short, if our parsing is not improved somehow, we'll always pay big > for loading rb files, and especially at startup. Rubygems is even not > *that* bad compared to rails, where there is just scary amount of > various rb loading. Just take a look: > > http://gist.github.com/247086 > > Also, I saw your bug about setAccessible() calls, was hoping that it > would speed things up somewhat, but in normal case, on PC, there is no > measurable difference. This particular issue should be fixed even if performance does not show up. Semantically a method which you are incapable of accessing due to security should still be there and fail on calling with an error to indicate actual problem. Just not adding it when you cannot setAccessible to it is not right. If it makes some platforms faster then that is a bonus. -Tom -- blog: http://blog.enebo.com twitter: tom_enebo mail: [email protected] --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email
