Ah, ouch. I'll take a look at it later. I saw it when I created the original Zlib too. It's probably something small, though.
/O At 09:36 2006-06-16, you wrote: >Also FYI, with or without my NIO fix, I'm getting the following error >while installing RubyGems: > > Successfully built RubyGem > Name: sources > Version: 0.0.1 > File: sources-0.0.1.gem >hook /home/headius/rubygems- 0.8.11/./post-install.rb failed: >java.io.EOFException: Unexpected end of ZLIB input stream > >Perhaps something small? > >On 6/16/06, Charles O Nutter ><<mailto:[EMAIL PROTECTED]>[EMAIL PROTECTED]> wrote: >FYI, I went ahead and committed the plaincharset.jar file, since the patch >seems to have mangled it. > > >On 6/16/06, Charles O Nutter < ><mailto:[EMAIL PROTECTED]>[EMAIL PROTECTED]> wrote: >So we all know our friend Ola has contributed some great pieces of work >over the past several months. As another JRubyist put it, "Ola is a f'n >machine". StringIO, YAML, and Zlib are now mostly Java and far, far faster >than they were. These contributions have made RubyGems far, far faster. > From the original install time on this Opteron 2.6G of almost an hour >(sans rdoc, mind you), we get this: > >real 5m35.341s >user 0m36.379s >sys 0m5.195s > >Impressive, by any measure. Ok, so I don't remember whether it was exactly >an hour to install before, but it was bloody long enough to hit the >drive-in for lunch. Grotesquely slow. Ola's changes alone dropped that >time down to comparatively nothing. I went through the pains this evening >of applying all his patches, cleaning up conflicts between them (since I >didn't have a complete picture), and tidying up a few minor things. Then I >ran a full Rails install. The time was impressive, no doubt about it. >However, something really, really bothered me. In the time between gem >download and gem install, the machine was sitting there, doing *nothing*. >No network traffic, 0% CPU utilization. Zippo. > >Then it hit me. > >You've all heard the story about the timeout library. net/http calls >net/protocol which reads in 1024-byte chunks from the stream with a >timeout thread for each one. I cried about this on my blog too, since it >seemed to be the source of a terrible performance problem. It turns out >that once again, Java has already optimized our troubles away, in this >case the cost of spinning up hundreds of threads. > >Our NIO-based IO handler had a tiny flaw when it came to Socket IO: it was >blocking. That meant that for the last read from the stream, the last few >bytes of a gem file, it would sit and wait to fill up net/protocol's >1024-byte buffer. I am no NIO expert, but channels appear to behave >somewhat differently when doing File versus Socket IO. At the end of a >File, the stream is empty and read(ByteBuffer) will immediately return -1. >With a Socket, however, if there are not enough bytes to fill the buffer >and the socket is still open, it will block. Such is the case here. The >HTTP get of a gem seems to leave the channel open. I'm not sure if this is >intentional in RubyGems (for retrieving multiple gems) or a bug >(potentially in JRuby) but leaving the Socket in blocking mode resulted in >that last chunk of bytes waiting until timeout kicked in. > >My fix (in the form of a patch to IOHandlerNIO) may or may not be correct; >like I said, I'm no NIO expert. However, it seems to work well enough: > >Attempting local installation of 'rails' >Local gem file not found: rails*.gem >Attempting remote installation of 'rails' >Updating Gem source index for: ><http://gems.rubyforge.org>http://gems.rubyforge.org >Successfully installed rails-1.1.2 >Successfully installed rake-0.7.1 >Successfully installed activesupport-1.3.1 >Successfully installed activerecord-1.14.2 >Successfully installed actionpack-1.12.1 >Successfully installed actionmailer-1.2.1 >Successfully installed actionwebservice-1.1.2 > >real 1m3.751s >user 0m53.817s >sys 0m5.722s > >For sake of comparison, Ruby installs Rails and its dependencies in about >25 seconds on my system. It's probably safe to say that we're only 2x >slower for non-rdoc gem installations, which seems pretty good to me. > >The NIO patch alone is attached, as is a complete patch with everything >changed on my system, including: > >- All Ola's stuff (stringio, zlib, and signal) >- My unrelated simplifications to Thread.critical= (not as deterministic >as before but less likely or unlikely to deadlock...needs a bit more testing) >- My NIO fix > >It will be nice to say that not only have we gotten RubyGems working, but >it's been sped up by perhaps an order of magnitude since we first started. > >-- >Charles Oliver Nutter @ <http://headius.blogspot.com>headius.blogspot.com >JRuby Developer @ <http://jruby.sourceforge.net>jruby.sourceforge.net >Application Architect @ <http://www.ventera.com>www.ventera.com > > > > >-- >Charles Oliver Nutter @ <http://headius.blogspot.com>headius.blogspot.com >JRuby Developer @ <http://jruby.sourceforge.net>jruby.sourceforge.net >Application Architect @ <http://www.ventera.com>www.ventera.com > > > > >-- >Charles Oliver Nutter @ <http://headius.blogspot.com>headius.blogspot.com >JRuby Developer @ <http://jruby.sourceforge.net>jruby.sourceforge.net >Application Architect @ <http://www.ventera.com>www.ventera.com >_______________________________________________ >Jruby-devel mailing list >Jruby-devel@lists.sourceforge.net >https://lists.sourceforge.net/lists/listinfo/jruby-devel _______________________________________________ Jruby-devel mailing list Jruby-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jruby-devel
