On Aug 29, 2007, at 08:10, Nick Sieger wrote: > The gem index yaml file is starting to get pretty big. Folks are > reporting Ruby's memory size gets in the range of 128-150MB when > loading up the index. Does anyone else think this is starting to get > problematic? > > It's especially a problem for JRuby since we need to specify the > memory size of the Java VM up front, so as the gem index grows, > eventually JRuby gem install fails with OutOfMemoryErrors. It seems > wasteful to us to simply hike the max memory of the VM up to 512MB > (currently the max is 256, but gem install is starting to err > consistently for some even at that level). > > Would something like a two-tier index work, with the main yaml/yaml.Z > just containing a list of all available gems/versions with pointers to > individual gemspec files available on the web server,
You just described the "quick index" perfectly. There's a problem with the index-building script on RubyForge that has been fixed in trunk but not deployed yet that causes bulk updates (yaml.Z/yaml) when they aren't necessary. > or could gem even get by w/o the gemspec before a gem is actually > downloaded and > installed? The index is needed to resolve versions and platforms across dependencies. > Or is the gem index not something to be mucked with at this stage? I think the fix to the indexer script will solve the problem unless the source index is extremely out of date. I get annoying bulk index updates when I'm doing things at just the right times of day. Still, we're open to improvements. Previously Ryan Davis proposed an index of just the latest gems, since 90% of the time that's all people want. I don't think it was fully implemented. -- Poor workers blame their tools. Good workers build better tools. The best workers get their tools to do the work for them. -- Syndicate Wars _______________________________________________ Rubygems-developers mailing list [email protected] http://rubyforge.org/mailman/listinfo/rubygems-developers
