On Tue, Mar 26, 2013 at 03:18:03PM +0100, Michal Fojtik wrote: > Hi everyone, > > First. Sorry for this long email :-) > > I started working on refactoring of how we handle gems loading in > Deltacloud API. This is what I discover: > > 1. In 'server', we don't use Bundler. If you run Deltacloud using: > > './bin/deltacloudd' > > we use the old-school 'require' all over the DC server code (do a > "grep 'require ' -r lib/"). Having this, we always use the version > of the gem which is currently available in the system. In most cases > is the latest version available. > > 2. For 'server' tests, we do use Bundler (we call the > Bundler.require in the Rakefile). So before you run 'rake test' you > need to run 'bundle install'. This is not true, when the user want > to execute just one single test file (ruby tests/.../test.rb) (yeah, > he can use 'bundle ruby ...', but srsly, how often you forget to do > it ;-). In that case, we again trust the rubygems and use the system > versions. > > I think what we currently have is a bit mess :-) I found that up to > 50% of problems, users have with DC installation is handling all > various gem dependencies and versions (tilt, sinatra, rabbit, > openstack, etc...). Also many bugs/jiras are related to breakages > caused by incompatible changes in upstream gems we use. > > So. How we can do things better? :-) > > *** Idea ***: > > Use bundler everywhere. I don't like bundler. But I think currently > it is the only sane way how to manage gems in DC. We should check in > the Gemfile.lock into 'server' dir, create an initialization file > with Bundler.require(:default) and use this to require all gems we > do need for server to operate. Also might we need to remove all > "require 'upstream_gem'" in DC, we we load the gems just once (when > starting DC). > > *** Problems ***: > > - Different drivers require different gems. Having them all loaded > by Bundler will increase DC boot time badly and increase the memory > DC consume. > > Solution: > > We can create a 'group' for each driver in Bundler, like: > > group :ec2 do > gem 'aws' > end > > group :rhevm do > gem 'rbovirt', :version => '...' > end > > The the 'driver' method in Deltacloud will need to be a bit smarter > and call the "Bundler.require(driver_symbol)" when the driver is > changed. > > - We need to make sure that 'gem install deltacloud-core' work > properly. Ruby gems does not play well with Gemfile.lock. > > Solution: > > We don't need 'gemspec', right? We do need gemspec **only** when we > are building a gem (release). Can we generate 'gemspec' from what we > have in Gemfile.lock before release? In that case we will make sure > all gems we require have the correct versions (the one we tested > on). > > ------- > > I think having all gems managed by Bundler and loaded on one place > (initializer) will make managing gems we use easier, where we can > track what exact version we are currently supporting and we can > avoid the issues similar to 'tilt' or 'sinatra'. > > There is a good article about clarification how to use gemspec vs Gemfile: > > http://yehudakatz.com/2010/12/16/clarifying-the-roles-of-the-gemspec-and-gemfile/ > > "When developing a gem, do not check your Gemfile.lock into version control" > > vs. > > "When developing an app, check in your Gemfile.lock" > > In our case, we develop an app, right? The 'gem' we produce is just > a distribution of this app. > > > What you think? Do you have a better idea?
This sounds like the right thing to do to me. When we ship Deltacloud as RPM packages, we can use the bundler-ext tool as we do in the rest of Aeolus. --H -- == Hugh Brock, hbr...@redhat.com == == Senior Engineering Manager, Cloud Engineering == == Aeolus Project: Manage virtual infrastructure across clouds. == == http://aeolusproject.org == "I know that you believe you understand what you think I said, but I’m not sure you realize that what you heard is not what I meant." --Robert McCloskey
