One thing we did to speed the process of building everything with each change was to put Ivy cache, source checkouts, and build tooling all on a large (12G) ram disk.
Absolutely huge difference. We're running a good 1,000+ builds a day (of this type alone). There are other approaches and patterns to apply if performance is still an issue, but that's a topic for a conference presentation. If I ever get off my bum and submit ;-) ~Tim On Jan 20, 2010, at 1:46 PM, Shawn Castrianni <shawn.castria...@halliburton.com > wrote: > I agree. We have 80 modules in our dependency hierarchy and used > QuickBuild2 to do continuous on each of the modules. But as Tim > pointed out, we got into compatibility issues when IVY detected > conflicts and picked one as the winner. We have since gone to doing > 4 full 80 module builds in dependency order 4 times per 24 hour > period. We still use QuickBuild2 because I believe it is the most > flexible and powerful out of the box and their plugin API, due out > later this month, will only make it better. > > --- > Shawn Castrianni > > -----Original Message----- > From: Tim Brown [mailto:tpbr...@gmail.com] > Sent: Wednesday, January 20, 2010 8:42 AM > To: ivy-user@ant.apache.org > Cc: timothy.as...@ca.ibm.com > Subject: Re: Building a module only when changed > > +1 to that. > > We use Cruise to trigger builds based upon SCM changes (amongst other > things). > > However, I _have_ setup builds where it determined if a particular > module > has been built since it's last change. This was using Subversion, > so excuse > the svn commands ;-) > > > - svn info the module directory, extract out the last changed > revision > - ivy findrevision that module, that revision into your repo > - if it doesn't exist, build it. > > That said, I don't recommend going with this approach. In my > experience, > the gains of not just rebuilding everything outweigh the expected > speed > improvement. You start finding cases where X hasn't changed, but > depends on > Y. Y changes, and is no longer compatible with X. > > You get into the impact-zone approach of testing, which adds to the > overhead > of determining if you need to build, etc. > > My $0.02. > > ~Tim > > > On Wed, Jan 20, 2010 at 6:33 AM, Gareth Western <gar...@garethwestern.com > >wrote: > >> This sounds more like an issue for your build system rather than the >> software. I think what you're looking for is a "Continuous >> Integration" server such as Hudson, CruiseControl, or Bamboo. There >> are loads of others, but those are the 3 I hear about most >> frequently. >> >> These typically monitor your SCM for changes and then invoke a build >> mechanism (Ant files, Maven, custom schell scripts, etc.), and then >> publish the resulting artifacts somewhere (upload to FTP, publish in >> Ivy, etc). >> >> Hudson: http://hudson-ci.org/ >> CruiseControl: http://cruisecontrol.sourceforge.net/ >> Bamboo: http://www.atlassian.com/software/bamboo/ >> >> Is that what you meant? >> >> On Wed, Jan 20, 2010 at 2:17 PM, Timothy Aston <timothy.as...@ca.ibm.com >> > >> wrote: >>> >>> >>> I'm wondering if anyone can advise on techniques for having a module >> build >>> only if there are changes. i.e. if the modified dates on any >>> files in a >>> particular fileset (the source files) are later than the >>> publication date >>> of the latest version of the module. I have a feeling this may >>> require >>> ant-contrib, which is OK as our build is already using that for >>> other >>> reasons. >>> >>> >>> -Tim >> > > ---------------------------------------------------------------------- > This e-mail, including any attached files, may contain confidential > and privileged information for the sole use of the intended > recipient. Any review, use, distribution, or disclosure by others > is strictly prohibited. If you are not the intended recipient (or > authorized to receive information for the intended recipient), > please contact the sender by reply e-mail and delete all copies of > this message.
