[random] are you using Hudson as your CI? if yes, are you using the crazy m2 project type in Hudson? if yes, are you using the advanced parallel builds option? if yes to all of the above: stop right now and switch to freestyle [/random]
2009/4/17 David Hoffer <[email protected]>: > I concur with what you are saying. I wish I had the build log to see the > reactor order...but I don't because the CI system wasn't keeping history. > So I can't say for certain the reactor order was wrong. I can sure say it > seems so because changing it manually fixed the build. > > The thing that is 'special' or the thing that seems to trigger the problem > is the following: > > 1. It happens only when we checkin code in various modules where if the > build is out-of-order it won't compile because the dependencies haven't been > updated. > 2. It only happens on the CI build. I'm not sure what to make of this > fact...developers wouldn't checkin if it didn't build on their system. > 3. Yes we always do clean [insert goal here] build on all systems including > CI. > 4. We do not purge the local repo before a build (shouldn't have to). > > IMHO, muti-module builds shouldn't even try to download module artifacts for > the build because by definition if your doing an install (or more) you are > creating these...I can't think of a case where you would want to continue > the build if anyone of these module artifacts could not be generated. > However maven is not doing this...it is perfectly happy to use one that > already exists which makes no sense to me. I.e. when I say mvn clean deploy > I want maven to generate all the module artifacts and build using ONLY the > ones just generated, I don't really care if their are prior snapshots or not > because they should never be used. > > -Dave > > > On Fri, Apr 17, 2009 at 6:55 AM, Todd Thiessen <[email protected]> wrote: > >> I think it may be difficult to provide much further insite without >> actually seeing your exact project. >> >> Are you absolutely sure that the reactor order is wrong? You started off >> by saying that you didn't see the build fail again after restoring the >> module order in the parent pom to the "wrong" order. So in some cases, >> it seems to work. >> >> What is special about the cases when it doesn't? This seems to be the >> mystery. When maven starts the build, is it actually displaying the >> project order incorrectly? I personally have never seen this but I >> suspect the nature of our projects are fairly different and it isn't an >> apples to apples comparison. >> >> Do this problem happen when you do a build with a clean local >> repository? I presume your CI build always does a build this way. If it >> isn't, odd things can happen if the local repository is stale. >> >> --- >> Todd Thiessen >> >> >> > -----Original Message----- >> > From: David Hoffer [mailto:[email protected]] >> > Sent: Friday, April 17, 2009 7:40 AM >> > To: Maven Users List >> > Subject: Re: Multi-module build is not building with current >> > module source code >> > >> > I did retry the test with the module order reset back to the >> > previous order where the build failed. >> > >> > Unfortunately, I was not able to see it fail again. I >> > suspect that these commands don't duplicate the conditions >> > where folks see build failures regarding muli-module builds. >> > I apparently am not alone with this issue...hence plugins >> > seem to have been developed to work around this core bug in maven. >> > >> > Here is what I know. >> > >> > 1. The reactor (the internal auto ordering of modules) does >> > not always get it right. Just prior to me sending this email >> > we had a case where a developer checked in a file(s) that >> > caused the CI build to fail. I fixed it by simply manually >> > changing the order of the modules in the parent pom. It was >> > clear from looking at the dependencies that moduleX had to be >> > built before moduleY; maven was un-able to figure this >> > out...all I did was manually reverse these and it worked. >> > Because of this I assumed that Maven had NO auto calculation >> > of module order. >> > >> > 2. Even when the module order is made perfect manually, maven >> > will still fail the build on occasion. Although it has built >> > moduleX first, it does not use this as the dependency for >> > moduleY; instead it uses a prior snapshot from the repo! Now >> > I cannot say for certain if the old snapshot was already in >> > the local repo or if maven pulled it down from the corporate >> > repo but that is what it used in the build. To fix this >> > problem I had to manually deploy the new version of this >> > dependent artifact...then the build could find it and build correctly. >> > >> > I wish I had the build log from issue 2, but it seems our CI >> > build is not keeping history, I will see if I can turn this on. >> > >> > What I wondering about is how does maven handle snapshot >> > versioning? It seems this second issue could be caused if >> > there are errors in how maven calculates snapshot timestamps >> > or something. For instance, if the previous snapshot >> > deployed by the CI server has a later timestamp than what is >> > being built currently, would maven prefer the former? (Our >> > CI is TeamCity so the actual builds can be performed on any >> > number of build agents but all publish to the same >> > Artifactory server. I'm not sure if the build sets the time >> > or if the Artifactory server does.) Now that I type this, I >> > think I will ask our CI guy to make sure the clocks on all >> > build systems are identical. I suspect this is a long shot however. >> > >> > At this point I don't know what to do, this seems a serious >> > flaw in maven. >> > Any insight is greatly appreciated. >> > >> > -Dave >> > >> > >> > On Thu, Apr 16, 2009 at 1:17 PM, Stephen Connolly < >> > [email protected]> wrote: >> > >> > > I'd retry the test with the module order reset back to the >> > way you had it. >> > > >> > > the commands I provided should be a fully clean build from >> > scratch, so >> > > should be equivalent to changing code >> > > >> > > -Stephen >> > > >> > > 2009/4/16 David Hoffer <[email protected]>: >> > > > Thanks that makes sense. >> > > > >> > > > I did a test using your commands after blocking my snapshot proxy >> > > > and it built just fine. >> > > > >> > > > This doesn't surprise me too much because I manually changed the >> > > > order of the modules in the parent pom based on my >> > premise that the >> > > > reactor >> > > ordering >> > > > logic is flawed. >> > > > >> > > > In addition i can't say that it works yet because I didn't change >> > > > any >> > > code. >> > > > I will have to try this again when we have added breaking code to >> > > dependent >> > > > module(s). >> > > > >> > > > This is a nasty issue! >> > > > >> > > > -Dave >> > > > >> > > > On Thu, Apr 16, 2009 at 12:02 PM, Stephen Connolly < >> > > > [email protected]> wrote: >> > > > >> > > >> 2009/4/16 David Hoffer <[email protected]>: >> > > >> > I have a few comments/questions. >> > > >> > >> > > >> > I do have the snapshot artifacts on a remote corporate >> > repo. (I >> > > >> > will >> > > see >> > > >> if >> > > >> > I can block this for this test.) >> > > >> >> > > >> If you have the artifacts on a remote repository, then when the >> > > >> plugin goes looking for the artifacts (from outside the >> > reactor) it >> > > >> will find them and the plugin at fault will therefore >> > not fail the build. >> > > >> >> > > >> By ensuring that the only source of the artifacts is from the >> > > >> reactor, then whatever plugin is pulling the artifacts from the >> > > >> repository will be forced to fail (thereby finding your >> > problem for >> > > >> you) >> > > >> >> > > >> > >> > > >> > What does -DreResolve do? >> > > >> >> > > >> reResolve will turn around and re-pull the dependencies from the >> > > >> remote repository unless you disable it with -DreResolve=false >> > > >> >> > > >> (reResolve is a parameter of the purge-... goal) >> > > >> >> > > >> > >> > > >> > -Dave >> > > >> > >> > > >> > On Thu, Apr 16, 2009 at 11:09 AM, Stephen Connolly < >> > > >> > [email protected]> wrote: >> > > >> > >> > > >> >> That would be my concern too. >> > > >> >> >> > > >> >> I suspect you can reproduce your build failures if you: >> > > >> >> >> > > >> >> mvn clean >> > > >> >> >> > > >> >> mvn dependency:purge-local-repository -DreResolve=false >> > > >> >> >> > > >> >> mvn install >> > > >> >> >> > > >> >> (assuming that you have not deployed any of the -SNAPSHOT >> > > >> >> artifacts >> > > to a >> > > >> >> remote repository) >> > > >> >> >> > > >> >> you should recreate the build failure and be able to identify >> > > >> >> the >> > > source >> > > >> of >> > > >> >> it better. >> > > >> >> >> > > >> >> -Stephen >> > > >> >> >> > > >> >> 2009/4/16 David Hoffer <[email protected]> >> > > >> >> >> > > >> >> > Isn't that adding yet another plugin to do what maven is >> > > >> >> > supposed >> > > to >> > > >> do >> > > >> >> out >> > > >> >> > of the box? I'm concerned about the increase in >> > complexity, >> > > >> >> > when >> > > >> >> something >> > > >> >> > doesn't work is it maven or the incremental plugin? >> > > >> >> > >> > > >> >> > -Dave >> > > >> >> > >> > > >> >> > On Thu, Apr 16, 2009 at 10:37 AM, Dmitry Skavish < >> > > [email protected]> >> > > >> >> > wrote: >> > > >> >> > >> > > >> >> > > I asked the same question on OSGi maillist and >> > they advised >> > > >> >> > > me to >> > > >> use >> > > >> >> > > incremental-build plugin: >> > > >> >> https://maven-incremental-build.dev.java.net/I >> > > >> >> > > does exactly what I need, check it out, it could >> > solve your >> > > problem >> > > >> as >> > > >> >> > > well. >> > > >> >> > > >> > > >> >> > > On Thu, Apr 16, 2009 at 12:33 PM, David Hoffer < >> > > [email protected]> >> > > >> >> > wrote: >> > > >> >> > > >> > > >> >> > > > Then there is a big bug here because I have a >> > multi-module >> > > project >> > > >> >> with >> > > >> >> > a >> > > >> >> > > > few modules, the dependent one was built first >> > (as seen in >> > > >> >> > > > the >> > > >> build >> > > >> >> > log) >> > > >> >> > > > but yet when the depending module was built it >> > did NOT use >> > > >> >> > > > the >> > > >> >> > dependent >> > > >> >> > > > build rather it went to the repo and downloaded the >> > > >> >> > > > previously >> > > >> >> deployed >> > > >> >> > > > artifact snapshot. >> > > >> >> > > > >> > > >> >> > > > -Dave >> > > >> >> > > > >> > > >> >> > > > On Thu, Apr 16, 2009 at 10:24 AM, Todd Thiessen < >> > > >> [email protected] >> > > >> >> > >> > > >> >> > > > wrote: >> > > >> >> > > > >> > > >> >> > > > > FYI. Here is one reference, >> > > >> >> > > > > >> > > >> >> > > > > http://maven.apache.org/pom.html#Aggregation >> > > >> >> > > > > >> > > >> >> > > > > --- >> > > >> >> > > > > Todd Thiessen >> > > >> >> > > > > >> > > >> >> > > > > >> > > >> >> > > > > > -----Original Message----- >> > > >> >> > > > > > From: David Hoffer [mailto:[email protected]] >> > > >> >> > > > > > Sent: Thursday, April 16, 2009 11:46 AM >> > > >> >> > > > > > To: Maven Users List >> > > >> >> > > > > > Subject: Re: Multi-module build is not building with >> > > current >> > > >> >> > > > > > module source code >> > > >> >> > > > > > >> > > >> >> > > > > > Then I'm understanding the order of the >> > reactor wrong. >> > > >> >> > > > > > I assumed its top to bottom, that is...just before >> > > >> >> > > > > > internal >> > > is >> > > >> >> > > > > > built...public is built; and just before >> > > >> >> > > > > > security-public is built...internal is built. >> > > >> >> > > > > > >> > > >> >> > > > > > Can you clarify the order? >> > > >> >> > > > > > >> > > >> >> > > > > > -Dave >> > > >> >> > > > > > >> > > >> >> > > > > > On Thu, Apr 16, 2009 at 9:39 AM, Nick Stolwijk >> > > >> >> > > > > > <[email protected]>wrote: >> > > >> >> > > > > > >> > > >> >> > > > > > > Maven always takes the artifacts out of the local >> > > >> >> > > > > > repository. However, >> > > >> >> > > > > > > this is not a problem, because the >> > reactor knows in >> > > >> >> > > > > > > which >> > > >> order >> > > >> >> > to >> > > >> >> > > > > > > built the projects. Just before your internal >> > > >> >> > > > > > > project is >> > > >> >> > > > > > built, maven >> > > >> >> > > > > > > has installed the most recent version of >> > > >> >> > > > > > > security-public >> > > in >> > > >> >> > > > > > the local >> > > >> >> > > > > > > repository. >> > > >> >> > > > > > > >> > > >> >> > > > > > > Maybe I don't understand your problem. If that is >> > > >> >> > > > > > > the >> > > case, >> > > >> >> > > > > > please clarify. >> > > >> >> > > > > > > >> > > >> >> > > > > > > Hth, >> > > >> >> > > > > > > >> > > >> >> > > > > > > Nick Stolwijk >> > > >> >> > > > > > > ~Java Developer~ >> > > >> >> > > > > > > >> > > >> >> > > > > > > Iprofs BV. >> > > >> >> > > > > > > Claus Sluterweg 125 >> > > >> >> > > > > > > 2012 WS Haarlem >> > > >> >> > > > > > > www.iprofs.nl >> > > >> >> > > > > > > >> > > >> >> > > > > > > >> > > >> >> > > > > > > >> > > >> >> > > > > > > On Thu, Apr 16, 2009 at 5:26 PM, Dmitry Skavish >> > > >> >> > > > > > <[email protected]> wrote: >> > > >> >> > > > > > > > I am having the same problem and would like to >> > > >> >> > > > > > > > know >> > > that >> > > >> >> > > > > > as well. Thanks! >> > > >> >> > > > > > > > >> > > >> >> > > > > > > > On Thu, Apr 16, 2009 at 10:19 AM, David Hoffer >> > > >> >> > > > > > <[email protected]> >> > > >> >> > > > > > > wrote: >> > > >> >> > > > > > > > >> > > >> >> > > > > > > >> I have a multi-module build where some modules >> > > >> >> > > > > > > >> are >> > > >> dependent >> > > >> >> > on >> > > >> >> > > > > > > >> other modules. What is happening is that the >> > > dependent >> > > >> >> > > > > > module is >> > > >> >> > > > > > > >> getting its dependency from the >> > local/corporate >> > > >> >> > > > > > > >> maven >> > > >> >> > > > > > repo instead >> > > >> >> > > > > > > >> of the source >> > > >> >> > > > > > > code >> > > >> >> > > > > > > >> that was just built. How do I specify that >> > > >> >> > > > > > > >> modules >> > > >> always >> > > >> >> > build >> > > >> >> > > > > > > >> using current source not prior built >> > snapshot jars? >> > > >> >> > > > > > > >> >> > > >> >> > > > > > > >> Here is an example of the problem (it is really >> > > simple) >> > > >> >> > > > > > > >> >> > > >> >> > > > > > > >> Parent pom: >> > > >> >> > > > > > > >> <version>0.1-SNAPSHOT</version> <modules> >> > > >> >> > > > > > > >> <module>public</module> >> > > >> >> > > > > > > >> <module>internal</module> >> > > >> >> > > > > > > >> <module>security-public</module> >> > > >> >> > > > > > > >> </modules> >> > > >> >> > > > > > > >> >> > > >> >> > > > > > > >> public pom: >> > > >> >> > > > > > > >> <version>0.1-SNAPSHOT</version> >> > > >> >> > > > > > > >> >> > > >> >> > > > > > > >> internal pom: >> > > >> >> > > > > > > >> <dependencies> >> > > >> >> > > > > > > >> <dependency> >> > > >> >> > > > > > > >> >> > <groupId>${project.groupId}</groupId> >> > > >> >> > > > > > > >> <artifactId>public</artifactId> >> > > >> >> > > > > > > >> <version>0.1-SNAPSHOT</version> >> > > >> >> > > > > > > >> </dependency> </dependencies> >> > > >> >> > > > > > > >> >> > > >> >> > > > > > > >> security-public: >> > > >> >> > > > > > > >> <dependency> >> > > >> >> > > > > > > >> >> > <groupId>${project.groupId}</groupId> >> > > >> >> > > > > > > >> <artifactId>public</artifactId> >> > > >> >> > > > > > > >> <version>0.1-SNAPSHOT</version> >> > > </dependency> >> > > >> >> > > > > > > >> >> > > >> >> > > > > > > >> So what is happening is that instead >> > of internal >> > > >> >> > > > > > > >> & >> > > >> >> > > > > > security-public >> > > >> >> > > > > > > >> building using the just built public >> > (note it is >> > > >> >> > > > > > > >> first >> > > so >> > > >> it >> > > >> >> > was >> > > >> >> > > > > > > >> built first) >> > > >> >> > > > > > > they >> > > >> >> > > > > > > >> go >> > > >> >> > > > > > > >> out and download the last deployed >> > snapshot and >> > > >> >> > > > > > > >> build >> > > >> using >> > > >> >> > that >> > > >> >> > > > > > > instead. >> > > >> >> > > > > > > >> >> > > >> >> > > > > > > >> Nothing in the pom dependency syntax >> > really says >> > > >> >> > > > > > > >> which >> > > >> >> > > > > > to use but I >> > > >> >> > > > > > > assumed >> > > >> >> > > > > > > >> that because maven 'knows' these are all in the >> > > reactor >> > > >> it >> > > >> >> > would >> > > >> >> > > > > > > >> use >> > > >> >> > > > > > > module >> > > >> >> > > > > > > >> source. However this doesn't seem to >> > work, what >> > > >> >> > > > > > > >> do I >> > > >> >> > > > > > need to do to >> > > >> >> > > > > > > >> fix this? >> > > >> >> > > > > > > >> >> > > >> >> > > > > > > >> BTW, the goals being run are 'clean deploy >> > > site-deploy' >> > > >> >> > > > > > > >> >> > > >> >> > > > > > > >> -Dave >> > > >> >> > > > > > > >> >> > > >> >> > > > > > > > >> > > >> >> > > > > > > > >> > > >> >> > > > > > > > >> > > >> >> > > > > > > > -- >> > > >> >> > > > > > > > Dmitry Skavish >> > > >> >> > > > > > > > >> > > >> >> > > > > > > >> > > >> >> > > > > > > >> > > >> >> > > > > > >> > > >> >> > >> > > >> > --------------------------------------------------------------------- >> > > >> >> > > > > > > To unsubscribe, e-mail: >> > > [email protected] >> > > >> >> > > > > > > For additional commands, e-mail: >> > > >> [email protected] >> > > >> >> > > > > > > >> > > >> >> > > > > > > >> > > >> >> > > > > > >> > > >> >> > > > > >> > > >> >> > > > > >> > > >> >> >> > ---------------------------------------------------------------- >> > > >> >> ----- >> > > >> >> > > > > To unsubscribe, e-mail: >> > > >> >> > > > > [email protected] >> > > >> >> > > > > For additional commands, e-mail: >> > > >> >> > > > > [email protected] >> > > >> >> > > > > >> > > >> >> > > > > >> > > >> >> > > > >> > > >> >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> >> > > -- >> > > >> >> > > Dmitry Skavish >> > > >> >> > > >> > > >> >> > >> > > >> >> >> > > >> > >> > > >> >> > > >> >> > ------------------------------------------------------------------- >> > > >> -- To unsubscribe, e-mail: [email protected] >> > > >> For additional commands, e-mail: [email protected] >> > > >> >> > > >> >> > > > >> > > >> > > >> > --------------------------------------------------------------------- >> > > To unsubscribe, e-mail: [email protected] >> > > For additional commands, e-mail: [email protected] >> > > >> > > >> > >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >> > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
