The assembly plugin example is perfectly fine if the dependencies are defined in a <dependency> section. If a SNAPSHOT dependency did change, then it will get a different md5. Of course if plugins resolve some dependencies programmatically, then those cases will not work easily. But that is considered bad practice anyway, isn't?
There are clearly 2 different aspects we need to check/implement to get this fully working: a.) artifact dependencies md5 changed -> invoke clean for each downstream module b.) each plugin must check itself if a required class changed and then update the output accordingly. If all plugins in the chain do this properly then we will do much better already. I'm currently focusing to get b.) implemented in the maven-compiler-plugin LieGrue, strub ----- Original Message ----- > From: Romain Manni-Bucau <rmannibu...@gmail.com> > To: Maven Developers List <dev@maven.apache.org> > Cc: > Sent: Monday, August 27, 2012 12:28 PM > Subject: Re: improving incremental builds > > Hi, > > The full invremental is great but not always possible (think to assembly > plugin, some case xill be very hard to handle in snapshot mode i guess) > > Maybe it is time to get a maven daemon to help to be able to store > information? > > - Romain > Le 27 août 2012 12:24, "Anders Hammar" <and...@hammar.net> a > écrit : > >> > +0 for auto detecting changed scenarios. If someone changes a profile > or >> the whole pom, then I'd expect he invokes a clean manually. We have to >> document this expectation of course. >> >> What do others think of this? I'm thinking it would be awesome (but >> maybe difficult) if plugins could determine if its configuration has >> changed, and then handle this automatically. If there are to many >> cases where a manual clean build is required, I believe people will >> continue to do "mvn clean install" (which I see all the time with >> developers), which I think is what we try to get away from. >> >> /Anders >> >> > >> > LieGrue, >> > strub >> > >> > >> > >> > >> > ----- Original Message ----- >> >> From: Anders Hammar <and...@hammar.net> >> >> To: Maven Developers List <dev@maven.apache.org> >> >> Cc: >> >> Sent: Monday, August 27, 2012 9:57 AM >> >> Subject: Re: improving incremental builds >> >> >> >>> I already started tweaking the m-compiler-plugin and found > quite scary >> >> issues. There is e.g. MCOMPILER-21 (open since 2005) which > prevents >> incremental >> >> build and would kick in in your scenario. >> >> >> >> This is interesting. I've looked a bit at Mojo's JAXB > plugin in the >> >> context of detecting stale generated files (i.e. need to > re-generate) >> >> and it's similar to this. It would extremely nice if there > would be a >> >> common "framework" for handling incremental builds. >> >> In addition to what's been mentioned in the jira (I just > browsed it >> >> quickly), we have cases when includes/excludes change, the > sourceDir >> >> changes, etc which should trigger cleanup and re-compilation. >> >> >> >> /Anders >> >> >> >>> >> >>> I talked about 2 scenarios >> >>> >> >>> a.) all phases >= package, e.g. mvn verify, mvn install, > ... Here we >> >> have an artifact on the disc and could test for the md5 of them, > right? >> >>> >> >>> b.) all phases < package. This is e.g. mvn compile or mvn > test. In >> that >> >> case we don't produce a jar/war/ear/... artifact but only get > the files >> on >> >> the disk linked in MavenProject#getProjectArtifacts()... file(). > This >> is the >> >> case where the maven-compiler-plugin kicks in. I'm currently > in the >> process >> >> of rewriting big parts of it, mainly I introduced >> >>> b.1 a check for project.artifacts/project.testArtifacts > .file() is a >> >> local file path .isDirectory() and has files which are newer > (actually >> >=) >> >> than the buildStarted timestamp. >> >>> b.2 check whether there are any *.java (sourceincludes) > files which >> are >> >> newer than their class files. >> >>> >> >>> In both cases I now force a FULL recompile of the module! > Compiling >> only >> >> parts produced classes which are actually broken! >> >>> >> >>> >> >>> My approach is the following: compiling a bit too much is > better than >> >> missing some files which need compilation. Because the later is > exactly >> the >> >> reason why noone uses mvn compile but always do a mvn clean > compile >> nowadays. If >> >> mvn compile is reliable, then we will end up being much faster > than an >> >> unconditional full compile on the whole project! >> >>> >> >>> >> >>> LieGrue, >> >>> strub >> >>> >> >>> >> >>> PS: We need to move all our plugins which still use the >> >> plugin-testing-harness to the maven-invoker-plugin as the > test-harness >> is broken >> >> with sisu. (sisu changed the 'container' field from plexus > original: >> >> protected to 'private') >> >>> >> >>> PPS: how do we maintain the plexus-compiler-utils stuff? This >> contains some >> >> weirdo bugs which should get fixed... >> >>> >> >>> >> >>> >> >>> ----- Original Message ----- >> >>>> From: Olivier Lamy <ol...@apache.org> >> >>>> To: Maven Developers List <dev@maven.apache.org>; > Mark Struberg >> >> <strub...@yahoo.de> >> >>>> Cc: >> >>>> Sent: Monday, August 27, 2012 9:13 AM >> >>>> Subject: Re: improving incremental builds >> >>>> >> >>>> Hi, >> >>>> Sounds good idea trying to improve that. >> >>>> My question: on what is based the md5 calculation ? >> >>>> If people don't use 'install' but only > 'test' >> >> (perso I use >> >>>> that to >> >>>> avoid too much io with jar creation etc..), so in such > case we cannot >> >>>> do md5 calculation on the produced artifact. >> >>>> >> >>>> 2012/8/26 Mark Struberg <strub...@yahoo.de>: >> >>>>> Hi David! >> >>>>> >> >>>>>> So your idea is to find the list of changed > modules and >> >>>>>> then build that list with -amd? >> >>>>> Yes, kind of. >> >>>>> >> >>>>> At the moment mvn -amd builds the dependents of the > _current_ >> >> module. If >> >>>> you use my example and change BeanA.java, then run mvn > -amd from the >> >> root module >> >>>> you will see that only moduleA gets rebuild and moduleB > remains >> >> original. >> >>>> Because moduleB is not a dependent of the root module. >> >>>>> >> >>>>> But yes, I'm completely with you that we have > most of the >> >> ingredients >> >>>> in the maven codebase already. At least for the start we > could easily >> >> detect >> >>>> changed build results and add those to the 'amd' > list. That >> >> would >> >>>> already be much better than what we have today imo. And > this should >> be >> >> pretty >> >>>> easy to implement. >> >>>>> >> >>>>> Indeed, what I proposed goes a bit beyond -amd. I > would not only >> >> check the >> >>>> current build tree like -amd does (even if that would be > a good >> start) >> >> but >> >>>> remember the md5 of all the dependencies of each module > (simply store >> >> them in a >> >>>> file in ./target/) And if you find a dependency which is > not in the >> >> list (e.g. >> >>>> after upgrade from one version to another) or has a > different md5 >> (this >> >> covers >> >>>> SNAPSHOT versions) , then we could force a full rebuild > (mvn -amd) of >> >> this >> >>>> module. >> >>>>> >> >>>>> >> >>>>> LieGrue, >> >>>>> strub >> >>>>> >> >>>>> >> >>>>> >> >>>>> ----- Original Message ----- >> >>>>>> From: David Jencks > <david_jen...@yahoo.com> >> >>>>>> To: Maven Developers List > <dev@maven.apache.org> >> >>>>>> Cc: >> >>>>>> Sent: Sunday, August 26, 2012 8:34 AM >> >>>>>> Subject: Re: improving incremental builds >> >>>>>> >> >>>>>> Is what you want different from what >> >>>>>> >> >>>>>> mvn -amd moduleA >> >>>>>> >> >>>>>> does? So your idea is to find the list of > changed modules and >> >> then >> >>>> build that >> >>>>>> list with -amd? >> >>>>>> >> >>>>>> thanks >> >>>>>> david jencks >> >>>>>> >> >>>>>> On Aug 25, 2012, at 1:32 PM, Mark Struberg > wrote: >> >>>>>> >> >>>>>>> Hi folks! >> >>>>>>> >> >>>>>>> After a short discussion with Kristian, > I've created >> >> a small >> >>>> app with 2 >> >>>>>> modules which shows a few problems with mavens > incremental >> >> build logic. >> >>>>>>> And since incremental builds do not work > well, people use >> >>>>>>> >> >>>>>>> $> mvn _clean_ install >> >>>>>>> all the time. >> >>>>>>> >> >>>>>>> We could speed up the development effort > heavily if we >> >> make >> >>>>>>> $> mvn install >> >>>>>>> (without an upfront clean) more reliable. >> >>>>>>> >> >>>>>>> >> >>>>>>> The sample [1] consists of moduleA and > moduleB with BeanA >> >> and >> >>>> BeanB >> >>>>>> respectively. >> >>>>>>> BeanB has a private BeanA field and moduleB > has a >> >> dependency on >> >>>> moduleA. >> >>>>>>> >> >>>>>>> If I change BeanA and just invoke mvn > install then only >> >> moduleA >> >>>> gets >> >>>>>> rebuilt. We currently do not rebuild moduleB, > but we should do >> >> to >> >>>> create a >> >>>>>> reliable output. >> >>>>>>> >> >>>>>>> In fact, the incremental build within a > single module >> >> already >> >>>> works to some >> >>>>>> degrees, but we must detect a change in > dependencies and >> >> trigger a full >> >>>> rebuild >> >>>>>> on all depending modules. This could be done by > storing the >> >> md5 of all >> >>>>>> dependency artifacts and compare them on the > next build. If >> >> the md5 of >> >>>> a >> >>>>>> dependency did change, then we need to build the > module full >> >> cycle. >> >>>>>>> Other ideas are welcome. Slaps as well if I > forgot some >> >> obvious >> >>>> stuff and >> >>>>>> all works well already. >> >>>>>>> >> >>>>>>> >> >>>>>>> LieGrue, >> >>>>>>> strub >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>> > --------------------------------------------------------------------- >> >>>>>>> To unsubscribe, e-mail: > dev-unsubscr...@maven.apache.org >> >>>>>>> For additional commands, e-mail: >> >> dev-h...@maven.apache.org >> >>>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> >> > --------------------------------------------------------------------- >> >>>>>> To unsubscribe, e-mail: > dev-unsubscr...@maven.apache.org >> >>>>>> For additional commands, e-mail: > dev-h...@maven.apache.org >> >>>>>> >> >>>>> >> >>>>> >> >> > --------------------------------------------------------------------- >> >>>>> To unsubscribe, e-mail: > dev-unsubscr...@maven.apache.org >> >>>>> For additional commands, e-mail: > dev-h...@maven.apache.org >> >>>>> >> >>>> >> >>>> >> >>>> >> >>>> -- >> >>>> Olivier Lamy >> >>>> Talend: http://coders.talend.com >> >>>> http://twitter.com/olamy | http://linkedin.com/in/olamy >> >>>> >> >>> >> >>> > --------------------------------------------------------------------- >> >>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >> >>> For additional commands, e-mail: dev-h...@maven.apache.org >> >>> >> >> >> >> > --------------------------------------------------------------------- >> >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >> >> For additional commands, e-mail: dev-h...@maven.apache.org >> >> >> > >> > --------------------------------------------------------------------- >> > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >> > For additional commands, e-mail: dev-h...@maven.apache.org >> > >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >> For additional commands, e-mail: dev-h...@maven.apache.org >> >> > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
