Benson any chance of *forcing* the second iteration to re-jar with <forceCreation>true</forceCreation> http://maven.apache.org/plugins/maven-jar-plugin/jar-mojo.html#forceCreation
Martin ______________________________________________ ..place long-winded disclaimer here.. > Date: Mon, 27 Aug 2012 13:09:59 -0400 > Subject: Re: improving incremental builds > From: [email protected] > To: [email protected]; [email protected] > > On Mon, Aug 27, 2012 at 12:07 PM, Mark Struberg <[email protected]> wrote: > > build1 and build 2 are different modules or different mvn invocations with > > some time inbetween? > > > > Of course we will need to test all plugins to behave incremental like! > > Means the maven-shade-plugin should check itself whether it needs shading > > or not at all. > > Two builds, separated in time, of exactly the same module. > > The problem here is that the jar plugin has already decided not to > rebuild the jar by the time the shade plugin gets there, and I have no > idea how to make the shade plugin play along. > > > > > > But the worst thing happening would be that we compile too much. Thats > > still better than a full mvn clean install :) > > > > LieGrue, > > strub > > > > > > > > ----- Original Message ----- > >> From: Benson Margulies <[email protected]> > >> To: Maven Developers List <[email protected]>; Mark Struberg > >> <[email protected]> > >> Cc: > >> Sent: Monday, August 27, 2012 5:07 PM > >> Subject: Re: improving incremental builds > >> > >> On Mon, Aug 27, 2012 at 8:40 AM, Mark Struberg <[email protected]> wrote: > >>> We might implement this by storing the list of all activated profiles + > >>> all > >> resolved environment properties in a file in target/ somewhere. > >>> That would be part of a.) in my previous mail to Romain. > >> > >> How about the situation with shade? > >> > >> Build 1 runs shade and it replaces the primary build artifact with a > >> shaded one. > >> > >> Build 2 sees that nothing has changed, but shade doesn't know, and it > >> reshades, eating its own output with a ton of messages. > >> > >> Can I fix this in shade with current technologies? > >> > >> > >> > >>> > >>> LieGrue, > >>> strub > >>> > >>> > >>> > >>> ----- Original Message ----- > >>>> From: Anders Hammar <[email protected]> > >>>> To: Maven Developers List <[email protected]>; Mark Struberg > >> <[email protected]> > >>>> Cc: > >>>> Sent: Monday, August 27, 2012 12:24 PM > >>>> Subject: Re: improving incremental builds > >>>> > >>>>> +0 for auto detecting changed scenarios. If someone changes a > >> profile or > >>>> the whole pom, then I'd expect he invokes a clean manually. We > >> have to > >>>> document this expectation of course. > >>>> > >>>> What do others think of this? I'm thinking it would be awesome (but > >>>> maybe difficult) if plugins could determine if its configuration has > >>>> changed, and then handle this automatically. If there are to many > >>>> cases where a manual clean build is required, I believe people will > >>>> continue to do "mvn clean install" (which I see all the time > >> with > >>>> developers), which I think is what we try to get away from. > >>>> > >>>> /Anders > >>>> > >>>>> > >>>>> LieGrue, > >>>>> strub > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> ----- Original Message ----- > >>>>>> From: Anders Hammar <[email protected]> > >>>>>> To: Maven Developers List <[email protected]> > >>>>>> Cc: > >>>>>> Sent: Monday, August 27, 2012 9:57 AM > >>>>>> Subject: Re: improving incremental builds > >>>>>> > >>>>>>> I already started tweaking the m-compiler-plugin and > >> found quite > >>>> scary > >>>>>> issues. There is e.g. MCOMPILER-21 (open since 2005) which > >> prevents > >>>> incremental > >>>>>> build and would kick in in your scenario. > >>>>>> > >>>>>> This is interesting. I've looked a bit at Mojo's JAXB > >> plugin in > >>>> the > >>>>>> context of detecting stale generated files (i.e. need to > >> re-generate) > >>>>>> and it's similar to this. It would extremely nice if there > >> would be > >>>> a > >>>>>> common "framework" for handling incremental builds. > >>>>>> In addition to what's been mentioned in the jira (I just > >> browsed it > >>>>>> quickly), we have cases when includes/excludes change, the > >> sourceDir > >>>>>> changes, etc which should trigger cleanup and re-compilation. > >>>>>> > >>>>>> /Anders > >>>>>> > >>>>>>> > >>>>>>> I talked about 2 scenarios > >>>>>>> > >>>>>>> a.) all phases >= package, e.g. mvn verify, mvn > >> install, ... > >>>> Here we > >>>>>> have an artifact on the disc and could test for the md5 of > >> them, right? > >>>>>>> > >>>>>>> b.) all phases < package. This is e.g. mvn compile or > >> mvn test. > >>>> In that > >>>>>> case we don't produce a jar/war/ear/... artifact but only > >> get the > >>>> files on > >>>>>> the disk linked in MavenProject#getProjectArtifacts()... > >> file(). This > >>>> is the > >>>>>> case where the maven-compiler-plugin kicks in. I'm > >> currently in the > >>>> process > >>>>>> of rewriting big parts of it, mainly I introduced > >>>>>>> b.1 a check for project.artifacts/project.testArtifacts > >> .file() > >>>> is a > >>>>>> local file path .isDirectory() and has files which are newer > >> (actually > >>>>> =) > >>>>>> than the buildStarted timestamp. > >>>>>>> b.2 check whether there are any *.java (sourceincludes) > >> files > >>>> which are > >>>>>> newer than their class files. > >>>>>>> > >>>>>>> In both cases I now force a FULL recompile of the module! > >>>> Compiling only > >>>>>> parts produced classes which are actually broken! > >>>>>>> > >>>>>>> > >>>>>>> My approach is the following: compiling a bit too much is > >> better > >>>> than > >>>>>> missing some files which need compilation. Because the later > >> is exactly > >>>> the > >>>>>> reason why noone uses mvn compile but always do a mvn clean > >> compile > >>>> nowadays. If > >>>>>> mvn compile is reliable, then we will end up being much faster > >> than an > >>>>>> unconditional full compile on the whole project! > >>>>>>> > >>>>>>> > >>>>>>> LieGrue, > >>>>>>> strub > >>>>>>> > >>>>>>> > >>>>>>> PS: We need to move all our plugins which still use the > >>>>>> plugin-testing-harness to the maven-invoker-plugin as the > >> test-harness > >>>> is broken > >>>>>> with sisu. (sisu changed the 'container' field from > >> plexus > >>>> original: > >>>>>> protected to 'private') > >>>>>>> > >>>>>>> PPS: how do we maintain the plexus-compiler-utils stuff? > >> This > >>>> contains some > >>>>>> weirdo bugs which should get fixed... > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> ----- Original Message ----- > >>>>>>>> From: Olivier Lamy <[email protected]> > >>>>>>>> To: Maven Developers List > >> <[email protected]>; Mark > >>>> Struberg > >>>>>> <[email protected]> > >>>>>>>> Cc: > >>>>>>>> Sent: Monday, August 27, 2012 9:13 AM > >>>>>>>> Subject: Re: improving incremental builds > >>>>>>>> > >>>>>>>> Hi, > >>>>>>>> Sounds good idea trying to improve that. > >>>>>>>> My question: on what is based the md5 calculation ? > >>>>>>>> If people don't use 'install' but only > >>>> 'test' > >>>>>> (perso I use > >>>>>>>> that to > >>>>>>>> avoid too much io with jar creation etc..), so in > >> such case we > >>>> cannot > >>>>>>>> do md5 calculation on the produced artifact. > >>>>>>>> > >>>>>>>> 2012/8/26 Mark Struberg <[email protected]>: > >>>>>>>>> Hi David! > >>>>>>>>> > >>>>>>>>>> So your idea is to find the list of changed > >> modules > >>>> and > >>>>>>>>>> then build that list with -amd? > >>>>>>>>> Yes, kind of. > >>>>>>>>> > >>>>>>>>> At the moment mvn -amd builds the dependents of > >> the > >>>> _current_ > >>>>>> module. If > >>>>>>>> you use my example and change BeanA.java, then run > >> mvn -amd > >>>> from the > >>>>>> root module > >>>>>>>> you will see that only moduleA gets rebuild and > >> moduleB > >>>> remains > >>>>>> original. > >>>>>>>> Because moduleB is not a dependent of the root > >> module. > >>>>>>>>> > >>>>>>>>> But yes, I'm completely with you that we > >> have most of > >>>> the > >>>>>> ingredients > >>>>>>>> in the maven codebase already. At least for the start > >> we could > >>>> easily > >>>>>> detect > >>>>>>>> changed build results and add those to the > >> 'amd' list. > >>>> That > >>>>>> would > >>>>>>>> already be much better than what we have today imo. > >> And this > >>>> should be > >>>>>> pretty > >>>>>>>> easy to implement. > >>>>>>>>> > >>>>>>>>> Indeed, what I proposed goes a bit beyond -amd. > >> I would > >>>> not only > >>>>>> check the > >>>>>>>> current build tree like -amd does (even if that would > >> be a > >>>> good start) > >>>>>> but > >>>>>>>> remember the md5 of all the dependencies of each > >> module > >>>> (simply store > >>>>>> them in a > >>>>>>>> file in ./target/) And if you find a dependency which > >> is not > >>>> in the > >>>>>> list (e.g. > >>>>>>>> after upgrade from one version to another) or has a > >> different > >>>> md5 (this > >>>>>> covers > >>>>>>>> SNAPSHOT versions) , then we could force a full > >> rebuild (mvn > >>>> -amd) of > >>>>>> this > >>>>>>>> module. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> LieGrue, > >>>>>>>>> strub > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> ----- Original Message ----- > >>>>>>>>>> From: David Jencks > >> <[email protected]> > >>>>>>>>>> To: Maven Developers List > >>>> <[email protected]> > >>>>>>>>>> Cc: > >>>>>>>>>> Sent: Sunday, August 26, 2012 8:34 AM > >>>>>>>>>> Subject: Re: improving incremental builds > >>>>>>>>>> > >>>>>>>>>> Is what you want different from what > >>>>>>>>>> > >>>>>>>>>> mvn -amd moduleA > >>>>>>>>>> > >>>>>>>>>> does? So your idea is to find the list of > >> changed > >>>> modules and > >>>>>> then > >>>>>>>> build that > >>>>>>>>>> list with -amd? > >>>>>>>>>> > >>>>>>>>>> thanks > >>>>>>>>>> david jencks > >>>>>>>>>> > >>>>>>>>>> On Aug 25, 2012, at 1:32 PM, Mark Struberg > >> wrote: > >>>>>>>>>> > >>>>>>>>>>> Hi folks! > >>>>>>>>>>> > >>>>>>>>>>> After a short discussion with Kristian, > >> I've > >>>> created > >>>>>> a small > >>>>>>>> app with 2 > >>>>>>>>>> modules which shows a few problems with > >> mavens > >>>> incremental > >>>>>> build logic. > >>>>>>>>>>> And since incremental builds do not > >> work well, > >>>> people use > >>>>>>>>>>> > >>>>>>>>>>> $> mvn _clean_ install > >>>>>>>>>>> all the time. > >>>>>>>>>>> > >>>>>>>>>>> We could speed up the development > >> effort heavily > >>>> if we > >>>>>> make > >>>>>>>>>>> $> mvn install > >>>>>>>>>>> (without an upfront clean) more > >> reliable. > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> The sample [1] consists of moduleA and > >> moduleB > >>>> with BeanA > >>>>>> and > >>>>>>>> BeanB > >>>>>>>>>> respectively. > >>>>>>>>>>> BeanB has a private BeanA field and > >> moduleB has > >>>> a > >>>>>> dependency on > >>>>>>>> moduleA. > >>>>>>>>>>> > >>>>>>>>>>> If I change BeanA and just invoke mvn > >> install > >>>> then only > >>>>>> moduleA > >>>>>>>> gets > >>>>>>>>>> rebuilt. We currently do not rebuild > >> moduleB, but we > >>>> should do > >>>>>> to > >>>>>>>> create a > >>>>>>>>>> reliable output. > >>>>>>>>>>> > >>>>>>>>>>> In fact, the incremental build within a > >> single > >>>> module > >>>>>> already > >>>>>>>> works to some > >>>>>>>>>> degrees, but we must detect a change in > >> dependencies > >>>> and > >>>>>> trigger a full > >>>>>>>> rebuild > >>>>>>>>>> on all depending modules. This could be done > >> by > >>>> storing the > >>>>>> md5 of all > >>>>>>>>>> dependency artifacts and compare them on the > >> next > >>>> build. If > >>>>>> the md5 of > >>>>>>>> a > >>>>>>>>>> dependency did change, then we need to build > >> the > >>>> module full > >>>>>> cycle. > >>>>>>>>>>> Other ideas are welcome. Slaps as well > >> if I > >>>> forgot some > >>>>>> obvious > >>>>>>>> stuff and > >>>>>>>>>> all works well already. > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> LieGrue, > >>>>>>>>>>> strub > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>> > >>>> --------------------------------------------------------------------- > >>>>>>>>>>> To unsubscribe, e-mail: > >>>> [email protected] > >>>>>>>>>>> For additional commands, e-mail: > >>>>>> [email protected] > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>> > >> --------------------------------------------------------------------- > >>>>>>>>>> To unsubscribe, e-mail: > >>>> [email protected] > >>>>>>>>>> For additional commands, e-mail: > >>>> [email protected] > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>> > >> --------------------------------------------------------------------- > >>>>>>>>> To unsubscribe, e-mail: > >> [email protected] > >>>>>>>>> For additional commands, e-mail: > >>>> [email protected] > >>>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> -- > >>>>>>>> Olivier Lamy > >>>>>>>> Talend: http://coders.talend.com > >>>>>>>> http://twitter.com/olamy | > >> http://linkedin.com/in/olamy > >>>>>>>> > >>>>>>> > >>>>>>> > >>>> --------------------------------------------------------------------- > >>>>>>> To unsubscribe, e-mail: [email protected] > >>>>>>> For additional commands, e-mail: > >> [email protected] > >>>>>>> > >>>>>> > >>>>>> > >> --------------------------------------------------------------------- > >>>>>> To unsubscribe, e-mail: [email protected] > >>>>>> For additional commands, e-mail: [email protected] > >>>>>> > >>>>> > >>>>> > >> --------------------------------------------------------------------- > >>>>> To unsubscribe, e-mail: [email protected] > >>>>> For additional commands, e-mail: [email protected] > >>>>> > >>>> > >>>> --------------------------------------------------------------------- > >>>> To unsubscribe, e-mail: [email protected] > >>>> For additional commands, e-mail: [email protected] > >>>> > >>> > >>> --------------------------------------------------------------------- > >>> To unsubscribe, e-mail: [email protected] > >>> For additional commands, e-mail: [email protected] > >>> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: [email protected] > >> For additional commands, e-mail: [email protected] > >> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] >
