On Mon, Aug 27, 2012 at 8:40 AM, Mark Struberg <strub...@yahoo.de> wrote: > We might implement this by storing the list of all activated profiles + all > resolved environment properties in a file in target/ somewhere. > That would be part of a.) in my previous mail to Romain.
How about the situation with shade? Build 1 runs shade and it replaces the primary build artifact with a shaded one. Build 2 sees that nothing has changed, but shade doesn't know, and it reshades, eating its own output with a ton of messages. Can I fix this in shade with current technologies? > > LieGrue, > strub > > > > ----- Original Message ----- >> From: Anders Hammar <and...@hammar.net> >> To: Maven Developers List <dev@maven.apache.org>; Mark Struberg >> <strub...@yahoo.de> >> Cc: >> Sent: Monday, August 27, 2012 12:24 PM >> Subject: Re: improving incremental builds >> >>> +0 for auto detecting changed scenarios. If someone changes a profile or >> the whole pom, then I'd expect he invokes a clean manually. We have to >> document this expectation of course. >> >> What do others think of this? I'm thinking it would be awesome (but >> maybe difficult) if plugins could determine if its configuration has >> changed, and then handle this automatically. If there are to many >> cases where a manual clean build is required, I believe people will >> continue to do "mvn clean install" (which I see all the time with >> developers), which I think is what we try to get away from. >> >> /Anders >> >>> >>> LieGrue, >>> strub >>> >>> >>> >>> >>> ----- Original Message ----- >>>> From: Anders Hammar <and...@hammar.net> >>>> To: Maven Developers List <dev@maven.apache.org> >>>> Cc: >>>> Sent: Monday, August 27, 2012 9:57 AM >>>> Subject: Re: improving incremental builds >>>> >>>>> I already started tweaking the m-compiler-plugin and found quite >> scary >>>> issues. There is e.g. MCOMPILER-21 (open since 2005) which prevents >> incremental >>>> build and would kick in in your scenario. >>>> >>>> This is interesting. I've looked a bit at Mojo's JAXB plugin in >> the >>>> context of detecting stale generated files (i.e. need to re-generate) >>>> and it's similar to this. It would extremely nice if there would be >> a >>>> common "framework" for handling incremental builds. >>>> In addition to what's been mentioned in the jira (I just browsed it >>>> quickly), we have cases when includes/excludes change, the sourceDir >>>> changes, etc which should trigger cleanup and re-compilation. >>>> >>>> /Anders >>>> >>>>> >>>>> I talked about 2 scenarios >>>>> >>>>> a.) all phases >= package, e.g. mvn verify, mvn install, ... >> Here we >>>> have an artifact on the disc and could test for the md5 of them, right? >>>>> >>>>> b.) all phases < package. This is e.g. mvn compile or mvn test. >> In that >>>> case we don't produce a jar/war/ear/... artifact but only get the >> files on >>>> the disk linked in MavenProject#getProjectArtifacts()... file(). This >> is the >>>> case where the maven-compiler-plugin kicks in. I'm currently in the >> process >>>> of rewriting big parts of it, mainly I introduced >>>>> b.1 a check for project.artifacts/project.testArtifacts .file() >> is a >>>> local file path .isDirectory() and has files which are newer (actually >>> =) >>>> than the buildStarted timestamp. >>>>> b.2 check whether there are any *.java (sourceincludes) files >> which are >>>> newer than their class files. >>>>> >>>>> In both cases I now force a FULL recompile of the module! >> Compiling only >>>> parts produced classes which are actually broken! >>>>> >>>>> >>>>> My approach is the following: compiling a bit too much is better >> than >>>> missing some files which need compilation. Because the later is exactly >> the >>>> reason why noone uses mvn compile but always do a mvn clean compile >> nowadays. If >>>> mvn compile is reliable, then we will end up being much faster than an >>>> unconditional full compile on the whole project! >>>>> >>>>> >>>>> LieGrue, >>>>> strub >>>>> >>>>> >>>>> PS: We need to move all our plugins which still use the >>>> plugin-testing-harness to the maven-invoker-plugin as the test-harness >> is broken >>>> with sisu. (sisu changed the 'container' field from plexus >> original: >>>> protected to 'private') >>>>> >>>>> PPS: how do we maintain the plexus-compiler-utils stuff? This >> contains some >>>> weirdo bugs which should get fixed... >>>>> >>>>> >>>>> >>>>> ----- Original Message ----- >>>>>> From: Olivier Lamy <ol...@apache.org> >>>>>> To: Maven Developers List <dev@maven.apache.org>; Mark >> Struberg >>>> <strub...@yahoo.de> >>>>>> Cc: >>>>>> Sent: Monday, August 27, 2012 9:13 AM >>>>>> Subject: Re: improving incremental builds >>>>>> >>>>>> Hi, >>>>>> Sounds good idea trying to improve that. >>>>>> My question: on what is based the md5 calculation ? >>>>>> If people don't use 'install' but only >> 'test' >>>> (perso I use >>>>>> that to >>>>>> avoid too much io with jar creation etc..), so in such case we >> cannot >>>>>> do md5 calculation on the produced artifact. >>>>>> >>>>>> 2012/8/26 Mark Struberg <strub...@yahoo.de>: >>>>>>> Hi David! >>>>>>> >>>>>>>> So your idea is to find the list of changed modules >> and >>>>>>>> then build that list with -amd? >>>>>>> Yes, kind of. >>>>>>> >>>>>>> At the moment mvn -amd builds the dependents of the >> _current_ >>>> module. If >>>>>> you use my example and change BeanA.java, then run mvn -amd >> from the >>>> root module >>>>>> you will see that only moduleA gets rebuild and moduleB >> remains >>>> original. >>>>>> Because moduleB is not a dependent of the root module. >>>>>>> >>>>>>> But yes, I'm completely with you that we have most of >> the >>>> ingredients >>>>>> in the maven codebase already. At least for the start we could >> easily >>>> detect >>>>>> changed build results and add those to the 'amd' list. >> That >>>> would >>>>>> already be much better than what we have today imo. And this >> should be >>>> pretty >>>>>> easy to implement. >>>>>>> >>>>>>> Indeed, what I proposed goes a bit beyond -amd. I would >> not only >>>> check the >>>>>> current build tree like -amd does (even if that would be a >> good start) >>>> but >>>>>> remember the md5 of all the dependencies of each module >> (simply store >>>> them in a >>>>>> file in ./target/) And if you find a dependency which is not >> in the >>>> list (e.g. >>>>>> after upgrade from one version to another) or has a different >> md5 (this >>>> covers >>>>>> SNAPSHOT versions) , then we could force a full rebuild (mvn >> -amd) of >>>> this >>>>>> module. >>>>>>> >>>>>>> >>>>>>> LieGrue, >>>>>>> strub >>>>>>> >>>>>>> >>>>>>> >>>>>>> ----- Original Message ----- >>>>>>>> From: David Jencks <david_jen...@yahoo.com> >>>>>>>> To: Maven Developers List >> <dev@maven.apache.org> >>>>>>>> Cc: >>>>>>>> Sent: Sunday, August 26, 2012 8:34 AM >>>>>>>> Subject: Re: improving incremental builds >>>>>>>> >>>>>>>> Is what you want different from what >>>>>>>> >>>>>>>> mvn -amd moduleA >>>>>>>> >>>>>>>> does? So your idea is to find the list of changed >> modules and >>>> then >>>>>> build that >>>>>>>> list with -amd? >>>>>>>> >>>>>>>> thanks >>>>>>>> david jencks >>>>>>>> >>>>>>>> On Aug 25, 2012, at 1:32 PM, Mark Struberg wrote: >>>>>>>> >>>>>>>>> Hi folks! >>>>>>>>> >>>>>>>>> After a short discussion with Kristian, I've >> created >>>> a small >>>>>> app with 2 >>>>>>>> modules which shows a few problems with mavens >> incremental >>>> build logic. >>>>>>>>> And since incremental builds do not work well, >> people use >>>>>>>>> >>>>>>>>> $> mvn _clean_ install >>>>>>>>> all the time. >>>>>>>>> >>>>>>>>> We could speed up the development effort heavily >> if we >>>> make >>>>>>>>> $> mvn install >>>>>>>>> (without an upfront clean) more reliable. >>>>>>>>> >>>>>>>>> >>>>>>>>> The sample [1] consists of moduleA and moduleB >> with BeanA >>>> and >>>>>> BeanB >>>>>>>> respectively. >>>>>>>>> BeanB has a private BeanA field and moduleB has >> a >>>> dependency on >>>>>> moduleA. >>>>>>>>> >>>>>>>>> If I change BeanA and just invoke mvn install >> then only >>>> moduleA >>>>>> gets >>>>>>>> rebuilt. We currently do not rebuild moduleB, but we >> should do >>>> to >>>>>> create a >>>>>>>> reliable output. >>>>>>>>> >>>>>>>>> In fact, the incremental build within a single >> module >>>> already >>>>>> works to some >>>>>>>> degrees, but we must detect a change in dependencies >> and >>>> trigger a full >>>>>> rebuild >>>>>>>> on all depending modules. This could be done by >> storing the >>>> md5 of all >>>>>>>> dependency artifacts and compare them on the next >> build. If >>>> the md5 of >>>>>> a >>>>>>>> dependency did change, then we need to build the >> module full >>>> cycle. >>>>>>>>> Other ideas are welcome. Slaps as well if I >> forgot some >>>> obvious >>>>>> stuff and >>>>>>>> all works well already. >>>>>>>>> >>>>>>>>> >>>>>>>>> LieGrue, >>>>>>>>> strub >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>> >> --------------------------------------------------------------------- >>>>>>>>> To unsubscribe, e-mail: >> dev-unsubscr...@maven.apache.org >>>>>>>>> For additional commands, e-mail: >>>> dev-h...@maven.apache.org >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>> --------------------------------------------------------------------- >>>>>>>> To unsubscribe, e-mail: >> dev-unsubscr...@maven.apache.org >>>>>>>> For additional commands, e-mail: >> dev-h...@maven.apache.org >>>>>>>> >>>>>>> >>>>>>> >>>> --------------------------------------------------------------------- >>>>>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >>>>>>> For additional commands, e-mail: >> dev-h...@maven.apache.org >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Olivier Lamy >>>>>> Talend: http://coders.talend.com >>>>>> http://twitter.com/olamy | http://linkedin.com/in/olamy >>>>>> >>>>> >>>>> >> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >>>>> For additional commands, e-mail: dev-h...@maven.apache.org >>>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >>>> For additional commands, e-mail: dev-h...@maven.apache.org >>>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >>> For additional commands, e-mail: dev-h...@maven.apache.org >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >> For additional commands, e-mail: dev-h...@maven.apache.org >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
