> I already started tweaking the m-compiler-plugin and found quite scary > issues. There is e.g. MCOMPILER-21 (open since 2005) which prevents > incremental build and would kick in in your scenario.
This is interesting. I've looked a bit at Mojo's JAXB plugin in the context of detecting stale generated files (i.e. need to re-generate) and it's similar to this. It would extremely nice if there would be a common "framework" for handling incremental builds. In addition to what's been mentioned in the jira (I just browsed it quickly), we have cases when includes/excludes change, the sourceDir changes, etc which should trigger cleanup and re-compilation. /Anders > > I talked about 2 scenarios > > a.) all phases >= package, e.g. mvn verify, mvn install, ... Here we have an > artifact on the disc and could test for the md5 of them, right? > > b.) all phases < package. This is e.g. mvn compile or mvn test. In that case > we don't produce a jar/war/ear/... artifact but only get the files on the > disk linked in MavenProject#getProjectArtifacts()... file(). This is the case > where the maven-compiler-plugin kicks in. I'm currently in the process of > rewriting big parts of it, mainly I introduced > b.1 a check for project.artifacts/project.testArtifacts .file() is a local > file path .isDirectory() and has files which are newer (actually >=) than the > buildStarted timestamp. > b.2 check whether there are any *.java (sourceincludes) files which are > newer than their class files. > > In both cases I now force a FULL recompile of the module! Compiling only > parts produced classes which are actually broken! > > > My approach is the following: compiling a bit too much is better than missing > some files which need compilation. Because the later is exactly the reason > why noone uses mvn compile but always do a mvn clean compile nowadays. If mvn > compile is reliable, then we will end up being much faster than an > unconditional full compile on the whole project! > > > LieGrue, > strub > > > PS: We need to move all our plugins which still use the > plugin-testing-harness to the maven-invoker-plugin as the test-harness is > broken with sisu. (sisu changed the 'container' field from plexus original: > protected to 'private') > > PPS: how do we maintain the plexus-compiler-utils stuff? This contains some > weirdo bugs which should get fixed... > > > > ----- Original Message ----- >> From: Olivier Lamy <ol...@apache.org> >> To: Maven Developers List <dev@maven.apache.org>; Mark Struberg >> <strub...@yahoo.de> >> Cc: >> Sent: Monday, August 27, 2012 9:13 AM >> Subject: Re: improving incremental builds >> >> Hi, >> Sounds good idea trying to improve that. >> My question: on what is based the md5 calculation ? >> If people don't use 'install' but only 'test' (perso I use >> that to >> avoid too much io with jar creation etc..), so in such case we cannot >> do md5 calculation on the produced artifact. >> >> 2012/8/26 Mark Struberg <strub...@yahoo.de>: >>> Hi David! >>> >>>> So your idea is to find the list of changed modules and >>>> then build that list with -amd? >>> Yes, kind of. >>> >>> At the moment mvn -amd builds the dependents of the _current_ module. If >> you use my example and change BeanA.java, then run mvn -amd from the root >> module >> you will see that only moduleA gets rebuild and moduleB remains original. >> Because moduleB is not a dependent of the root module. >>> >>> But yes, I'm completely with you that we have most of the ingredients >> in the maven codebase already. At least for the start we could easily detect >> changed build results and add those to the 'amd' list. That would >> already be much better than what we have today imo. And this should be pretty >> easy to implement. >>> >>> Indeed, what I proposed goes a bit beyond -amd. I would not only check the >> current build tree like -amd does (even if that would be a good start) but >> remember the md5 of all the dependencies of each module (simply store them >> in a >> file in ./target/) And if you find a dependency which is not in the list >> (e.g. >> after upgrade from one version to another) or has a different md5 (this >> covers >> SNAPSHOT versions) , then we could force a full rebuild (mvn -amd) of this >> module. >>> >>> >>> LieGrue, >>> strub >>> >>> >>> >>> ----- Original Message ----- >>>> From: David Jencks <david_jen...@yahoo.com> >>>> To: Maven Developers List <dev@maven.apache.org> >>>> Cc: >>>> Sent: Sunday, August 26, 2012 8:34 AM >>>> Subject: Re: improving incremental builds >>>> >>>> Is what you want different from what >>>> >>>> mvn -amd moduleA >>>> >>>> does? So your idea is to find the list of changed modules and then >> build that >>>> list with -amd? >>>> >>>> thanks >>>> david jencks >>>> >>>> On Aug 25, 2012, at 1:32 PM, Mark Struberg wrote: >>>> >>>>> Hi folks! >>>>> >>>>> After a short discussion with Kristian, I've created a small >> app with 2 >>>> modules which shows a few problems with mavens incremental build logic. >>>>> And since incremental builds do not work well, people use >>>>> >>>>> $> mvn _clean_ install >>>>> all the time. >>>>> >>>>> We could speed up the development effort heavily if we make >>>>> $> mvn install >>>>> (without an upfront clean) more reliable. >>>>> >>>>> >>>>> The sample [1] consists of moduleA and moduleB with BeanA and >> BeanB >>>> respectively. >>>>> BeanB has a private BeanA field and moduleB has a dependency on >> moduleA. >>>>> >>>>> If I change BeanA and just invoke mvn install then only moduleA >> gets >>>> rebuilt. We currently do not rebuild moduleB, but we should do to >> create a >>>> reliable output. >>>>> >>>>> In fact, the incremental build within a single module already >> works to some >>>> degrees, but we must detect a change in dependencies and trigger a full >> rebuild >>>> on all depending modules. This could be done by storing the md5 of all >>>> dependency artifacts and compare them on the next build. If the md5 of >> a >>>> dependency did change, then we need to build the module full cycle. >>>>> Other ideas are welcome. Slaps as well if I forgot some obvious >> stuff and >>>> all works well already. >>>>> >>>>> >>>>> LieGrue, >>>>> strub >>>>> >>>>> >>>>> >> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >>>>> For additional commands, e-mail: dev-h...@maven.apache.org >>>>> >>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >>>> For additional commands, e-mail: dev-h...@maven.apache.org >>>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >>> For additional commands, e-mail: dev-h...@maven.apache.org >>> >> >> >> >> -- >> Olivier Lamy >> Talend: http://coders.talend.com >> http://twitter.com/olamy | http://linkedin.com/in/olamy >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
