Hi Olivier! I already started tweaking the m-compiler-plugin and found quite scary issues. There is e.g. MCOMPILER-21 (open since 2005) which prevents incremental build and would kick in in your scenario.
I talked about 2 scenarios a.) all phases >= package, e.g. mvn verify, mvn install, ... Here we have an artifact on the disc and could test for the md5 of them, right? b.) all phases < package. This is e.g. mvn compile or mvn test. In that case we don't produce a jar/war/ear/... artifact but only get the files on the disk linked in MavenProject#getProjectArtifacts()... file(). This is the case where the maven-compiler-plugin kicks in. I'm currently in the process of rewriting big parts of it, mainly I introduced b.1 a check for project.artifacts/project.testArtifacts .file() is a local file path .isDirectory() and has files which are newer (actually >=) than the buildStarted timestamp. b.2 check whether there are any *.java (sourceincludes) files which are newer than their class files. In both cases I now force a FULL recompile of the module! Compiling only parts produced classes which are actually broken! My approach is the following: compiling a bit too much is better than missing some files which need compilation. Because the later is exactly the reason why noone uses mvn compile but always do a mvn clean compile nowadays. If mvn compile is reliable, then we will end up being much faster than an unconditional full compile on the whole project! LieGrue, strub PS: We need to move all our plugins which still use the plugin-testing-harness to the maven-invoker-plugin as the test-harness is broken with sisu. (sisu changed the 'container' field from plexus original: protected to 'private') PPS: how do we maintain the plexus-compiler-utils stuff? This contains some weirdo bugs which should get fixed... ----- Original Message ----- > From: Olivier Lamy <ol...@apache.org> > To: Maven Developers List <dev@maven.apache.org>; Mark Struberg > <strub...@yahoo.de> > Cc: > Sent: Monday, August 27, 2012 9:13 AM > Subject: Re: improving incremental builds > > Hi, > Sounds good idea trying to improve that. > My question: on what is based the md5 calculation ? > If people don't use 'install' but only 'test' (perso I use > that to > avoid too much io with jar creation etc..), so in such case we cannot > do md5 calculation on the produced artifact. > > 2012/8/26 Mark Struberg <strub...@yahoo.de>: >> Hi David! >> >>> So your idea is to find the list of changed modules and >>> then build that list with -amd? >> Yes, kind of. >> >> At the moment mvn -amd builds the dependents of the _current_ module. If > you use my example and change BeanA.java, then run mvn -amd from the root > module > you will see that only moduleA gets rebuild and moduleB remains original. > Because moduleB is not a dependent of the root module. >> >> But yes, I'm completely with you that we have most of the ingredients > in the maven codebase already. At least for the start we could easily detect > changed build results and add those to the 'amd' list. That would > already be much better than what we have today imo. And this should be pretty > easy to implement. >> >> Indeed, what I proposed goes a bit beyond -amd. I would not only check the > current build tree like -amd does (even if that would be a good start) but > remember the md5 of all the dependencies of each module (simply store them in > a > file in ./target/) And if you find a dependency which is not in the list > (e.g. > after upgrade from one version to another) or has a different md5 (this > covers > SNAPSHOT versions) , then we could force a full rebuild (mvn -amd) of this > module. >> >> >> LieGrue, >> strub >> >> >> >> ----- Original Message ----- >>> From: David Jencks <david_jen...@yahoo.com> >>> To: Maven Developers List <dev@maven.apache.org> >>> Cc: >>> Sent: Sunday, August 26, 2012 8:34 AM >>> Subject: Re: improving incremental builds >>> >>> Is what you want different from what >>> >>> mvn -amd moduleA >>> >>> does? So your idea is to find the list of changed modules and then > build that >>> list with -amd? >>> >>> thanks >>> david jencks >>> >>> On Aug 25, 2012, at 1:32 PM, Mark Struberg wrote: >>> >>>> Hi folks! >>>> >>>> After a short discussion with Kristian, I've created a small > app with 2 >>> modules which shows a few problems with mavens incremental build logic. >>>> And since incremental builds do not work well, people use >>>> >>>> $> mvn _clean_ install >>>> all the time. >>>> >>>> We could speed up the development effort heavily if we make >>>> $> mvn install >>>> (without an upfront clean) more reliable. >>>> >>>> >>>> The sample [1] consists of moduleA and moduleB with BeanA and > BeanB >>> respectively. >>>> BeanB has a private BeanA field and moduleB has a dependency on > moduleA. >>>> >>>> If I change BeanA and just invoke mvn install then only moduleA > gets >>> rebuilt. We currently do not rebuild moduleB, but we should do to > create a >>> reliable output. >>>> >>>> In fact, the incremental build within a single module already > works to some >>> degrees, but we must detect a change in dependencies and trigger a full > rebuild >>> on all depending modules. This could be done by storing the md5 of all >>> dependency artifacts and compare them on the next build. If the md5 of > a >>> dependency did change, then we need to build the module full cycle. >>>> Other ideas are welcome. Slaps as well if I forgot some obvious > stuff and >>> all works well already. >>>> >>>> >>>> LieGrue, >>>> strub >>>> >>>> >>>> > --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >>>> For additional commands, e-mail: dev-h...@maven.apache.org >>>> >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >>> For additional commands, e-mail: dev-h...@maven.apache.org >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >> For additional commands, e-mail: dev-h...@maven.apache.org >> > > > > -- > Olivier Lamy > Talend: http://coders.talend.com > http://twitter.com/olamy | http://linkedin.com/in/olamy > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
