agree for common framework. Maybe using the existing o.s.p.b.i.BuildContext (or an other new one) others inline.
2012/8/27 Mark Struberg <strub...@yahoo.de>: > +1 for a common framework in maven-shared or similar. > > +0 for auto detecting changed scenarios. If someone changes a profile or the > whole pom, then I'd expect he invokes a clean manually. We have to document > this expectation of course. > > LieGrue, > strub > > > > > ----- Original Message ----- >> From: Anders Hammar <and...@hammar.net> >> To: Maven Developers List <dev@maven.apache.org> >> Cc: >> Sent: Monday, August 27, 2012 9:57 AM >> Subject: Re: improving incremental builds >> >>> I already started tweaking the m-compiler-plugin and found quite scary >> issues. There is e.g. MCOMPILER-21 (open since 2005) which prevents >> incremental >> build and would kick in in your scenario. >> >> This is interesting. I've looked a bit at Mojo's JAXB plugin in the >> context of detecting stale generated files (i.e. need to re-generate) >> and it's similar to this. It would extremely nice if there would be a >> common "framework" for handling incremental builds. >> In addition to what's been mentioned in the jira (I just browsed it >> quickly), we have cases when includes/excludes change, the sourceDir >> changes, etc which should trigger cleanup and re-compilation. >> >> /Anders >> >>> >>> I talked about 2 scenarios >>> >>> a.) all phases >= package, e.g. mvn verify, mvn install, ... Here we >> have an artifact on the disc and could test for the md5 of them, right? >>> >>> b.) all phases < package. This is e.g. mvn compile or mvn test. In that >> case we don't produce a jar/war/ear/... artifact but only get the files on >> the disk linked in MavenProject#getProjectArtifacts()... file(). This is the >> case where the maven-compiler-plugin kicks in. I'm currently in the process >> of rewriting big parts of it, mainly I introduced >>> b.1 a check for project.artifacts/project.testArtifacts .file() is a >> local file path .isDirectory() and has files which are newer (actually >=) >> than the buildStarted timestamp. >>> b.2 check whether there are any *.java (sourceincludes) files which are >> newer than their class files. >>> >>> In both cases I now force a FULL recompile of the module! Compiling only >> parts produced classes which are actually broken! >>> >>> >>> My approach is the following: compiling a bit too much is better than >> missing some files which need compilation. Because the later is exactly the >> reason why noone uses mvn compile but always do a mvn clean compile >> nowadays. If >> mvn compile is reliable, then we will end up being much faster than an >> unconditional full compile on the whole project! >>> >>> >>> LieGrue, >>> strub >>> >>> >>> PS: We need to move all our plugins which still use the >> plugin-testing-harness to the maven-invoker-plugin as the test-harness is >> broken >> with sisu. (sisu changed the 'container' field from plexus original: >> protected to 'private') version 2.0-alpha-1 should fix that (see site plugin) >>> >>> PPS: how do we maintain the plexus-compiler-utils stuff? This contains some >> weirdo bugs which should get fixed... we maintain that see changelog history https://github.com/sonatype/plexus-compiler/commits/master/ bugs which ones ? :-) >>> >>> >>> >>> ----- Original Message ----- >>>> From: Olivier Lamy <ol...@apache.org> >>>> To: Maven Developers List <dev@maven.apache.org>; Mark Struberg >> <strub...@yahoo.de> >>>> Cc: >>>> Sent: Monday, August 27, 2012 9:13 AM >>>> Subject: Re: improving incremental builds >>>> >>>> Hi, >>>> Sounds good idea trying to improve that. >>>> My question: on what is based the md5 calculation ? >>>> If people don't use 'install' but only 'test' >> (perso I use >>>> that to >>>> avoid too much io with jar creation etc..), so in such case we cannot >>>> do md5 calculation on the produced artifact. >>>> >>>> 2012/8/26 Mark Struberg <strub...@yahoo.de>: >>>>> Hi David! >>>>> >>>>>> So your idea is to find the list of changed modules and >>>>>> then build that list with -amd? >>>>> Yes, kind of. >>>>> >>>>> At the moment mvn -amd builds the dependents of the _current_ >> module. If >>>> you use my example and change BeanA.java, then run mvn -amd from the >> root module >>>> you will see that only moduleA gets rebuild and moduleB remains >> original. >>>> Because moduleB is not a dependent of the root module. >>>>> >>>>> But yes, I'm completely with you that we have most of the >> ingredients >>>> in the maven codebase already. At least for the start we could easily >> detect >>>> changed build results and add those to the 'amd' list. That >> would >>>> already be much better than what we have today imo. And this should be >> pretty >>>> easy to implement. >>>>> >>>>> Indeed, what I proposed goes a bit beyond -amd. I would not only >> check the >>>> current build tree like -amd does (even if that would be a good start) >> but >>>> remember the md5 of all the dependencies of each module (simply store >> them in a >>>> file in ./target/) And if you find a dependency which is not in the >> list (e.g. >>>> after upgrade from one version to another) or has a different md5 (this >> covers >>>> SNAPSHOT versions) , then we could force a full rebuild (mvn -amd) of >> this >>>> module. >>>>> >>>>> >>>>> LieGrue, >>>>> strub >>>>> >>>>> >>>>> >>>>> ----- Original Message ----- >>>>>> From: David Jencks <david_jen...@yahoo.com> >>>>>> To: Maven Developers List <dev@maven.apache.org> >>>>>> Cc: >>>>>> Sent: Sunday, August 26, 2012 8:34 AM >>>>>> Subject: Re: improving incremental builds >>>>>> >>>>>> Is what you want different from what >>>>>> >>>>>> mvn -amd moduleA >>>>>> >>>>>> does? So your idea is to find the list of changed modules and >> then >>>> build that >>>>>> list with -amd? >>>>>> >>>>>> thanks >>>>>> david jencks >>>>>> >>>>>> On Aug 25, 2012, at 1:32 PM, Mark Struberg wrote: >>>>>> >>>>>>> Hi folks! >>>>>>> >>>>>>> After a short discussion with Kristian, I've created >> a small >>>> app with 2 >>>>>> modules which shows a few problems with mavens incremental >> build logic. >>>>>>> And since incremental builds do not work well, people use >>>>>>> >>>>>>> $> mvn _clean_ install >>>>>>> all the time. >>>>>>> >>>>>>> We could speed up the development effort heavily if we >> make >>>>>>> $> mvn install >>>>>>> (without an upfront clean) more reliable. >>>>>>> >>>>>>> >>>>>>> The sample [1] consists of moduleA and moduleB with BeanA >> and >>>> BeanB >>>>>> respectively. >>>>>>> BeanB has a private BeanA field and moduleB has a >> dependency on >>>> moduleA. >>>>>>> >>>>>>> If I change BeanA and just invoke mvn install then only >> moduleA >>>> gets >>>>>> rebuilt. We currently do not rebuild moduleB, but we should do >> to >>>> create a >>>>>> reliable output. >>>>>>> >>>>>>> In fact, the incremental build within a single module >> already >>>> works to some >>>>>> degrees, but we must detect a change in dependencies and >> trigger a full >>>> rebuild >>>>>> on all depending modules. This could be done by storing the >> md5 of all >>>>>> dependency artifacts and compare them on the next build. If >> the md5 of >>>> a >>>>>> dependency did change, then we need to build the module full >> cycle. >>>>>>> Other ideas are welcome. Slaps as well if I forgot some >> obvious >>>> stuff and >>>>>> all works well already. >>>>>>> >>>>>>> >>>>>>> LieGrue, >>>>>>> strub >>>>>>> >>>>>>> >>>>>>> >>>> --------------------------------------------------------------------- >>>>>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >>>>>>> For additional commands, e-mail: >> dev-h...@maven.apache.org >>>>>>> >>>>>> >>>>>> >>>>>> >> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >>>>>> For additional commands, e-mail: dev-h...@maven.apache.org >>>>>> >>>>> >>>>> >> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >>>>> For additional commands, e-mail: dev-h...@maven.apache.org >>>>> >>>> >>>> >>>> >>>> -- >>>> Olivier Lamy >>>> Talend: http://coders.talend.com >>>> http://twitter.com/olamy | http://linkedin.com/in/olamy >>>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >>> For additional commands, e-mail: dev-h...@maven.apache.org >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >> For additional commands, e-mail: dev-h...@maven.apache.org >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > -- Olivier Lamy Talend: http://coders.talend.com http://twitter.com/olamy | http://linkedin.com/in/olamy --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
