Re: improving incremental builds
Hi, Sounds good idea trying to improve that. My question: on what is based the md5 calculation ? If people don't use 'install' but only 'test' (perso I use that to avoid too much io with jar creation etc..), so in such case we cannot do md5 calculation on the produced artifact. 2012/8/26 Mark Struberg strub...@yahoo.de: Hi David! So your idea is to find the list of changed modules and then build that list with -amd? Yes, kind of. At the moment mvn -amd builds the dependents of the _current_ module. If you use my example and change BeanA.java, then run mvn -amd from the root module you will see that only moduleA gets rebuild and moduleB remains original. Because moduleB is not a dependent of the root module. But yes, I'm completely with you that we have most of the ingredients in the maven codebase already. At least for the start we could easily detect changed build results and add those to the 'amd' list. That would already be much better than what we have today imo. And this should be pretty easy to implement. Indeed, what I proposed goes a bit beyond -amd. I would not only check the current build tree like -amd does (even if that would be a good start) but remember the md5 of all the dependencies of each module (simply store them in a file in ./target/) And if you find a dependency which is not in the list (e.g. after upgrade from one version to another) or has a different md5 (this covers SNAPSHOT versions) , then we could force a full rebuild (mvn -amd) of this module. LieGrue, strub - Original Message - From: David Jencks david_jen...@yahoo.com To: Maven Developers List dev@maven.apache.org Cc: Sent: Sunday, August 26, 2012 8:34 AM Subject: Re: improving incremental builds Is what you want different from what mvn -amd moduleA does? So your idea is to find the list of changed modules and then build that list with -amd? thanks david jencks On Aug 25, 2012, at 1:32 PM, Mark Struberg wrote: Hi folks! After a short discussion with Kristian, I've created a small app with 2 modules which shows a few problems with mavens incremental build logic. And since incremental builds do not work well, people use $ mvn _clean_ install all the time. We could speed up the development effort heavily if we make $ mvn install (without an upfront clean) more reliable. The sample [1] consists of moduleA and moduleB with BeanA and BeanB respectively. BeanB has a private BeanA field and moduleB has a dependency on moduleA. If I change BeanA and just invoke mvn install then only moduleA gets rebuilt. We currently do not rebuild moduleB, but we should do to create a reliable output. In fact, the incremental build within a single module already works to some degrees, but we must detect a change in dependencies and trigger a full rebuild on all depending modules. This could be done by storing the md5 of all dependency artifacts and compare them on the next build. If the md5 of a dependency did change, then we need to build the module full cycle. Other ideas are welcome. Slaps as well if I forgot some obvious stuff and all works well already. LieGrue, strub - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org -- Olivier Lamy Talend: http://coders.talend.com http://twitter.com/olamy | http://linkedin.com/in/olamy - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: improving incremental builds
Hi Olivier! I already started tweaking the m-compiler-plugin and found quite scary issues. There is e.g. MCOMPILER-21 (open since 2005) which prevents incremental build and would kick in in your scenario. I talked about 2 scenarios a.) all phases = package, e.g. mvn verify, mvn install, ... Here we have an artifact on the disc and could test for the md5 of them, right? b.) all phases package. This is e.g. mvn compile or mvn test. In that case we don't produce a jar/war/ear/... artifact but only get the files on the disk linked in MavenProject#getProjectArtifacts()... file(). This is the case where the maven-compiler-plugin kicks in. I'm currently in the process of rewriting big parts of it, mainly I introduced b.1 a check for project.artifacts/project.testArtifacts .file() is a local file path .isDirectory() and has files which are newer (actually =) than the buildStarted timestamp. b.2 check whether there are any *.java (sourceincludes) files which are newer than their class files. In both cases I now force a FULL recompile of the module! Compiling only parts produced classes which are actually broken! My approach is the following: compiling a bit too much is better than missing some files which need compilation. Because the later is exactly the reason why noone uses mvn compile but always do a mvn clean compile nowadays. If mvn compile is reliable, then we will end up being much faster than an unconditional full compile on the whole project! LieGrue, strub PS: We need to move all our plugins which still use the plugin-testing-harness to the maven-invoker-plugin as the test-harness is broken with sisu. (sisu changed the 'container' field from plexus original: protected to 'private') PPS: how do we maintain the plexus-compiler-utils stuff? This contains some weirdo bugs which should get fixed... - Original Message - From: Olivier Lamy ol...@apache.org To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 9:13 AM Subject: Re: improving incremental builds Hi, Sounds good idea trying to improve that. My question: on what is based the md5 calculation ? If people don't use 'install' but only 'test' (perso I use that to avoid too much io with jar creation etc..), so in such case we cannot do md5 calculation on the produced artifact. 2012/8/26 Mark Struberg strub...@yahoo.de: Hi David! So your idea is to find the list of changed modules and then build that list with -amd? Yes, kind of. At the moment mvn -amd builds the dependents of the _current_ module. If you use my example and change BeanA.java, then run mvn -amd from the root module you will see that only moduleA gets rebuild and moduleB remains original. Because moduleB is not a dependent of the root module. But yes, I'm completely with you that we have most of the ingredients in the maven codebase already. At least for the start we could easily detect changed build results and add those to the 'amd' list. That would already be much better than what we have today imo. And this should be pretty easy to implement. Indeed, what I proposed goes a bit beyond -amd. I would not only check the current build tree like -amd does (even if that would be a good start) but remember the md5 of all the dependencies of each module (simply store them in a file in ./target/) And if you find a dependency which is not in the list (e.g. after upgrade from one version to another) or has a different md5 (this covers SNAPSHOT versions) , then we could force a full rebuild (mvn -amd) of this module. LieGrue, strub - Original Message - From: David Jencks david_jen...@yahoo.com To: Maven Developers List dev@maven.apache.org Cc: Sent: Sunday, August 26, 2012 8:34 AM Subject: Re: improving incremental builds Is what you want different from what mvn -amd moduleA does? So your idea is to find the list of changed modules and then build that list with -amd? thanks david jencks On Aug 25, 2012, at 1:32 PM, Mark Struberg wrote: Hi folks! After a short discussion with Kristian, I've created a small app with 2 modules which shows a few problems with mavens incremental build logic. And since incremental builds do not work well, people use $ mvn _clean_ install all the time. We could speed up the development effort heavily if we make $ mvn install (without an upfront clean) more reliable. The sample [1] consists of moduleA and moduleB with BeanA and BeanB respectively. BeanB has a private BeanA field and moduleB has a dependency on moduleA. If I change BeanA and just invoke mvn install then only moduleA gets rebuilt. We currently do not rebuild moduleB, but we should do to create a reliable output. In fact, the incremental build within a single module already works to some degrees,
Re: [VOTE] Release Maven Project Info Reports Plugin version 2.5.1
1. Source distribution builds with ITs passing [OK] 2. RAT report [OK] There is only one file missing a header in the RAT report, and I am willing to bet that the file format for that file does not support comments. +1 from me -Stephen * Summary --- Generated at: 2012-08-27T08:46:25+01:00 Notes: 2 Binaries: 1 Archives: 0 Standards: 117 Apache Licensed: 116 Generated Documents: 0 JavaDocs are generated and so license header is optional Generated files do not required license headers 1 Unknown Licenses *** Unapproved licenses: src/main/resources/org/apache/maven/report/projectinfo/resources/resources.txt *** On 26 August 2012 21:11, Hervé BOUTEMY herve.bout...@free.fr wrote: Hi, We solved 5 issues: http://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=11142version=18715styleName=Html There are still a couple of issues left in JIRA: http://jira.codehaus.org/secure/IssueNavigator.jspa?reset=truepid=11142status=1 Staging repo: https://repository.apache.org/content/repositories/maven-013/ https://repository.apache.org/content/repositories/maven-013/org/apache/maven/plugins/maven- project-info-reports-plugin/2.5.1/maven-project-info-reports-plugin-2.5.1- source-release.zip Staging site: (sync pending) http://maven.apache.org/plugins/maven-project-info-reports-plugin-2.5.1/ Guide to testing staged releases: http://maven.apache.org/guides/development/guide-testing-releases.html Vote open for 72 hours. [ ] +1 [ ] +0 [ ] -1 - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: improving incremental builds
I already started tweaking the m-compiler-plugin and found quite scary issues. There is e.g. MCOMPILER-21 (open since 2005) which prevents incremental build and would kick in in your scenario. This is interesting. I've looked a bit at Mojo's JAXB plugin in the context of detecting stale generated files (i.e. need to re-generate) and it's similar to this. It would extremely nice if there would be a common framework for handling incremental builds. In addition to what's been mentioned in the jira (I just browsed it quickly), we have cases when includes/excludes change, the sourceDir changes, etc which should trigger cleanup and re-compilation. /Anders I talked about 2 scenarios a.) all phases = package, e.g. mvn verify, mvn install, ... Here we have an artifact on the disc and could test for the md5 of them, right? b.) all phases package. This is e.g. mvn compile or mvn test. In that case we don't produce a jar/war/ear/... artifact but only get the files on the disk linked in MavenProject#getProjectArtifacts()... file(). This is the case where the maven-compiler-plugin kicks in. I'm currently in the process of rewriting big parts of it, mainly I introduced b.1 a check for project.artifacts/project.testArtifacts .file() is a local file path .isDirectory() and has files which are newer (actually =) than the buildStarted timestamp. b.2 check whether there are any *.java (sourceincludes) files which are newer than their class files. In both cases I now force a FULL recompile of the module! Compiling only parts produced classes which are actually broken! My approach is the following: compiling a bit too much is better than missing some files which need compilation. Because the later is exactly the reason why noone uses mvn compile but always do a mvn clean compile nowadays. If mvn compile is reliable, then we will end up being much faster than an unconditional full compile on the whole project! LieGrue, strub PS: We need to move all our plugins which still use the plugin-testing-harness to the maven-invoker-plugin as the test-harness is broken with sisu. (sisu changed the 'container' field from plexus original: protected to 'private') PPS: how do we maintain the plexus-compiler-utils stuff? This contains some weirdo bugs which should get fixed... - Original Message - From: Olivier Lamy ol...@apache.org To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 9:13 AM Subject: Re: improving incremental builds Hi, Sounds good idea trying to improve that. My question: on what is based the md5 calculation ? If people don't use 'install' but only 'test' (perso I use that to avoid too much io with jar creation etc..), so in such case we cannot do md5 calculation on the produced artifact. 2012/8/26 Mark Struberg strub...@yahoo.de: Hi David! So your idea is to find the list of changed modules and then build that list with -amd? Yes, kind of. At the moment mvn -amd builds the dependents of the _current_ module. If you use my example and change BeanA.java, then run mvn -amd from the root module you will see that only moduleA gets rebuild and moduleB remains original. Because moduleB is not a dependent of the root module. But yes, I'm completely with you that we have most of the ingredients in the maven codebase already. At least for the start we could easily detect changed build results and add those to the 'amd' list. That would already be much better than what we have today imo. And this should be pretty easy to implement. Indeed, what I proposed goes a bit beyond -amd. I would not only check the current build tree like -amd does (even if that would be a good start) but remember the md5 of all the dependencies of each module (simply store them in a file in ./target/) And if you find a dependency which is not in the list (e.g. after upgrade from one version to another) or has a different md5 (this covers SNAPSHOT versions) , then we could force a full rebuild (mvn -amd) of this module. LieGrue, strub - Original Message - From: David Jencks david_jen...@yahoo.com To: Maven Developers List dev@maven.apache.org Cc: Sent: Sunday, August 26, 2012 8:34 AM Subject: Re: improving incremental builds Is what you want different from what mvn -amd moduleA does? So your idea is to find the list of changed modules and then build that list with -amd? thanks david jencks On Aug 25, 2012, at 1:32 PM, Mark Struberg wrote: Hi folks! After a short discussion with Kristian, I've created a small app with 2 modules which shows a few problems with mavens incremental build logic. And since incremental builds do not work well, people use $ mvn _clean_ install all the time. We could speed up the development effort heavily if we make $ mvn install (without an upfront clean) more
Re: [VOTE] Release Maven Project Info Reports Plugin version 2.5.1
+1 key looks fine, source builds fine, reports ok in my project. LieGrue, strub - Original Message - From: Stephen Connolly stephen.alan.conno...@gmail.com To: Maven Developers List dev@maven.apache.org Cc: Sent: Monday, August 27, 2012 9:53 AM Subject: Re: [VOTE] Release Maven Project Info Reports Plugin version 2.5.1 1. Source distribution builds with ITs passing [OK] 2. RAT report [OK] There is only one file missing a header in the RAT report, and I am willing to bet that the file format for that file does not support comments. +1 from me -Stephen * Summary --- Generated at: 2012-08-27T08:46:25+01:00 Notes: 2 Binaries: 1 Archives: 0 Standards: 117 Apache Licensed: 116 Generated Documents: 0 JavaDocs are generated and so license header is optional Generated files do not required license headers 1 Unknown Licenses *** Unapproved licenses: src/main/resources/org/apache/maven/report/projectinfo/resources/resources.txt *** On 26 August 2012 21:11, Hervé BOUTEMY herve.bout...@free.fr wrote: Hi, We solved 5 issues: http://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=11142version=18715styleName=Html There are still a couple of issues left in JIRA: http://jira.codehaus.org/secure/IssueNavigator.jspa?reset=truepid=11142status=1 Staging repo: https://repository.apache.org/content/repositories/maven-013/ https://repository.apache.org/content/repositories/maven-013/org/apache/maven/plugins/maven- project-info-reports-plugin/2.5.1/maven-project-info-reports-plugin-2.5.1- source-release.zip Staging site: (sync pending) http://maven.apache.org/plugins/maven-project-info-reports-plugin-2.5.1/ Guide to testing staged releases: http://maven.apache.org/guides/development/guide-testing-releases.html Vote open for 72 hours. [ ] +1 [ ] +0 [ ] -1 - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: improving incremental builds
+1 for a common framework in maven-shared or similar. +0 for auto detecting changed scenarios. If someone changes a profile or the whole pom, then I'd expect he invokes a clean manually. We have to document this expectation of course. LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org Cc: Sent: Monday, August 27, 2012 9:57 AM Subject: Re: improving incremental builds I already started tweaking the m-compiler-plugin and found quite scary issues. There is e.g. MCOMPILER-21 (open since 2005) which prevents incremental build and would kick in in your scenario. This is interesting. I've looked a bit at Mojo's JAXB plugin in the context of detecting stale generated files (i.e. need to re-generate) and it's similar to this. It would extremely nice if there would be a common framework for handling incremental builds. In addition to what's been mentioned in the jira (I just browsed it quickly), we have cases when includes/excludes change, the sourceDir changes, etc which should trigger cleanup and re-compilation. /Anders I talked about 2 scenarios a.) all phases = package, e.g. mvn verify, mvn install, ... Here we have an artifact on the disc and could test for the md5 of them, right? b.) all phases package. This is e.g. mvn compile or mvn test. In that case we don't produce a jar/war/ear/... artifact but only get the files on the disk linked in MavenProject#getProjectArtifacts()... file(). This is the case where the maven-compiler-plugin kicks in. I'm currently in the process of rewriting big parts of it, mainly I introduced b.1 a check for project.artifacts/project.testArtifacts .file() is a local file path .isDirectory() and has files which are newer (actually =) than the buildStarted timestamp. b.2 check whether there are any *.java (sourceincludes) files which are newer than their class files. In both cases I now force a FULL recompile of the module! Compiling only parts produced classes which are actually broken! My approach is the following: compiling a bit too much is better than missing some files which need compilation. Because the later is exactly the reason why noone uses mvn compile but always do a mvn clean compile nowadays. If mvn compile is reliable, then we will end up being much faster than an unconditional full compile on the whole project! LieGrue, strub PS: We need to move all our plugins which still use the plugin-testing-harness to the maven-invoker-plugin as the test-harness is broken with sisu. (sisu changed the 'container' field from plexus original: protected to 'private') PPS: how do we maintain the plexus-compiler-utils stuff? This contains some weirdo bugs which should get fixed... - Original Message - From: Olivier Lamy ol...@apache.org To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 9:13 AM Subject: Re: improving incremental builds Hi, Sounds good idea trying to improve that. My question: on what is based the md5 calculation ? If people don't use 'install' but only 'test' (perso I use that to avoid too much io with jar creation etc..), so in such case we cannot do md5 calculation on the produced artifact. 2012/8/26 Mark Struberg strub...@yahoo.de: Hi David! So your idea is to find the list of changed modules and then build that list with -amd? Yes, kind of. At the moment mvn -amd builds the dependents of the _current_ module. If you use my example and change BeanA.java, then run mvn -amd from the root module you will see that only moduleA gets rebuild and moduleB remains original. Because moduleB is not a dependent of the root module. But yes, I'm completely with you that we have most of the ingredients in the maven codebase already. At least for the start we could easily detect changed build results and add those to the 'amd' list. That would already be much better than what we have today imo. And this should be pretty easy to implement. Indeed, what I proposed goes a bit beyond -amd. I would not only check the current build tree like -amd does (even if that would be a good start) but remember the md5 of all the dependencies of each module (simply store them in a file in ./target/) And if you find a dependency which is not in the list (e.g. after upgrade from one version to another) or has a different md5 (this covers SNAPSHOT versions) , then we could force a full rebuild (mvn -amd) of this module. LieGrue, strub - Original Message - From: David Jencks david_jen...@yahoo.com To: Maven Developers List dev@maven.apache.org Cc: Sent: Sunday, August 26, 2012 8:34 AM Subject: Re: improving incremental builds Is what you want different from what
Re: [VOTE] Apache Maven Dependency Plugin 2.5.1
+1 2012/8/24 Olivier Lamy ol...@apache.org: Hi, I'd like to release Maven Dependency Plugin 2.5.1. This release contains a regression issue fix and a perf improvement (https://jira.codehaus.org/browse/MDEP/fixforversion/18718) Staging repository: https://repository.apache.org/content/repositories/maven-003/ Staging site: http://maven.apache.org/plugins/maven-dependency-plugin-2.5.1 (wait sync) Guide to testing staged releases: http://maven.apache.org/guides/development/guide-testing-releases.html Vote open for 72 hours. [ ] +1 [ ] +0 [ ] -1 Thanks, -- Olivier Lamy Talend: http://coders.talend.com http://twitter.com/olamy | http://linkedin.com/in/olamy -- Olivier Lamy Talend: http://coders.talend.com http://twitter.com/olamy | http://linkedin.com/in/olamy - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
[RESULT][VOTE] Apache Maven Dependency Plugin 2.5.1
Hi, The vote has passed with the following result. +1 (binding): Stephen Connolly, Kristian Rosenvold, Stephan Nicoll, Hervé Boutemy, Olivier Lamy +1 (non binding): Keith Sader, Tony Chemit I will continue the release process. Thanks! -- Olivier Lamy Talend: http://coders.talend.com http://twitter.com/olamy | http://linkedin.com/in/olamy - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: improving incremental builds
agree for common framework. Maybe using the existing o.s.p.b.i.BuildContext (or an other new one) others inline. 2012/8/27 Mark Struberg strub...@yahoo.de: +1 for a common framework in maven-shared or similar. +0 for auto detecting changed scenarios. If someone changes a profile or the whole pom, then I'd expect he invokes a clean manually. We have to document this expectation of course. LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org Cc: Sent: Monday, August 27, 2012 9:57 AM Subject: Re: improving incremental builds I already started tweaking the m-compiler-plugin and found quite scary issues. There is e.g. MCOMPILER-21 (open since 2005) which prevents incremental build and would kick in in your scenario. This is interesting. I've looked a bit at Mojo's JAXB plugin in the context of detecting stale generated files (i.e. need to re-generate) and it's similar to this. It would extremely nice if there would be a common framework for handling incremental builds. In addition to what's been mentioned in the jira (I just browsed it quickly), we have cases when includes/excludes change, the sourceDir changes, etc which should trigger cleanup and re-compilation. /Anders I talked about 2 scenarios a.) all phases = package, e.g. mvn verify, mvn install, ... Here we have an artifact on the disc and could test for the md5 of them, right? b.) all phases package. This is e.g. mvn compile or mvn test. In that case we don't produce a jar/war/ear/... artifact but only get the files on the disk linked in MavenProject#getProjectArtifacts()... file(). This is the case where the maven-compiler-plugin kicks in. I'm currently in the process of rewriting big parts of it, mainly I introduced b.1 a check for project.artifacts/project.testArtifacts .file() is a local file path .isDirectory() and has files which are newer (actually =) than the buildStarted timestamp. b.2 check whether there are any *.java (sourceincludes) files which are newer than their class files. In both cases I now force a FULL recompile of the module! Compiling only parts produced classes which are actually broken! My approach is the following: compiling a bit too much is better than missing some files which need compilation. Because the later is exactly the reason why noone uses mvn compile but always do a mvn clean compile nowadays. If mvn compile is reliable, then we will end up being much faster than an unconditional full compile on the whole project! LieGrue, strub PS: We need to move all our plugins which still use the plugin-testing-harness to the maven-invoker-plugin as the test-harness is broken with sisu. (sisu changed the 'container' field from plexus original: protected to 'private') version 2.0-alpha-1 should fix that (see site plugin) PPS: how do we maintain the plexus-compiler-utils stuff? This contains some weirdo bugs which should get fixed... we maintain that see changelog history https://github.com/sonatype/plexus-compiler/commits/master/ bugs which ones ? :-) - Original Message - From: Olivier Lamy ol...@apache.org To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 9:13 AM Subject: Re: improving incremental builds Hi, Sounds good idea trying to improve that. My question: on what is based the md5 calculation ? If people don't use 'install' but only 'test' (perso I use that to avoid too much io with jar creation etc..), so in such case we cannot do md5 calculation on the produced artifact. 2012/8/26 Mark Struberg strub...@yahoo.de: Hi David! So your idea is to find the list of changed modules and then build that list with -amd? Yes, kind of. At the moment mvn -amd builds the dependents of the _current_ module. If you use my example and change BeanA.java, then run mvn -amd from the root module you will see that only moduleA gets rebuild and moduleB remains original. Because moduleB is not a dependent of the root module. But yes, I'm completely with you that we have most of the ingredients in the maven codebase already. At least for the start we could easily detect changed build results and add those to the 'amd' list. That would already be much better than what we have today imo. And this should be pretty easy to implement. Indeed, what I proposed goes a bit beyond -amd. I would not only check the current build tree like -amd does (even if that would be a good start) but remember the md5 of all the dependencies of each module (simply store them in a file in ./target/) And if you find a dependency which is not in the list (e.g. after upgrade from one version to another) or has a different md5 (this covers SNAPSHOT versions) , then we could force a full rebuild (mvn -amd) of this module.
Re: [VOTE] Release Maven Project Info Reports Plugin version 2.5.1
+1 S. On Sun, Aug 26, 2012 at 10:11 PM, Hervé BOUTEMY herve.bout...@free.fr wrote: Hi, We solved 5 issues: http://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=11142version=18715styleName=Html There are still a couple of issues left in JIRA: http://jira.codehaus.org/secure/IssueNavigator.jspa?reset=truepid=11142status=1 Staging repo: https://repository.apache.org/content/repositories/maven-013/ https://repository.apache.org/content/repositories/maven-013/org/apache/maven/plugins/maven- project-info-reports-plugin/2.5.1/maven-project-info-reports-plugin-2.5.1- source-release.zip Staging site: (sync pending) http://maven.apache.org/plugins/maven-project-info-reports-plugin-2.5.1/ Guide to testing staged releases: http://maven.apache.org/guides/development/guide-testing-releases.html Vote open for 72 hours. [ ] +1 [ ] +0 [ ] -1 - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: [VOTE] Release Maven Project Info Reports Plugin version 2.5.1
+1 2012/8/26 Hervé BOUTEMY herve.bout...@free.fr: Hi, We solved 5 issues: http://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=11142version=18715styleName=Html There are still a couple of issues left in JIRA: http://jira.codehaus.org/secure/IssueNavigator.jspa?reset=truepid=11142status=1 Staging repo: https://repository.apache.org/content/repositories/maven-013/ https://repository.apache.org/content/repositories/maven-013/org/apache/maven/plugins/maven- project-info-reports-plugin/2.5.1/maven-project-info-reports-plugin-2.5.1- source-release.zip Staging site: (sync pending) http://maven.apache.org/plugins/maven-project-info-reports-plugin-2.5.1/ Guide to testing staged releases: http://maven.apache.org/guides/development/guide-testing-releases.html Vote open for 72 hours. [ ] +1 [ ] +0 [ ] -1 - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org -- Olivier Lamy Talend: http://coders.talend.com http://twitter.com/olamy | http://linkedin.com/in/olamy - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: [VOTE] Release Maven Surefire version 2.12.3
+1 2012/8/25 Kristian Rosenvold kristian.rosenv...@gmail.com: Hi, We solved 11 issues: https://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=10541version=18704 2.12.3 should be slightly faster than previous versions, but don't expect too much. There are still lots of issues left in JIRA: http://jira.codehaus.org/secure/IssueNavigator.jspa?reset=truemode=hidejqlQuery=project+%3D+SUREFIRE+AND+resolution+%3D+Unresolved+ORDER+BY+updated+DESC Staging repo: https://repository.apache.org/content/repositories/maven-011/ Staging site: (Sync pending) http://maven.apache.org/surefire/staging/ (Links to sub-sites work when transfered to production) http://maven.apache.org/plugins/maven-failsafe-plugin-2.12.3/ http://maven.apache.org/plugins/maven-surefire-plugin-2.12.3/ http://maven.apache.org/plugins/maven-surefire-reports-plugin-2.12.3/ Guide to testing staged releases: http://maven.apache.org/guides/development/guide-testing-releases.html Vote open for 72 hours. [ ] +1 [ ] +0 [ ] -1 And here's my +1 Kristian - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org -- Olivier Lamy Talend: http://coders.talend.com http://twitter.com/olamy | http://linkedin.com/in/olamy - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: improving incremental builds
+0 for auto detecting changed scenarios. If someone changes a profile or the whole pom, then I'd expect he invokes a clean manually. We have to document this expectation of course. What do others think of this? I'm thinking it would be awesome (but maybe difficult) if plugins could determine if its configuration has changed, and then handle this automatically. If there are to many cases where a manual clean build is required, I believe people will continue to do mvn clean install (which I see all the time with developers), which I think is what we try to get away from. /Anders LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org Cc: Sent: Monday, August 27, 2012 9:57 AM Subject: Re: improving incremental builds I already started tweaking the m-compiler-plugin and found quite scary issues. There is e.g. MCOMPILER-21 (open since 2005) which prevents incremental build and would kick in in your scenario. This is interesting. I've looked a bit at Mojo's JAXB plugin in the context of detecting stale generated files (i.e. need to re-generate) and it's similar to this. It would extremely nice if there would be a common framework for handling incremental builds. In addition to what's been mentioned in the jira (I just browsed it quickly), we have cases when includes/excludes change, the sourceDir changes, etc which should trigger cleanup and re-compilation. /Anders I talked about 2 scenarios a.) all phases = package, e.g. mvn verify, mvn install, ... Here we have an artifact on the disc and could test for the md5 of them, right? b.) all phases package. This is e.g. mvn compile or mvn test. In that case we don't produce a jar/war/ear/... artifact but only get the files on the disk linked in MavenProject#getProjectArtifacts()... file(). This is the case where the maven-compiler-plugin kicks in. I'm currently in the process of rewriting big parts of it, mainly I introduced b.1 a check for project.artifacts/project.testArtifacts .file() is a local file path .isDirectory() and has files which are newer (actually =) than the buildStarted timestamp. b.2 check whether there are any *.java (sourceincludes) files which are newer than their class files. In both cases I now force a FULL recompile of the module! Compiling only parts produced classes which are actually broken! My approach is the following: compiling a bit too much is better than missing some files which need compilation. Because the later is exactly the reason why noone uses mvn compile but always do a mvn clean compile nowadays. If mvn compile is reliable, then we will end up being much faster than an unconditional full compile on the whole project! LieGrue, strub PS: We need to move all our plugins which still use the plugin-testing-harness to the maven-invoker-plugin as the test-harness is broken with sisu. (sisu changed the 'container' field from plexus original: protected to 'private') PPS: how do we maintain the plexus-compiler-utils stuff? This contains some weirdo bugs which should get fixed... - Original Message - From: Olivier Lamy ol...@apache.org To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 9:13 AM Subject: Re: improving incremental builds Hi, Sounds good idea trying to improve that. My question: on what is based the md5 calculation ? If people don't use 'install' but only 'test' (perso I use that to avoid too much io with jar creation etc..), so in such case we cannot do md5 calculation on the produced artifact. 2012/8/26 Mark Struberg strub...@yahoo.de: Hi David! So your idea is to find the list of changed modules and then build that list with -amd? Yes, kind of. At the moment mvn -amd builds the dependents of the _current_ module. If you use my example and change BeanA.java, then run mvn -amd from the root module you will see that only moduleA gets rebuild and moduleB remains original. Because moduleB is not a dependent of the root module. But yes, I'm completely with you that we have most of the ingredients in the maven codebase already. At least for the start we could easily detect changed build results and add those to the 'amd' list. That would already be much better than what we have today imo. And this should be pretty easy to implement. Indeed, what I proposed goes a bit beyond -amd. I would not only check the current build tree like -amd does (even if that would be a good start) but remember the md5 of all the dependencies of each module (simply store them in a file in ./target/) And if you find a dependency which is not in the list (e.g. after upgrade from one version to another) or has a different md5 (this covers SNAPSHOT versions) , then we could force a full rebuild (mvn -amd) of this
Re: improving incremental builds
Hi, The full invremental is great but not always possible (think to assembly plugin, some case xill be very hard to handle in snapshot mode i guess) Maybe it is time to get a maven daemon to help to be able to store information? - Romain Le 27 août 2012 12:24, Anders Hammar and...@hammar.net a écrit : +0 for auto detecting changed scenarios. If someone changes a profile or the whole pom, then I'd expect he invokes a clean manually. We have to document this expectation of course. What do others think of this? I'm thinking it would be awesome (but maybe difficult) if plugins could determine if its configuration has changed, and then handle this automatically. If there are to many cases where a manual clean build is required, I believe people will continue to do mvn clean install (which I see all the time with developers), which I think is what we try to get away from. /Anders LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org Cc: Sent: Monday, August 27, 2012 9:57 AM Subject: Re: improving incremental builds I already started tweaking the m-compiler-plugin and found quite scary issues. There is e.g. MCOMPILER-21 (open since 2005) which prevents incremental build and would kick in in your scenario. This is interesting. I've looked a bit at Mojo's JAXB plugin in the context of detecting stale generated files (i.e. need to re-generate) and it's similar to this. It would extremely nice if there would be a common framework for handling incremental builds. In addition to what's been mentioned in the jira (I just browsed it quickly), we have cases when includes/excludes change, the sourceDir changes, etc which should trigger cleanup and re-compilation. /Anders I talked about 2 scenarios a.) all phases = package, e.g. mvn verify, mvn install, ... Here we have an artifact on the disc and could test for the md5 of them, right? b.) all phases package. This is e.g. mvn compile or mvn test. In that case we don't produce a jar/war/ear/... artifact but only get the files on the disk linked in MavenProject#getProjectArtifacts()... file(). This is the case where the maven-compiler-plugin kicks in. I'm currently in the process of rewriting big parts of it, mainly I introduced b.1 a check for project.artifacts/project.testArtifacts .file() is a local file path .isDirectory() and has files which are newer (actually =) than the buildStarted timestamp. b.2 check whether there are any *.java (sourceincludes) files which are newer than their class files. In both cases I now force a FULL recompile of the module! Compiling only parts produced classes which are actually broken! My approach is the following: compiling a bit too much is better than missing some files which need compilation. Because the later is exactly the reason why noone uses mvn compile but always do a mvn clean compile nowadays. If mvn compile is reliable, then we will end up being much faster than an unconditional full compile on the whole project! LieGrue, strub PS: We need to move all our plugins which still use the plugin-testing-harness to the maven-invoker-plugin as the test-harness is broken with sisu. (sisu changed the 'container' field from plexus original: protected to 'private') PPS: how do we maintain the plexus-compiler-utils stuff? This contains some weirdo bugs which should get fixed... - Original Message - From: Olivier Lamy ol...@apache.org To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 9:13 AM Subject: Re: improving incremental builds Hi, Sounds good idea trying to improve that. My question: on what is based the md5 calculation ? If people don't use 'install' but only 'test' (perso I use that to avoid too much io with jar creation etc..), so in such case we cannot do md5 calculation on the produced artifact. 2012/8/26 Mark Struberg strub...@yahoo.de: Hi David! So your idea is to find the list of changed modules and then build that list with -amd? Yes, kind of. At the moment mvn -amd builds the dependents of the _current_ module. If you use my example and change BeanA.java, then run mvn -amd from the root module you will see that only moduleA gets rebuild and moduleB remains original. Because moduleB is not a dependent of the root module. But yes, I'm completely with you that we have most of the ingredients in the maven codebase already. At least for the start we could easily detect changed build results and add those to the 'amd' list. That would already be much better than what we have today imo. And this should be pretty easy to implement. Indeed, what I proposed goes a bit beyond -amd. I
Re: [VOTE] Release Maven Surefire version 2.12.3
On Sat, 25 Aug 2012 00:11:55 +0200 Kristian Rosenvold kristian.rosenv...@gmail.com wrote: +1 works fine to me, thanks, tony. Hi, We solved 11 issues: https://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=10541version=18704 2.12.3 should be slightly faster than previous versions, but don't expect too much. There are still lots of issues left in JIRA: http://jira.codehaus.org/secure/IssueNavigator.jspa?reset=truemode=hidejqlQuery=project+%3D+SUREFIRE+AND+resolution+%3D+Unresolved+ORDER+BY+updated+DESC Staging repo: https://repository.apache.org/content/repositories/maven-011/ Staging site: (Sync pending) http://maven.apache.org/surefire/staging/ (Links to sub-sites work when transfered to production) http://maven.apache.org/plugins/maven-failsafe-plugin-2.12.3/ http://maven.apache.org/plugins/maven-surefire-plugin-2.12.3/ http://maven.apache.org/plugins/maven-surefire-reports-plugin-2.12.3/ Guide to testing staged releases: http://maven.apache.org/guides/development/guide-testing-releases.html Vote open for 72 hours. [ ] +1 [ ] +0 [ ] -1 And here's my +1 Kristian - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org -- Tony Chemit tél: +33 (0) 2 40 50 29 28 email: che...@codelutin.com http://www.codelutin.com - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: [VOTE] Release Maven Project Info Reports Plugin version 2.5.1
+1 -Lukas Hervé BOUTEMY wrote: Hi, We solved 5 issues: http://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=11142version=18715styleName=Html There are still a couple of issues left in JIRA: http://jira.codehaus.org/secure/IssueNavigator.jspa?reset=truepid=11142status=1 Staging repo: https://repository.apache.org/content/repositories/maven-013/ https://repository.apache.org/content/repositories/maven-013/org/apache/maven/plugins/maven- project-info-reports-plugin/2.5.1/maven-project-info-reports-plugin-2.5.1- source-release.zip Staging site: (sync pending) http://maven.apache.org/plugins/maven-project-info-reports-plugin-2.5.1/ Guide to testing staged releases: http://maven.apache.org/guides/development/guide-testing-releases.html Vote open for 72 hours. [ ] +1 [ ] +0 [ ] -1 - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: improving incremental builds
The assembly plugin example is perfectly fine if the dependencies are defined in a dependency section. If a SNAPSHOT dependency did change, then it will get a different md5. Of course if plugins resolve some dependencies programmatically, then those cases will not work easily. But that is considered bad practice anyway, isn't? There are clearly 2 different aspects we need to check/implement to get this fully working: a.) artifact dependencies md5 changed - invoke clean for each downstream module b.) each plugin must check itself if a required class changed and then update the output accordingly. If all plugins in the chain do this properly then we will do much better already. I'm currently focusing to get b.) implemented in the maven-compiler-plugin LieGrue, strub - Original Message - From: Romain Manni-Bucau rmannibu...@gmail.com To: Maven Developers List dev@maven.apache.org Cc: Sent: Monday, August 27, 2012 12:28 PM Subject: Re: improving incremental builds Hi, The full invremental is great but not always possible (think to assembly plugin, some case xill be very hard to handle in snapshot mode i guess) Maybe it is time to get a maven daemon to help to be able to store information? - Romain Le 27 août 2012 12:24, Anders Hammar and...@hammar.net a écrit : +0 for auto detecting changed scenarios. If someone changes a profile or the whole pom, then I'd expect he invokes a clean manually. We have to document this expectation of course. What do others think of this? I'm thinking it would be awesome (but maybe difficult) if plugins could determine if its configuration has changed, and then handle this automatically. If there are to many cases where a manual clean build is required, I believe people will continue to do mvn clean install (which I see all the time with developers), which I think is what we try to get away from. /Anders LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org Cc: Sent: Monday, August 27, 2012 9:57 AM Subject: Re: improving incremental builds I already started tweaking the m-compiler-plugin and found quite scary issues. There is e.g. MCOMPILER-21 (open since 2005) which prevents incremental build and would kick in in your scenario. This is interesting. I've looked a bit at Mojo's JAXB plugin in the context of detecting stale generated files (i.e. need to re-generate) and it's similar to this. It would extremely nice if there would be a common framework for handling incremental builds. In addition to what's been mentioned in the jira (I just browsed it quickly), we have cases when includes/excludes change, the sourceDir changes, etc which should trigger cleanup and re-compilation. /Anders I talked about 2 scenarios a.) all phases = package, e.g. mvn verify, mvn install, ... Here we have an artifact on the disc and could test for the md5 of them, right? b.) all phases package. This is e.g. mvn compile or mvn test. In that case we don't produce a jar/war/ear/... artifact but only get the files on the disk linked in MavenProject#getProjectArtifacts()... file(). This is the case where the maven-compiler-plugin kicks in. I'm currently in the process of rewriting big parts of it, mainly I introduced b.1 a check for project.artifacts/project.testArtifacts .file() is a local file path .isDirectory() and has files which are newer (actually =) than the buildStarted timestamp. b.2 check whether there are any *.java (sourceincludes) files which are newer than their class files. In both cases I now force a FULL recompile of the module! Compiling only parts produced classes which are actually broken! My approach is the following: compiling a bit too much is better than missing some files which need compilation. Because the later is exactly the reason why noone uses mvn compile but always do a mvn clean compile nowadays. If mvn compile is reliable, then we will end up being much faster than an unconditional full compile on the whole project! LieGrue, strub PS: We need to move all our plugins which still use the plugin-testing-harness to the maven-invoker-plugin as the test-harness is broken with sisu. (sisu changed the 'container' field from plexus original: protected to 'private') PPS: how do we maintain the plexus-compiler-utils stuff? This contains some weirdo bugs which should get fixed... - Original Message - From: Olivier Lamy ol...@apache.org To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 9:13 AM Subject: Re: improving incremental builds
[VOTE] Maven Skins parent 7 and Maven Fluido skin 1.3.0
Hi, We solved 10 issues: http://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=11430styleName=Htmlversion=18713 Staging repo: https://repository.apache.org/content/repositories/maven-015/ Staging site: http://maven.apache.org/skins/maven-fluido-skin-1.3.0/ Guide to testing staged releases: http://maven.apache.org/guides/development/guide-testing-releases.html Vote open for 72 hours. [ ] +1 [ ] +0 [ ] -1 -- Olivier Lamy Talend: http://coders.talend.com http://twitter.com/olamy | http://linkedin.com/in/olamy - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: [VOTE] Maven Skins parent 7 and Maven Fluido skin 1.3.0
+1!!! thanks, -Simo http://people.apache.org/~simonetripodi/ http://simonetripodi.livejournal.com/ http://twitter.com/simonetripodi http://www.99soft.org/ On Mon, Aug 27, 2012 at 3:13 PM, Olivier Lamy ol...@apache.org wrote: Hi, We solved 10 issues: http://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=11430styleName=Htmlversion=18713 Staging repo: https://repository.apache.org/content/repositories/maven-015/ Staging site: http://maven.apache.org/skins/maven-fluido-skin-1.3.0/ Guide to testing staged releases: http://maven.apache.org/guides/development/guide-testing-releases.html Vote open for 72 hours. [ ] +1 [ ] +0 [ ] -1 -- Olivier Lamy Talend: http://coders.talend.com http://twitter.com/olamy | http://linkedin.com/in/olamy - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
[ANN] Maven Dependency Plugin 2.5.1 Released
Hi, The Apache Maven team is pleased to announce the release of the Maven Dependency Plugin, version 2.5.1 The dependency plugin provides the capability to manipulate artifacts. It can copy and/or unpack artifacts from local or remote repositories to a specified location. http://maven.apache.org/plugins/maven-dependency-plugin/ You should specify the version in your project's plugin configuration: plugin groupIdorg.apache.maven.plugins/groupId artifactIdmaven-dependency-plugin/artifactId version2.5.1/version /plugin Release Notes - Maven 2.x Dependency Plugin - Version 2.5.1 ** Bug * [MDEP-365] - includes/excludes doesn't work anymore with unpack-dependencies 2.5 ** Improvement * [MDEP-368] - Configure plexus-archiver to use java.io.File methods to set permissions when available. Have Fun! -- The Apache Maven team - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
maven-enforcer pull request: [MENFORCER-138] Rule to ban all transitive dep...
GitHub user jsenko opened a pull request: https://github.com/apache/maven-enforcer/pull/2 [MENFORCER-138] Rule to ban all transitive dependencies In some projects it's desirable to have all dependencies specified in pom. It would be nice to have an enforcer rule that would ban all transitive dependencies so that the user could either add the transitive dependency directly to the pom (if it's actually needed), or exclude the dependency. http://jira.codehaus.org/browse/MENFORCER-138 This rule will ban all transitive dependencies, unless configured otherwise. The configuration works identically to http://maven.apache.org/enforcer/enforcer-rules/bannedDependencies.html . User can exclude certain dependencies from being checked (anywhere in the dependency tree) add also add exceptions to the excludes for finer configuration (see javadoc for details). If it fails, the rule will generate message showing dependencies which caused the failure and their descendants in a tree, with info which dependencies were excluded in that branch. Alternatively, custom message can be used. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jsenko/maven-enforcer trunk Alternatively you can review and apply these changes as the patch at: https://github.com/apache/maven-enforcer/pull/2.patch commit d7ba589ea198146a3a5aabd4b202557be1d852cc Author: jsenko jse...@redhat.com Date: 2012-08-24T04:34:21-07:00 Updated pom.xml commit 8fdb711023c97b1e7dce0b88505f569ed64b8d72 Author: jsenko jse...@redhat.com Date: 2012-08-24T04:35:04-07:00 Created ArtifactMatcher commit bdc5c66050218a2dcbf8760080055ff2658f3d75 Author: jsenko jse...@redhat.com Date: 2012-08-24T04:36:10-07:00 Created BanTransitiveDependencies enforcer rule commit b481246a38afa19cd7134aaf9745740a4f9894d5 Author: jsenko jse...@redhat.com Date: 2012-08-27T06:48:52-07:00 Updated BanTransitiveDependencies commit 2f4767498259134b7bf91054795faff48659ea53 Author: jsenko jse...@redhat.com Date: 2012-08-27T06:49:43-07:00 Updated ArtifactMatcher commit 8f321d1070624668a1c68879f868702ff4735aaf Author: jsenko jse...@redhat.com Date: 2012-08-27T06:51:20-07:00 Created test for ArtifactMatcher commit 93cf4d5cbff42cdbfc67c8b419cc988d9a3127ce Author: jsenko jse...@redhat.com Date: 2012-08-27T06:53:35-07:00 Added license comment. commit 481e9a0b744dd7e366efc75b3b7a69602c645309 Author: jsenko jse...@redhat.com Date: 2012-08-27T07:05:23-07:00 Minor whitespace and comment modifications. - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: improving incremental builds
On Mon, Aug 27, 2012 at 8:40 AM, Mark Struberg strub...@yahoo.de wrote: We might implement this by storing the list of all activated profiles + all resolved environment properties in a file in target/ somewhere. That would be part of a.) in my previous mail to Romain. How about the situation with shade? Build 1 runs shade and it replaces the primary build artifact with a shaded one. Build 2 sees that nothing has changed, but shade doesn't know, and it reshades, eating its own output with a ton of messages. Can I fix this in shade with current technologies? LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 12:24 PM Subject: Re: improving incremental builds +0 for auto detecting changed scenarios. If someone changes a profile or the whole pom, then I'd expect he invokes a clean manually. We have to document this expectation of course. What do others think of this? I'm thinking it would be awesome (but maybe difficult) if plugins could determine if its configuration has changed, and then handle this automatically. If there are to many cases where a manual clean build is required, I believe people will continue to do mvn clean install (which I see all the time with developers), which I think is what we try to get away from. /Anders LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org Cc: Sent: Monday, August 27, 2012 9:57 AM Subject: Re: improving incremental builds I already started tweaking the m-compiler-plugin and found quite scary issues. There is e.g. MCOMPILER-21 (open since 2005) which prevents incremental build and would kick in in your scenario. This is interesting. I've looked a bit at Mojo's JAXB plugin in the context of detecting stale generated files (i.e. need to re-generate) and it's similar to this. It would extremely nice if there would be a common framework for handling incremental builds. In addition to what's been mentioned in the jira (I just browsed it quickly), we have cases when includes/excludes change, the sourceDir changes, etc which should trigger cleanup and re-compilation. /Anders I talked about 2 scenarios a.) all phases = package, e.g. mvn verify, mvn install, ... Here we have an artifact on the disc and could test for the md5 of them, right? b.) all phases package. This is e.g. mvn compile or mvn test. In that case we don't produce a jar/war/ear/... artifact but only get the files on the disk linked in MavenProject#getProjectArtifacts()... file(). This is the case where the maven-compiler-plugin kicks in. I'm currently in the process of rewriting big parts of it, mainly I introduced b.1 a check for project.artifacts/project.testArtifacts .file() is a local file path .isDirectory() and has files which are newer (actually =) than the buildStarted timestamp. b.2 check whether there are any *.java (sourceincludes) files which are newer than their class files. In both cases I now force a FULL recompile of the module! Compiling only parts produced classes which are actually broken! My approach is the following: compiling a bit too much is better than missing some files which need compilation. Because the later is exactly the reason why noone uses mvn compile but always do a mvn clean compile nowadays. If mvn compile is reliable, then we will end up being much faster than an unconditional full compile on the whole project! LieGrue, strub PS: We need to move all our plugins which still use the plugin-testing-harness to the maven-invoker-plugin as the test-harness is broken with sisu. (sisu changed the 'container' field from plexus original: protected to 'private') PPS: how do we maintain the plexus-compiler-utils stuff? This contains some weirdo bugs which should get fixed... - Original Message - From: Olivier Lamy ol...@apache.org To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 9:13 AM Subject: Re: improving incremental builds Hi, Sounds good idea trying to improve that. My question: on what is based the md5 calculation ? If people don't use 'install' but only 'test' (perso I use that to avoid too much io with jar creation etc..), so in such case we cannot do md5 calculation on the produced artifact. 2012/8/26 Mark Struberg strub...@yahoo.de: Hi David! So your idea is to find the list of changed modules and then build that list with -amd? Yes, kind of. At the moment mvn -amd builds the dependents of the _current_ module. If you use my example and change BeanA.java, then run mvn -amd from the root module you will see
Shade DRP location
I've been thinking about my abortive attempt to deal with multiple builds colliding over the dependency-reduced-pom.xml. My attempt was to move the thing into 'target', but that blew up relative pathnames. Here's another plan: give the thing a uuid-based name, instead. Can anyone think of a problem with that? - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: improving incremental builds
build1 and build 2 are different modules or different mvn invocations with some time inbetween? Of course we will need to test all plugins to behave incremental like! Means the maven-shade-plugin should check itself whether it needs shading or not at all. But the worst thing happening would be that we compile too much. Thats still better than a full mvn clean install :) LieGrue, strub - Original Message - From: Benson Margulies bimargul...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 5:07 PM Subject: Re: improving incremental builds On Mon, Aug 27, 2012 at 8:40 AM, Mark Struberg strub...@yahoo.de wrote: We might implement this by storing the list of all activated profiles + all resolved environment properties in a file in target/ somewhere. That would be part of a.) in my previous mail to Romain. How about the situation with shade? Build 1 runs shade and it replaces the primary build artifact with a shaded one. Build 2 sees that nothing has changed, but shade doesn't know, and it reshades, eating its own output with a ton of messages. Can I fix this in shade with current technologies? LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 12:24 PM Subject: Re: improving incremental builds +0 for auto detecting changed scenarios. If someone changes a profile or the whole pom, then I'd expect he invokes a clean manually. We have to document this expectation of course. What do others think of this? I'm thinking it would be awesome (but maybe difficult) if plugins could determine if its configuration has changed, and then handle this automatically. If there are to many cases where a manual clean build is required, I believe people will continue to do mvn clean install (which I see all the time with developers), which I think is what we try to get away from. /Anders LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org Cc: Sent: Monday, August 27, 2012 9:57 AM Subject: Re: improving incremental builds I already started tweaking the m-compiler-plugin and found quite scary issues. There is e.g. MCOMPILER-21 (open since 2005) which prevents incremental build and would kick in in your scenario. This is interesting. I've looked a bit at Mojo's JAXB plugin in the context of detecting stale generated files (i.e. need to re-generate) and it's similar to this. It would extremely nice if there would be a common framework for handling incremental builds. In addition to what's been mentioned in the jira (I just browsed it quickly), we have cases when includes/excludes change, the sourceDir changes, etc which should trigger cleanup and re-compilation. /Anders I talked about 2 scenarios a.) all phases = package, e.g. mvn verify, mvn install, ... Here we have an artifact on the disc and could test for the md5 of them, right? b.) all phases package. This is e.g. mvn compile or mvn test. In that case we don't produce a jar/war/ear/... artifact but only get the files on the disk linked in MavenProject#getProjectArtifacts()... file(). This is the case where the maven-compiler-plugin kicks in. I'm currently in the process of rewriting big parts of it, mainly I introduced b.1 a check for project.artifacts/project.testArtifacts .file() is a local file path .isDirectory() and has files which are newer (actually =) than the buildStarted timestamp. b.2 check whether there are any *.java (sourceincludes) files which are newer than their class files. In both cases I now force a FULL recompile of the module! Compiling only parts produced classes which are actually broken! My approach is the following: compiling a bit too much is better than missing some files which need compilation. Because the later is exactly the reason why noone uses mvn compile but always do a mvn clean compile nowadays. If mvn compile is reliable, then we will end up being much faster than an unconditional full compile on the whole project! LieGrue, strub PS: We need to move all our plugins which still use the plugin-testing-harness to the maven-invoker-plugin as the test-harness is broken with sisu. (sisu changed the 'container' field from plexus original: protected to 'private') PPS: how do we maintain the plexus-compiler-utils stuff? This contains some weirdo bugs which should get fixed... - Original Message - From: Olivier Lamy ol...@apache.org To: Maven
Re: [VOTE] Release Maven Surefire version 2.12.3
+1 Hervé Le samedi 25 août 2012 00:11:55 Kristian Rosenvold a écrit : Hi, We solved 11 issues: https://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=10541version=18 704 2.12.3 should be slightly faster than previous versions, but don't expect too much. There are still lots of issues left in JIRA: http://jira.codehaus.org/secure/IssueNavigator.jspa?reset=truemode=hidejql Query=project+%3D+SUREFIRE+AND+resolution+%3D+Unresolved+ORDER+BY+updated+DE SC Staging repo: https://repository.apache.org/content/repositories/maven-011/ Staging site: (Sync pending) http://maven.apache.org/surefire/staging/ (Links to sub-sites work when transfered to production) http://maven.apache.org/plugins/maven-failsafe-plugin-2.12.3/ http://maven.apache.org/plugins/maven-surefire-plugin-2.12.3/ http://maven.apache.org/plugins/maven-surefire-reports-plugin-2.12.3/ Guide to testing staged releases: http://maven.apache.org/guides/development/guide-testing-releases.html Vote open for 72 hours. [ ] +1 [ ] +0 [ ] -1 And here's my +1 Kristian - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: improving incremental builds
On Mon, Aug 27, 2012 at 12:07 PM, Mark Struberg strub...@yahoo.de wrote: build1 and build 2 are different modules or different mvn invocations with some time inbetween? Of course we will need to test all plugins to behave incremental like! Means the maven-shade-plugin should check itself whether it needs shading or not at all. Two builds, separated in time, of exactly the same module. The problem here is that the jar plugin has already decided not to rebuild the jar by the time the shade plugin gets there, and I have no idea how to make the shade plugin play along. But the worst thing happening would be that we compile too much. Thats still better than a full mvn clean install :) LieGrue, strub - Original Message - From: Benson Margulies bimargul...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 5:07 PM Subject: Re: improving incremental builds On Mon, Aug 27, 2012 at 8:40 AM, Mark Struberg strub...@yahoo.de wrote: We might implement this by storing the list of all activated profiles + all resolved environment properties in a file in target/ somewhere. That would be part of a.) in my previous mail to Romain. How about the situation with shade? Build 1 runs shade and it replaces the primary build artifact with a shaded one. Build 2 sees that nothing has changed, but shade doesn't know, and it reshades, eating its own output with a ton of messages. Can I fix this in shade with current technologies? LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 12:24 PM Subject: Re: improving incremental builds +0 for auto detecting changed scenarios. If someone changes a profile or the whole pom, then I'd expect he invokes a clean manually. We have to document this expectation of course. What do others think of this? I'm thinking it would be awesome (but maybe difficult) if plugins could determine if its configuration has changed, and then handle this automatically. If there are to many cases where a manual clean build is required, I believe people will continue to do mvn clean install (which I see all the time with developers), which I think is what we try to get away from. /Anders LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org Cc: Sent: Monday, August 27, 2012 9:57 AM Subject: Re: improving incremental builds I already started tweaking the m-compiler-plugin and found quite scary issues. There is e.g. MCOMPILER-21 (open since 2005) which prevents incremental build and would kick in in your scenario. This is interesting. I've looked a bit at Mojo's JAXB plugin in the context of detecting stale generated files (i.e. need to re-generate) and it's similar to this. It would extremely nice if there would be a common framework for handling incremental builds. In addition to what's been mentioned in the jira (I just browsed it quickly), we have cases when includes/excludes change, the sourceDir changes, etc which should trigger cleanup and re-compilation. /Anders I talked about 2 scenarios a.) all phases = package, e.g. mvn verify, mvn install, ... Here we have an artifact on the disc and could test for the md5 of them, right? b.) all phases package. This is e.g. mvn compile or mvn test. In that case we don't produce a jar/war/ear/... artifact but only get the files on the disk linked in MavenProject#getProjectArtifacts()... file(). This is the case where the maven-compiler-plugin kicks in. I'm currently in the process of rewriting big parts of it, mainly I introduced b.1 a check for project.artifacts/project.testArtifacts .file() is a local file path .isDirectory() and has files which are newer (actually =) than the buildStarted timestamp. b.2 check whether there are any *.java (sourceincludes) files which are newer than their class files. In both cases I now force a FULL recompile of the module! Compiling only parts produced classes which are actually broken! My approach is the following: compiling a bit too much is better than missing some files which need compilation. Because the later is exactly the reason why noone uses mvn compile but always do a mvn clean compile nowadays. If mvn compile is reliable, then we will end up being much faster than an unconditional full compile on the whole project! LieGrue, strub PS: We need to move all our plugins which still use the plugin-testing-harness to the maven-invoker-plugin as the test-harness is broken with sisu. (sisu changed the 'container' field from
RE: improving incremental builds
Benson any chance of *forcing* the second iteration to re-jar with forceCreationtrue/forceCreation http://maven.apache.org/plugins/maven-jar-plugin/jar-mojo.html#forceCreation Martin __ ..place long-winded disclaimer here.. Date: Mon, 27 Aug 2012 13:09:59 -0400 Subject: Re: improving incremental builds From: bimargul...@gmail.com To: dev@maven.apache.org; strub...@yahoo.de On Mon, Aug 27, 2012 at 12:07 PM, Mark Struberg strub...@yahoo.de wrote: build1 and build 2 are different modules or different mvn invocations with some time inbetween? Of course we will need to test all plugins to behave incremental like! Means the maven-shade-plugin should check itself whether it needs shading or not at all. Two builds, separated in time, of exactly the same module. The problem here is that the jar plugin has already decided not to rebuild the jar by the time the shade plugin gets there, and I have no idea how to make the shade plugin play along. But the worst thing happening would be that we compile too much. Thats still better than a full mvn clean install :) LieGrue, strub - Original Message - From: Benson Margulies bimargul...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 5:07 PM Subject: Re: improving incremental builds On Mon, Aug 27, 2012 at 8:40 AM, Mark Struberg strub...@yahoo.de wrote: We might implement this by storing the list of all activated profiles + all resolved environment properties in a file in target/ somewhere. That would be part of a.) in my previous mail to Romain. How about the situation with shade? Build 1 runs shade and it replaces the primary build artifact with a shaded one. Build 2 sees that nothing has changed, but shade doesn't know, and it reshades, eating its own output with a ton of messages. Can I fix this in shade with current technologies? LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 12:24 PM Subject: Re: improving incremental builds +0 for auto detecting changed scenarios. If someone changes a profile or the whole pom, then I'd expect he invokes a clean manually. We have to document this expectation of course. What do others think of this? I'm thinking it would be awesome (but maybe difficult) if plugins could determine if its configuration has changed, and then handle this automatically. If there are to many cases where a manual clean build is required, I believe people will continue to do mvn clean install (which I see all the time with developers), which I think is what we try to get away from. /Anders LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org Cc: Sent: Monday, August 27, 2012 9:57 AM Subject: Re: improving incremental builds I already started tweaking the m-compiler-plugin and found quite scary issues. There is e.g. MCOMPILER-21 (open since 2005) which prevents incremental build and would kick in in your scenario. This is interesting. I've looked a bit at Mojo's JAXB plugin in the context of detecting stale generated files (i.e. need to re-generate) and it's similar to this. It would extremely nice if there would be a common framework for handling incremental builds. In addition to what's been mentioned in the jira (I just browsed it quickly), we have cases when includes/excludes change, the sourceDir changes, etc which should trigger cleanup and re-compilation. /Anders I talked about 2 scenarios a.) all phases = package, e.g. mvn verify, mvn install, ... Here we have an artifact on the disc and could test for the md5 of them, right? b.) all phases package. This is e.g. mvn compile or mvn test. In that case we don't produce a jar/war/ear/... artifact but only get the files on the disk linked in MavenProject#getProjectArtifacts()... file(). This is the case where the maven-compiler-plugin kicks in. I'm currently in the process of rewriting big parts of it, mainly I introduced b.1 a check for project.artifacts/project.testArtifacts .file() is a local file path .isDirectory() and has files which are newer (actually =) than the buildStarted timestamp. b.2 check whether there are any *.java (sourceincludes) files which are newer than their class files. In both cases I now force a FULL recompile of the module! Compiling only parts produced classes which are actually broken! My
Re: improving incremental builds
On 27 August 2012 18:09, Benson Margulies bimargul...@gmail.com wrote: On Mon, Aug 27, 2012 at 12:07 PM, Mark Struberg strub...@yahoo.de wrote: build1 and build 2 are different modules or different mvn invocations with some time inbetween? Of course we will need to test all plugins to behave incremental like! Means the maven-shade-plugin should check itself whether it needs shading or not at all. Two builds, separated in time, of exactly the same module. The problem here is that the jar plugin has already decided not to rebuild the jar by the time the shade plugin gets there, and I have no idea how to make the shade plugin play along. Any chance we can get the jar plugin to inspect some of the artifacts that it generates into the jar to see if they are consistent with what the jar plugin produces... I'm looking at you META-INF/MANIFEST.MF I.O.W. if the Created-By: Apache Maven 3.0.4 (Maven JAR Plugin) is switched to Created-By: Apache Maven 3.0.4 (Maven Shade Plugin) Then the Maven JAR plugin can infer that the timestamp check is pointless, and force regen -Stephen But the worst thing happening would be that we compile too much. Thats still better than a full mvn clean install :) LieGrue, strub - Original Message - From: Benson Margulies bimargul...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 5:07 PM Subject: Re: improving incremental builds On Mon, Aug 27, 2012 at 8:40 AM, Mark Struberg strub...@yahoo.de wrote: We might implement this by storing the list of all activated profiles + all resolved environment properties in a file in target/ somewhere. That would be part of a.) in my previous mail to Romain. How about the situation with shade? Build 1 runs shade and it replaces the primary build artifact with a shaded one. Build 2 sees that nothing has changed, but shade doesn't know, and it reshades, eating its own output with a ton of messages. Can I fix this in shade with current technologies? LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 12:24 PM Subject: Re: improving incremental builds +0 for auto detecting changed scenarios. If someone changes a profile or the whole pom, then I'd expect he invokes a clean manually. We have to document this expectation of course. What do others think of this? I'm thinking it would be awesome (but maybe difficult) if plugins could determine if its configuration has changed, and then handle this automatically. If there are to many cases where a manual clean build is required, I believe people will continue to do mvn clean install (which I see all the time with developers), which I think is what we try to get away from. /Anders LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org Cc: Sent: Monday, August 27, 2012 9:57 AM Subject: Re: improving incremental builds I already started tweaking the m-compiler-plugin and found quite scary issues. There is e.g. MCOMPILER-21 (open since 2005) which prevents incremental build and would kick in in your scenario. This is interesting. I've looked a bit at Mojo's JAXB plugin in the context of detecting stale generated files (i.e. need to re-generate) and it's similar to this. It would extremely nice if there would be a common framework for handling incremental builds. In addition to what's been mentioned in the jira (I just browsed it quickly), we have cases when includes/excludes change, the sourceDir changes, etc which should trigger cleanup and re-compilation. /Anders I talked about 2 scenarios a.) all phases = package, e.g. mvn verify, mvn install, ... Here we have an artifact on the disc and could test for the md5 of them, right? b.) all phases package. This is e.g. mvn compile or mvn test. In that case we don't produce a jar/war/ear/... artifact but only get the files on the disk linked in MavenProject#getProjectArtifacts()... file(). This is the case where the maven-compiler-plugin kicks in. I'm currently in the process of rewriting big parts of it, mainly I introduced b.1 a check for project.artifacts/project.testArtifacts .file() is a local file path .isDirectory() and has files which are newer (actually =) than the buildStarted timestamp. b.2 check whether there are any *.java (sourceincludes) files which are newer than their class files. In both cases I now force a FULL recompile of the module! Compiling
Re: improving incremental builds
On 27 August 2012 19:10, Stephen Connolly stephen.alan.conno...@gmail.comwrote: On 27 August 2012 18:09, Benson Margulies bimargul...@gmail.com wrote: On Mon, Aug 27, 2012 at 12:07 PM, Mark Struberg strub...@yahoo.de wrote: build1 and build 2 are different modules or different mvn invocations with some time inbetween? Of course we will need to test all plugins to behave incremental like! Means the maven-shade-plugin should check itself whether it needs shading or not at all. Two builds, separated in time, of exactly the same module. The problem here is that the jar plugin has already decided not to rebuild the jar by the time the shade plugin gets there, and I have no idea how to make the shade plugin play along. Any chance we can get the jar plugin to inspect some of the artifacts that it generates into the jar to see if they are consistent with what the jar plugin produces... I'm looking at you META-INF/MANIFEST.MF I.O.W. if the Created-By: Apache Maven 3.0.4 (Maven JAR Plugin) is switched to Created-By: Apache Maven 3.0.4 (Maven Shade Plugin) Then the Maven JAR plugin can infer that the timestamp check is pointless, and force regen And people customizing that header can just live with the issues that creates. -Stephen But the worst thing happening would be that we compile too much. Thats still better than a full mvn clean install :) LieGrue, strub - Original Message - From: Benson Margulies bimargul...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 5:07 PM Subject: Re: improving incremental builds On Mon, Aug 27, 2012 at 8:40 AM, Mark Struberg strub...@yahoo.de wrote: We might implement this by storing the list of all activated profiles + all resolved environment properties in a file in target/ somewhere. That would be part of a.) in my previous mail to Romain. How about the situation with shade? Build 1 runs shade and it replaces the primary build artifact with a shaded one. Build 2 sees that nothing has changed, but shade doesn't know, and it reshades, eating its own output with a ton of messages. Can I fix this in shade with current technologies? LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 12:24 PM Subject: Re: improving incremental builds +0 for auto detecting changed scenarios. If someone changes a profile or the whole pom, then I'd expect he invokes a clean manually. We have to document this expectation of course. What do others think of this? I'm thinking it would be awesome (but maybe difficult) if plugins could determine if its configuration has changed, and then handle this automatically. If there are to many cases where a manual clean build is required, I believe people will continue to do mvn clean install (which I see all the time with developers), which I think is what we try to get away from. /Anders LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org Cc: Sent: Monday, August 27, 2012 9:57 AM Subject: Re: improving incremental builds I already started tweaking the m-compiler-plugin and found quite scary issues. There is e.g. MCOMPILER-21 (open since 2005) which prevents incremental build and would kick in in your scenario. This is interesting. I've looked a bit at Mojo's JAXB plugin in the context of detecting stale generated files (i.e. need to re-generate) and it's similar to this. It would extremely nice if there would be a common framework for handling incremental builds. In addition to what's been mentioned in the jira (I just browsed it quickly), we have cases when includes/excludes change, the sourceDir changes, etc which should trigger cleanup and re-compilation. /Anders I talked about 2 scenarios a.) all phases = package, e.g. mvn verify, mvn install, ... Here we have an artifact on the disc and could test for the md5 of them, right? b.) all phases package. This is e.g. mvn compile or mvn test. In that case we don't produce a jar/war/ear/... artifact but only get the files on the disk linked in MavenProject#getProjectArtifacts()... file(). This is the case where the maven-compiler-plugin kicks in. I'm currently in the process of rewriting big parts of it, mainly I introduced b.1 a check for project.artifacts/project.testArtifacts .file() is a local file path .isDirectory() and has files which are newer (actually =) than the buildStarted timestamp. b.2 check
Re: improving incremental builds
Not sure if I get the problem. The timestamp of the input files is still than the timestamp of the jar file. Regardless if it later got modified by the shade-plugin or not. LieGrue, strub From: Stephen Connolly stephen.alan.conno...@gmail.com To: Maven Developers List dev@maven.apache.org Cc: Mark Struberg strub...@yahoo.de Sent: Monday, August 27, 2012 8:11 PM Subject: Re: improving incremental builds On 27 August 2012 19:10, Stephen Connolly stephen.alan.conno...@gmail.com wrote: On 27 August 2012 18:09, Benson Margulies bimargul...@gmail.com wrote: On Mon, Aug 27, 2012 at 12:07 PM, Mark Struberg strub...@yahoo.de wrote: build1 and build 2 are different modules or different mvn invocations with some time inbetween? Of course we will need to test all plugins to behave incremental like! Means the maven-shade-plugin should check itself whether it needs shading or not at all. Two builds, separated in time, of exactly the same module. The problem here is that the jar plugin has already decided not to rebuild the jar by the time the shade plugin gets there, and I have no idea how to make the shade plugin play along. Any chance we can get the jar plugin to inspect some of the artifacts that it generates into the jar to see if they are consistent with what the jar plugin produces... I'm looking at you META-INF/MANIFEST.MF I.O.W. if the Created-By: Apache Maven 3.0.4 (Maven JAR Plugin) is switched to Created-By: Apache Maven 3.0.4 (Maven Shade Plugin) Then the Maven JAR plugin can infer that the timestamp check is pointless, and force regen And people customizing that header can just live with the issues that creates. -Stephen But the worst thing happening would be that we compile too much. Thats still better than a full mvn clean install :) LieGrue, strub - Original Message - From: Benson Margulies bimargul...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 5:07 PM Subject: Re: improving incremental builds On Mon, Aug 27, 2012 at 8:40 AM, Mark Struberg strub...@yahoo.de wrote: We might implement this by storing the list of all activated profiles + all resolved environment properties in a file in target/ somewhere. That would be part of a.) in my previous mail to Romain. How about the situation with shade? Build 1 runs shade and it replaces the primary build artifact with a shaded one. Build 2 sees that nothing has changed, but shade doesn't know, and it reshades, eating its own output with a ton of messages. Can I fix this in shade with current technologies? LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 12:24 PM Subject: Re: improving incremental builds +0 for auto detecting changed scenarios. If someone changes a profile or the whole pom, then I'd expect he invokes a clean manually. We have to document this expectation of course. What do others think of this? I'm thinking it would be awesome (but maybe difficult) if plugins could determine if its configuration has changed, and then handle this automatically. If there are to many cases where a manual clean build is required, I believe people will continue to do mvn clean install (which I see all the time with developers), which I think is what we try to get away from. /Anders LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org Cc: Sent: Monday, August 27, 2012 9:57 AM Subject: Re: improving incremental builds I already started tweaking the m-compiler-plugin and found quite scary issues. There is e.g. MCOMPILER-21 (open since 2005) which prevents incremental build and would kick in in your scenario. This is interesting. I've looked a bit at Mojo's JAXB plugin in the context of detecting stale generated files (i.e. need to re-generate) and it's similar to this. It would extremely nice if there would be a common framework for handling incremental builds. In addition to what's been mentioned in the jira (I just browsed it quickly), we have cases when includes/excludes change, the sourceDir changes, etc which should trigger cleanup and re-compilation. /Anders I talked about 2 scenarios a.) all phases = package, e.g. mvn verify, mvn install, ... Here we have an artifact on the disc and could test for the md5 of them, right? b.) all phases package. This is e.g. mvn compile or mvn test. In that case we don't produce a jar/war/ear/... artifact but only get the files on the disk linked in MavenProject#getProjectArtifacts()... file(). This is the case where the
Re: improving incremental builds
On Mon, Aug 27, 2012 at 1:27 PM, Martin Gainty mgai...@hotmail.com wrote: Benson any chance of *forcing* the second iteration to re-jar with forceCreationtrue/forceCreation http://maven.apache.org/plugins/maven-jar-plugin/jar-mojo.html#forceCreation Yes, in my personal projects. That does not solve the real problem, in my opinion, which is that somehow the core should allow these plugins to communicate and 'just get it right'. It's complex -- changes to dependencies that require re-shading would need to trigger re-jarring. Martin __ ..place long-winded disclaimer here.. Date: Mon, 27 Aug 2012 13:09:59 -0400 Subject: Re: improving incremental builds From: bimargul...@gmail.com To: dev@maven.apache.org; strub...@yahoo.de On Mon, Aug 27, 2012 at 12:07 PM, Mark Struberg strub...@yahoo.de wrote: build1 and build 2 are different modules or different mvn invocations with some time inbetween? Of course we will need to test all plugins to behave incremental like! Means the maven-shade-plugin should check itself whether it needs shading or not at all. Two builds, separated in time, of exactly the same module. The problem here is that the jar plugin has already decided not to rebuild the jar by the time the shade plugin gets there, and I have no idea how to make the shade plugin play along. But the worst thing happening would be that we compile too much. Thats still better than a full mvn clean install :) LieGrue, strub - Original Message - From: Benson Margulies bimargul...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 5:07 PM Subject: Re: improving incremental builds On Mon, Aug 27, 2012 at 8:40 AM, Mark Struberg strub...@yahoo.de wrote: We might implement this by storing the list of all activated profiles + all resolved environment properties in a file in target/ somewhere. That would be part of a.) in my previous mail to Romain. How about the situation with shade? Build 1 runs shade and it replaces the primary build artifact with a shaded one. Build 2 sees that nothing has changed, but shade doesn't know, and it reshades, eating its own output with a ton of messages. Can I fix this in shade with current technologies? LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 12:24 PM Subject: Re: improving incremental builds +0 for auto detecting changed scenarios. If someone changes a profile or the whole pom, then I'd expect he invokes a clean manually. We have to document this expectation of course. What do others think of this? I'm thinking it would be awesome (but maybe difficult) if plugins could determine if its configuration has changed, and then handle this automatically. If there are to many cases where a manual clean build is required, I believe people will continue to do mvn clean install (which I see all the time with developers), which I think is what we try to get away from. /Anders LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org Cc: Sent: Monday, August 27, 2012 9:57 AM Subject: Re: improving incremental builds I already started tweaking the m-compiler-plugin and found quite scary issues. There is e.g. MCOMPILER-21 (open since 2005) which prevents incremental build and would kick in in your scenario. This is interesting. I've looked a bit at Mojo's JAXB plugin in the context of detecting stale generated files (i.e. need to re-generate) and it's similar to this. It would extremely nice if there would be a common framework for handling incremental builds. In addition to what's been mentioned in the jira (I just browsed it quickly), we have cases when includes/excludes change, the sourceDir changes, etc which should trigger cleanup and re-compilation. /Anders I talked about 2 scenarios a.) all phases = package, e.g. mvn verify, mvn install, ... Here we have an artifact on the disc and could test for the md5 of them, right? b.) all phases package. This is e.g. mvn compile or mvn test. In that case we don't produce a jar/war/ear/... artifact but only get the files on the disk linked in MavenProject#getProjectArtifacts()... file(). This is the case where the maven-compiler-plugin kicks in. I'm currently in the process of rewriting big parts of it, mainly I introduced b.1 a check for project.artifacts/project.testArtifacts .file() is a local file path
Re: [VOTE] Maven Skins parent 7 and Maven Fluido skin 1.3.0
+1 (non-binding). Tested this with one of my multi-module projects, looks good. Regards Mirko On Mon, Aug 27, 2012 at 3:13 PM, Olivier Lamy ol...@apache.org wrote: Hi, We solved 10 issues: http://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=11430styleName=Htmlversion=18713 Staging repo: https://repository.apache.org/content/repositories/maven-015/ Staging site: http://maven.apache.org/skins/maven-fluido-skin-1.3.0/ Guide to testing staged releases: http://maven.apache.org/guides/development/guide-testing-releases.html Vote open for 72 hours. [ ] +1 [ ] +0 [ ] -1 -- Olivier Lamy Talend: http://coders.talend.com http://twitter.com/olamy | http://linkedin.com/in/olamy - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: [VOTE] Maven Skins parent 7 and Maven Fluido skin 1.3.0
On Mon, 27 Aug 2012 15:13:21 +0200 Olivier Lamy ol...@apache.org wrote: +1 (none binding), works fine to me. well done guys, tony. Hi, We solved 10 issues: http://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=11430styleName=Htmlversion=18713 Staging repo: https://repository.apache.org/content/repositories/maven-015/ Staging site: http://maven.apache.org/skins/maven-fluido-skin-1.3.0/ Guide to testing staged releases: http://maven.apache.org/guides/development/guide-testing-releases.html Vote open for 72 hours. [ ] +1 [ ] +0 [ ] -1 -- Tony Chemit tél: +33 (0) 2 40 50 29 28 email: che...@codelutin.com http://www.codelutin.com - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: [VOTE] Maven Skins parent 7 and Maven Fluido skin 1.3.0
+1 and thanks for the great teamwork! Robert Op Mon, 27 Aug 2012 21:36:20 +0200 schreef Tony Chemit che...@codelutin.com: On Mon, 27 Aug 2012 15:13:21 +0200 Olivier Lamy ol...@apache.org wrote: +1 (none binding), works fine to me. well done guys, tony. Hi, We solved 10 issues: http://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=11430styleName=Htmlversion=18713 Staging repo: https://repository.apache.org/content/repositories/maven-015/ Staging site: http://maven.apache.org/skins/maven-fluido-skin-1.3.0/ Guide to testing staged releases: http://maven.apache.org/guides/development/guide-testing-releases.html Vote open for 72 hours. [ ] +1 [ ] +0 [ ] -1 - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: [VOTE] Release Maven Surefire version 2.12.3
+1 (non-binding) Regards Mirko On Mon, Aug 27, 2012 at 6:57 PM, Hervé BOUTEMY herve.bout...@free.fr wrote: +1 Hervé Le samedi 25 août 2012 00:11:55 Kristian Rosenvold a écrit : Hi, We solved 11 issues: https://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=10541version=18 704 2.12.3 should be slightly faster than previous versions, but don't expect too much. There are still lots of issues left in JIRA: http://jira.codehaus.org/secure/IssueNavigator.jspa?reset=truemode=hidejql Query=project+%3D+SUREFIRE+AND+resolution+%3D+Unresolved+ORDER+BY+updated+DE SC Staging repo: https://repository.apache.org/content/repositories/maven-011/ Staging site: (Sync pending) http://maven.apache.org/surefire/staging/ (Links to sub-sites work when transfered to production) http://maven.apache.org/plugins/maven-failsafe-plugin-2.12.3/ http://maven.apache.org/plugins/maven-surefire-plugin-2.12.3/ http://maven.apache.org/plugins/maven-surefire-reports-plugin-2.12.3/ Guide to testing staged releases: http://maven.apache.org/guides/development/guide-testing-releases.html Vote open for 72 hours. [ ] +1 [ ] +0 [ ] -1 And here's my +1 Kristian - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: improving incremental builds
The issue (for benson) is when the shade plugin modifies after the class files have changed... the jar plugin will currently skip recreating the jar, and he has no way of knowing if the jar file has been shaded or not the solution is to have the manifest.mf include an entry that indicates created by shade, so that shade can safely skip On 27 August 2012 19:39, Mark Struberg strub...@yahoo.de wrote: Not sure if I get the problem. The timestamp of the input files is still than the timestamp of the jar file. Regardless if it later got modified by the shade-plugin or not. LieGrue, strub From: Stephen Connolly stephen.alan.conno...@gmail.com To: Maven Developers List dev@maven.apache.org Cc: Mark Struberg strub...@yahoo.de Sent: Monday, August 27, 2012 8:11 PM Subject: Re: improving incremental builds On 27 August 2012 19:10, Stephen Connolly stephen.alan.conno...@gmail.com wrote: On 27 August 2012 18:09, Benson Margulies bimargul...@gmail.com wrote: On Mon, Aug 27, 2012 at 12:07 PM, Mark Struberg strub...@yahoo.de wrote: build1 and build 2 are different modules or different mvn invocations with some time inbetween? Of course we will need to test all plugins to behave incremental like! Means the maven-shade-plugin should check itself whether it needs shading or not at all. Two builds, separated in time, of exactly the same module. The problem here is that the jar plugin has already decided not to rebuild the jar by the time the shade plugin gets there, and I have no idea how to make the shade plugin play along. Any chance we can get the jar plugin to inspect some of the artifacts that it generates into the jar to see if they are consistent with what the jar plugin produces... I'm looking at you META-INF/MANIFEST.MF I.O.W. if the Created-By: Apache Maven 3.0.4 (Maven JAR Plugin) is switched to Created-By: Apache Maven 3.0.4 (Maven Shade Plugin) Then the Maven JAR plugin can infer that the timestamp check is pointless, and force regen And people customizing that header can just live with the issues that creates. -Stephen But the worst thing happening would be that we compile too much. Thats still better than a full mvn clean install :) LieGrue, strub - Original Message - From: Benson Margulies bimargul...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 5:07 PM Subject: Re: improving incremental builds On Mon, Aug 27, 2012 at 8:40 AM, Mark Struberg strub...@yahoo.de wrote: We might implement this by storing the list of all activated profiles + all resolved environment properties in a file in target/ somewhere. That would be part of a.) in my previous mail to Romain. How about the situation with shade? Build 1 runs shade and it replaces the primary build artifact with a shaded one. Build 2 sees that nothing has changed, but shade doesn't know, and it reshades, eating its own output with a ton of messages. Can I fix this in shade with current technologies? LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 12:24 PM Subject: Re: improving incremental builds +0 for auto detecting changed scenarios. If someone changes a profile or the whole pom, then I'd expect he invokes a clean manually. We have to document this expectation of course. What do others think of this? I'm thinking it would be awesome (but maybe difficult) if plugins could determine if its configuration has changed, and then handle this automatically. If there are to many cases where a manual clean build is required, I believe people will continue to do mvn clean install (which I see all the time with developers), which I think is what we try to get away from. /Anders LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org Cc: Sent: Monday, August 27, 2012 9:57 AM Subject: Re: improving incremental builds I already started tweaking the m-compiler-plugin and found quite scary issues. There is e.g. MCOMPILER-21 (open since 2005) which prevents incremental build and would kick in in your scenario. This is interesting. I've looked a bit at Mojo's JAXB plugin in the context of detecting stale generated files (i.e. need to re-generate) and it's similar to this. It would extremely nice if there would be a common framework for handling incremental builds. In addition to what's been mentioned in the jira (I just browsed it quickly), we have cases when includes/excludes
Re: improving incremental builds
the shade plugin would just need to store a local status file with the md5 of the created result. Or add the info to the manifest as you proposed. The main point is that this should be up to the maven-shade-plugin, as it is the only one who knows when to repeat this step. LieGrue, strub From: Stephen Connolly stephen.alan.conno...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Sent: Monday, August 27, 2012 9:53 PM Subject: Re: improving incremental builds The issue (for benson) is when the shade plugin modifies after the class files have changed... the jar plugin will currently skip recreating the jar, and he has no way of knowing if the jar file has been shaded or not the solution is to have the manifest.mf include an entry that indicates created by shade, so that shade can safely skip On 27 August 2012 19:39, Mark Struberg strub...@yahoo.de wrote: Not sure if I get the problem. The timestamp of the input files is still than the timestamp of the jar file. Regardless if it later got modified by the shade-plugin or not. LieGrue, strub From: Stephen Connolly stephen.alan.conno...@gmail.com To: Maven Developers List dev@maven.apache.org Cc: Mark Struberg strub...@yahoo.de Sent: Monday, August 27, 2012 8:11 PM Subject: Re: improving incremental builds On 27 August 2012 19:10, Stephen Connolly stephen.alan.conno...@gmail.com wrote: On 27 August 2012 18:09, Benson Margulies bimargul...@gmail.com wrote: On Mon, Aug 27, 2012 at 12:07 PM, Mark Struberg strub...@yahoo.de wrote: build1 and build 2 are different modules or different mvn invocations with some time inbetween? Of course we will need to test all plugins to behave incremental like! Means the maven-shade-plugin should check itself whether it needs shading or not at all. Two builds, separated in time, of exactly the same module. The problem here is that the jar plugin has already decided not to rebuild the jar by the time the shade plugin gets there, and I have no idea how to make the shade plugin play along. Any chance we can get the jar plugin to inspect some of the artifacts that it generates into the jar to see if they are consistent with what the jar plugin produces... I'm looking at you META-INF/MANIFEST.MF I.O.W. if the Created-By: Apache Maven 3.0.4 (Maven JAR Plugin) is switched to Created-By: Apache Maven 3.0.4 (Maven Shade Plugin) Then the Maven JAR plugin can infer that the timestamp check is pointless, and force regen And people customizing that header can just live with the issues that creates. -Stephen But the worst thing happening would be that we compile too much. Thats still better than a full mvn clean install :) LieGrue, strub - Original Message - From: Benson Margulies bimargul...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 5:07 PM Subject: Re: improving incremental builds On Mon, Aug 27, 2012 at 8:40 AM, Mark Struberg strub...@yahoo.de wrote: We might implement this by storing the list of all activated profiles + all resolved environment properties in a file in target/ somewhere. That would be part of a.) in my previous mail to Romain. How about the situation with shade? Build 1 runs shade and it replaces the primary build artifact with a shaded one. Build 2 sees that nothing has changed, but shade doesn't know, and it reshades, eating its own output with a ton of messages. Can I fix this in shade with current technologies? LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 12:24 PM Subject: Re: improving incremental builds +0 for auto detecting changed scenarios. If someone changes a profile or the whole pom, then I'd expect he invokes a clean manually. We have to document this expectation of course. What do others think of this? I'm thinking it would be awesome (but maybe difficult) if plugins could determine if its configuration has changed, and then handle this automatically. If there are to many cases where a manual clean build is required, I believe people will continue to do mvn clean install (which I see all the time with developers), which I think is what we try to get away from. /Anders LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org Cc: Sent: Monday, August 27, 2012 9:57 AM Subject: Re: improving incremental builds I already started tweaking the m-compiler-plugin and found quite scary issues. There is e.g. MCOMPILER-21 (open since 2005) which prevents incremental build and would kick in in
Re: [VOTE] Release Maven Project Info Reports Plugin version 2.5.1
+1 (non-binding) tested on one multi-module project. Regards Mirko On Sun, Aug 26, 2012 at 10:11 PM, Hervé BOUTEMY herve.bout...@free.fr wrote: Hi, We solved 5 issues: http://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=11142version=18715styleName=Html There are still a couple of issues left in JIRA: http://jira.codehaus.org/secure/IssueNavigator.jspa?reset=truepid=11142status=1 Staging repo: https://repository.apache.org/content/repositories/maven-013/ https://repository.apache.org/content/repositories/maven-013/org/apache/maven/plugins/maven- project-info-reports-plugin/2.5.1/maven-project-info-reports-plugin-2.5.1- source-release.zip Staging site: (sync pending) http://maven.apache.org/plugins/maven-project-info-reports-plugin-2.5.1/ Guide to testing staged releases: http://maven.apache.org/guides/development/guide-testing-releases.html Vote open for 72 hours. [ ] +1 [ ] +0 [ ] -1 - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: improving incremental builds
I've wrote up some stuff in our Wiki: https://cwiki.apache.org/confluence/display/MAVEN/Incremental+Builds Feel free to add more info/ideas. LieGrue, strub PS: a small test app can be found on https://github.com/struberg/maventest Try this with the latest compiler plugin 2.6-SNAPSHOT from trunk - Original Message - From: Mark Struberg strub...@yahoo.de To: Maven Developers List dev@maven.apache.org Cc: Sent: Monday, August 27, 2012 10:04 PM Subject: Re: improving incremental builds t he shade plugin would just need to store a local status file with the md5 of the created result. Or add the info to the manifest as you proposed. The main point is that this should be up to the maven-shade-plugin, as it is the only one who knows when to repeat this step. LieGrue, strub From: Stephen Connolly stephen.alan.conno...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Sent: Monday, August 27, 2012 9:53 PM Subject: Re: improving incremental builds The issue (for benson) is when the shade plugin modifies after the class files have changed... the jar plugin will currently skip recreating the jar, and he has no way of knowing if the jar file has been shaded or not the solution is to have the manifest.mf include an entry that indicates created by shade, so that shade can safely skip On 27 August 2012 19:39, Mark Struberg strub...@yahoo.de wrote: Not sure if I get the problem. The timestamp of the input files is still than the timestamp of the jar file. Regardless if it later got modified by the shade-plugin or not. LieGrue, strub From: Stephen Connolly stephen.alan.conno...@gmail.com To: Maven Developers List dev@maven.apache.org Cc: Mark Struberg strub...@yahoo.de Sent: Monday, August 27, 2012 8:11 PM Subject: Re: improving incremental builds On 27 August 2012 19:10, Stephen Connolly stephen.alan.conno...@gmail.com wrote: On 27 August 2012 18:09, Benson Margulies bimargul...@gmail.com wrote: On Mon, Aug 27, 2012 at 12:07 PM, Mark Struberg strub...@yahoo.de wrote: build1 and build 2 are different modules or different mvn invocations with some time inbetween? Of course we will need to test all plugins to behave incremental like! Means the maven-shade-plugin should check itself whether it needs shading or not at all. Two builds, separated in time, of exactly the same module. The problem here is that the jar plugin has already decided not to rebuild the jar by the time the shade plugin gets there, and I have no idea how to make the shade plugin play along. Any chance we can get the jar plugin to inspect some of the artifacts that it generates into the jar to see if they are consistent with what the jar plugin produces... I'm looking at you META-INF/MANIFEST.MF I.O.W. if the Created-By: Apache Maven 3.0.4 (Maven JAR Plugin) is switched to Created-By: Apache Maven 3.0.4 (Maven Shade Plugin) Then the Maven JAR plugin can infer that the timestamp check is pointless, and force regen And people customizing that header can just live with the issues that creates. -Stephen But the worst thing happening would be that we compile too much. Thats still better than a full mvn clean install :) LieGrue, strub - Original Message - From: Benson Margulies bimargul...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 5:07 PM Subject: Re: improving incremental builds On Mon, Aug 27, 2012 at 8:40 AM, Mark Struberg strub...@yahoo.de wrote: We might implement this by storing the list of all activated profiles + all resolved environment properties in a file in target/ somewhere. That would be part of a.) in my previous mail to Romain. How about the situation with shade? Build 1 runs shade and it replaces the primary build artifact with a shaded one. Build 2 sees that nothing has changed, but shade doesn't know, and it reshades, eating its own output with a ton of messages. Can I fix this in shade with current technologies? LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 12:24 PM Subject: Re: improving incremental builds +0 for auto detecting changed scenarios. If someone changes a profile or the whole pom, then I'd expect he invokes a clean manually. We have to document this expectation of course. What do others think of this? I'm thinking it would be awesome (but maybe difficult) if plugins could determine if its configuration has changed, and
Re: improving incremental builds
There are some plugins which should not act incremental by default. Mainly the test stuff. People would not like it if they invoke $ mvn test and their test would not run because they already did run some time before and no code/dependency changed since. Thus I propose to introduce 2 new MavenCLI parameters: $ mvn -i test -i for 'build Incrementally' resp xor $ mvn -ni test -ni for 'build Non-Incrementally' Not sure what the default should be though... From the backward compat point probably -ni LieGrue, strub - Original Message - From: Mark Struberg strub...@yahoo.de To: Maven Developers List dev@maven.apache.org Cc: Sent: Monday, August 27, 2012 10:33 PM Subject: Re: improving incremental builds I've wrote up some stuff in our Wiki: https://cwiki.apache.org/confluence/display/MAVEN/Incremental+Builds Feel free to add more info/ideas. LieGrue, strub PS: a small test app can be found on https://github.com/struberg/maventest Try this with the latest compiler plugin 2.6-SNAPSHOT from trunk - Original Message - From: Mark Struberg strub...@yahoo.de To: Maven Developers List dev@maven.apache.org Cc: Sent: Monday, August 27, 2012 10:04 PM Subject: Re: improving incremental builds t he shade plugin would just need to store a local status file with the md5 of the created result. Or add the info to the manifest as you proposed. The main point is that this should be up to the maven-shade-plugin, as it is the only one who knows when to repeat this step. LieGrue, strub From: Stephen Connolly stephen.alan.conno...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Sent: Monday, August 27, 2012 9:53 PM Subject: Re: improving incremental builds The issue (for benson) is when the shade plugin modifies after the class files have changed... the jar plugin will currently skip recreating the jar, and he has no way of knowing if the jar file has been shaded or not the solution is to have the manifest.mf include an entry that indicates created by shade, so that shade can safely skip On 27 August 2012 19:39, Mark Struberg strub...@yahoo.de wrote: Not sure if I get the problem. The timestamp of the input files is still than the timestamp of the jar file. Regardless if it later got modified by the shade-plugin or not. LieGrue, strub From: Stephen Connolly stephen.alan.conno...@gmail.com To: Maven Developers List dev@maven.apache.org Cc: Mark Struberg strub...@yahoo.de Sent: Monday, August 27, 2012 8:11 PM Subject: Re: improving incremental builds On 27 August 2012 19:10, Stephen Connolly stephen.alan.conno...@gmail.com wrote: On 27 August 2012 18:09, Benson Margulies bimargul...@gmail.com wrote: On Mon, Aug 27, 2012 at 12:07 PM, Mark Struberg strub...@yahoo.de wrote: build1 and build 2 are different modules or different mvn invocations with some time inbetween? Of course we will need to test all plugins to behave incremental like! Means the maven-shade-plugin should check itself whether it needs shading or not at all. Two builds, separated in time, of exactly the same module. The problem here is that the jar plugin has already decided not to rebuild the jar by the time the shade plugin gets there, and I have no idea how to make the shade plugin play along. Any chance we can get the jar plugin to inspect some of the artifacts that it generates into the jar to see if they are consistent with what the jar plugin produces... I'm looking at you META-INF/MANIFEST.MF I.O.W. if the Created-By: Apache Maven 3.0.4 (Maven JAR Plugin) is switched to Created-By: Apache Maven 3.0.4 (Maven Shade Plugin) Then the Maven JAR plugin can infer that the timestamp check is pointless, and force regen And people customizing that header can just live with the issues that creates. -Stephen But the worst thing happening would be that we compile too much. Thats still better than a full mvn clean install :) LieGrue, strub - Original Message - From: Benson Margulies bimargul...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 5:07 PM Subject: Re: improving incremental builds On Mon, Aug 27, 2012 at 8:40 AM, Mark Struberg strub...@yahoo.de wrote: We might implement this by storing the list of all activated profiles + all resolved environment properties in a file in target/ somewhere. That would be part of a.) in my previous mail to Romain. How about the situation with shade? Build 1 runs shade and it replaces the primary build
Re: improving incremental builds
Mark, I don't see how this helps. If the jar plugin hasn't rebuilt the jar, the shade plugin can't rebuild the jar, even if, for other reasons, it needs to. It has knowledge that the jar plugin lacks. On Mon, Aug 27, 2012 at 4:04 PM, Mark Struberg strub...@yahoo.de wrote: the shade plugin would just need to store a local status file with the md5 of the created result. Or add the info to the manifest as you proposed. The main point is that this should be up to the maven-shade-plugin, as it is the only one who knows when to repeat this step. LieGrue, strub From: Stephen Connolly stephen.alan.conno...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Sent: Monday, August 27, 2012 9:53 PM Subject: Re: improving incremental builds The issue (for benson) is when the shade plugin modifies after the class files have changed... the jar plugin will currently skip recreating the jar, and he has no way of knowing if the jar file has been shaded or not the solution is to have the manifest.mf include an entry that indicates created by shade, so that shade can safely skip On 27 August 2012 19:39, Mark Struberg strub...@yahoo.de wrote: Not sure if I get the problem. The timestamp of the input files is still than the timestamp of the jar file. Regardless if it later got modified by the shade-plugin or not. LieGrue, strub From: Stephen Connolly stephen.alan.conno...@gmail.com To: Maven Developers List dev@maven.apache.org Cc: Mark Struberg strub...@yahoo.de Sent: Monday, August 27, 2012 8:11 PM Subject: Re: improving incremental builds On 27 August 2012 19:10, Stephen Connolly stephen.alan.conno...@gmail.com wrote: On 27 August 2012 18:09, Benson Margulies bimargul...@gmail.com wrote: On Mon, Aug 27, 2012 at 12:07 PM, Mark Struberg strub...@yahoo.de wrote: build1 and build 2 are different modules or different mvn invocations with some time inbetween? Of course we will need to test all plugins to behave incremental like! Means the maven-shade-plugin should check itself whether it needs shading or not at all. Two builds, separated in time, of exactly the same module. The problem here is that the jar plugin has already decided not to rebuild the jar by the time the shade plugin gets there, and I have no idea how to make the shade plugin play along. Any chance we can get the jar plugin to inspect some of the artifacts that it generates into the jar to see if they are consistent with what the jar plugin produces... I'm looking at you META-INF/MANIFEST.MF I.O.W. if the Created-By: Apache Maven 3.0.4 (Maven JAR Plugin) is switched to Created-By: Apache Maven 3.0.4 (Maven Shade Plugin) Then the Maven JAR plugin can infer that the timestamp check is pointless, and force regen And people customizing that header can just live with the issues that creates. -Stephen But the worst thing happening would be that we compile too much. Thats still better than a full mvn clean install :) LieGrue, strub - Original Message - From: Benson Margulies bimargul...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 5:07 PM Subject: Re: improving incremental builds On Mon, Aug 27, 2012 at 8:40 AM, Mark Struberg strub...@yahoo.de wrote: We might implement this by storing the list of all activated profiles + all resolved environment properties in a file in target/ somewhere. That would be part of a.) in my previous mail to Romain. How about the situation with shade? Build 1 runs shade and it replaces the primary build artifact with a shaded one. Build 2 sees that nothing has changed, but shade doesn't know, and it reshades, eating its own output with a ton of messages. Can I fix this in shade with current technologies? LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 12:24 PM Subject: Re: improving incremental builds +0 for auto detecting changed scenarios. If someone changes a profile or the whole pom, then I'd expect he invokes a clean manually. We have to document this expectation of course. What do others think of this? I'm thinking it would be awesome (but maybe difficult) if plugins could determine if its configuration has changed, and then handle this automatically. If there are to many cases where a manual clean build is required, I believe people will continue to do mvn clean install (which I see all the time with developers), which I think is what we try to get away from. /Anders LieGrue, strub - Original Message - From: Anders Hammar and...@hammar.net To: Maven Developers List dev@maven.apache.org Cc:
Re: improving incremental builds
It's a chain. The maven-compiler-plugin sees that there are stale sources or changed dependencies and compiles the sources. The jar plugin sees that there are changed classes which have a newer timestamp than the jar file, so it creates the jar again. The maven-shade-plugin will see the md5 of the jar changed (or detect otherwise that this is not a shaded jar) and trigger the shading. Of course, most of those plugins do not yet support this. But there is nothing stopping us from hacking that ;) LieGrue, strub - Original Message - From: Benson Margulies bimargul...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 11:13 PM Subject: Re: improving incremental builds Mark, I don't see how this helps. If the jar plugin hasn't rebuilt the jar, the shade plugin can't rebuild the jar, even if, for other reasons, it needs to. It has knowledge that the jar plugin lacks. On Mon, Aug 27, 2012 at 4:04 PM, Mark Struberg strub...@yahoo.de wrote: the shade plugin would just need to store a local status file with the md5 of the created result. Or add the info to the manifest as you proposed. The main point is that this should be up to the maven-shade-plugin, as it is the only one who knows when to repeat this step. LieGrue, strub From: Stephen Connolly stephen.alan.conno...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Sent: Monday, August 27, 2012 9:53 PM Subject: Re: improving incremental builds The issue (for benson) is when the shade plugin modifies after the class files have changed... the jar plugin will currently skip recreating the jar, and he has no way of knowing if the jar file has been shaded or not the solution is to have the manifest.mf include an entry that indicates created by shade, so that shade can safely skip On 27 August 2012 19:39, Mark Struberg strub...@yahoo.de wrote: Not sure if I get the problem. The timestamp of the input files is still than the timestamp of the jar file. Regardless if it later got modified by the shade-plugin or not. LieGrue, strub From: Stephen Connolly stephen.alan.conno...@gmail.com To: Maven Developers List dev@maven.apache.org Cc: Mark Struberg strub...@yahoo.de Sent: Monday, August 27, 2012 8:11 PM Subject: Re: improving incremental builds On 27 August 2012 19:10, Stephen Connolly stephen.alan.conno...@gmail.com wrote: On 27 August 2012 18:09, Benson Margulies bimargul...@gmail.com wrote: On Mon, Aug 27, 2012 at 12:07 PM, Mark Struberg strub...@yahoo.de wrote: build1 and build 2 are different modules or different mvn invocations with some time inbetween? Of course we will need to test all plugins to behave incremental like! Means the maven-shade-plugin should check itself whether it needs shading or not at all. Two builds, separated in time, of exactly the same module. The problem here is that the jar plugin has already decided not to rebuild the jar by the time the shade plugin gets there, and I have no idea how to make the shade plugin play along. Any chance we can get the jar plugin to inspect some of the artifacts that it generates into the jar to see if they are consistent with what the jar plugin produces... I'm looking at you META-INF/MANIFEST.MF I.O.W. if the Created-By: Apache Maven 3.0.4 (Maven JAR Plugin) is switched to Created-By: Apache Maven 3.0.4 (Maven Shade Plugin) Then the Maven JAR plugin can infer that the timestamp check is pointless, and force regen And people customizing that header can just live with the issues that creates. -Stephen But the worst thing happening would be that we compile too much. Thats still better than a full mvn clean install :) LieGrue, strub - Original Message - From: Benson Margulies bimargul...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 5:07 PM Subject: Re: improving incremental builds On Mon, Aug 27, 2012 at 8:40 AM, Mark Struberg strub...@yahoo.de wrote: We might implement this by storing the list of all activated profiles + all resolved environment properties in a file in target/ somewhere. That would be part of a.) in my previous mail to Romain. How about the situation with shade? Build 1 runs shade and it replaces the primary build artifact with a shaded one. Build 2 sees that nothing has changed, but shade doesn't know, and it reshades, eating its own output with a ton of messages. Can I fix this in shade with current technologies? LieGrue, strub - Original Message - From: Anders Hammar
Re: improving incremental builds
On Mon, Aug 27, 2012 at 5:56 PM, Mark Struberg strub...@yahoo.de wrote: It's a chain. The maven-compiler-plugin sees that there are stale sources or changed dependencies and compiles the sources. The jar plugin sees that there are changed classes which have a newer timestamp than the jar file, so it creates the jar again. The maven-shade-plugin will see the md5 of the jar changed (or detect otherwise that this is not a shaded jar) and trigger the shading. Not good enough. Let me describe the failure mode. No sources have changed. So the compiler plugin doesn't recompile. So, no .class files have changed, so the jar plugin doesn't jar. However, a dependency *has* changed, so the shade plugin needs a clean input. But it doesn't have a clean input, since the only thing around is the jar it shaded last time around. The only fix I can see is if the shade plugin keeps around a copy of the original unshaded jar when it shades, and uses that to redo the shade when all of this happens. Of course, most of those plugins do not yet support this. But there is nothing stopping us from hacking that ;) LieGrue, strub - Original Message - From: Benson Margulies bimargul...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 11:13 PM Subject: Re: improving incremental builds Mark, I don't see how this helps. If the jar plugin hasn't rebuilt the jar, the shade plugin can't rebuild the jar, even if, for other reasons, it needs to. It has knowledge that the jar plugin lacks. On Mon, Aug 27, 2012 at 4:04 PM, Mark Struberg strub...@yahoo.de wrote: the shade plugin would just need to store a local status file with the md5 of the created result. Or add the info to the manifest as you proposed. The main point is that this should be up to the maven-shade-plugin, as it is the only one who knows when to repeat this step. LieGrue, strub From: Stephen Connolly stephen.alan.conno...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Sent: Monday, August 27, 2012 9:53 PM Subject: Re: improving incremental builds The issue (for benson) is when the shade plugin modifies after the class files have changed... the jar plugin will currently skip recreating the jar, and he has no way of knowing if the jar file has been shaded or not the solution is to have the manifest.mf include an entry that indicates created by shade, so that shade can safely skip On 27 August 2012 19:39, Mark Struberg strub...@yahoo.de wrote: Not sure if I get the problem. The timestamp of the input files is still than the timestamp of the jar file. Regardless if it later got modified by the shade-plugin or not. LieGrue, strub From: Stephen Connolly stephen.alan.conno...@gmail.com To: Maven Developers List dev@maven.apache.org Cc: Mark Struberg strub...@yahoo.de Sent: Monday, August 27, 2012 8:11 PM Subject: Re: improving incremental builds On 27 August 2012 19:10, Stephen Connolly stephen.alan.conno...@gmail.com wrote: On 27 August 2012 18:09, Benson Margulies bimargul...@gmail.com wrote: On Mon, Aug 27, 2012 at 12:07 PM, Mark Struberg strub...@yahoo.de wrote: build1 and build 2 are different modules or different mvn invocations with some time inbetween? Of course we will need to test all plugins to behave incremental like! Means the maven-shade-plugin should check itself whether it needs shading or not at all. Two builds, separated in time, of exactly the same module. The problem here is that the jar plugin has already decided not to rebuild the jar by the time the shade plugin gets there, and I have no idea how to make the shade plugin play along. Any chance we can get the jar plugin to inspect some of the artifacts that it generates into the jar to see if they are consistent with what the jar plugin produces... I'm looking at you META-INF/MANIFEST.MF I.O.W. if the Created-By: Apache Maven 3.0.4 (Maven JAR Plugin) is switched to Created-By: Apache Maven 3.0.4 (Maven Shade Plugin) Then the Maven JAR plugin can infer that the timestamp check is pointless, and force regen And people customizing that header can just live with the issues that creates. -Stephen But the worst thing happening would be that we compile too much. Thats still better than a full mvn clean install :) LieGrue, strub - Original Message - From: Benson Margulies bimargul...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 5:07 PM Subject: Re: improving incremental builds On Mon, Aug 27, 2012 at 8:40 AM, Mark Struberg strub...@yahoo.de wrote: We might implement this by storing the list of all activated profiles + all resolved
Re: improving incremental builds
Hi Benson! However, a dependency *has* changed, Yea, thanks for the example! I think I kind of know what you mean now (I hope). That is why I introduced point A.) in the WiKi description [1]. If any of those tests indicate a change then we force a 'clean' on the module and on all depending downstream modules. This will kind of make an intelligent mvn clean install, where the 'clean' lifecycle only gets invoked if an external change got detected. And only on the modules which are affected, others will remain untouched. I'm even trying to spare running the tests for those unaffected modules. Just need to figure a way to force running the tests manually. Thats why I thought about -ni and -i ... What makes it a bit more complicated is that there is a slightly different behaviour for all phases package and = package in maven. But by combining A.) and B.) we should get this working as well. LieGrue, strub [1] https://cwiki.apache.org/confluence/display/MAVEN/Incremental+Builds - Original Message - From: Benson Margulies bimargul...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 11:58 PM Subject: Re: improving incremental builds On Mon, Aug 27, 2012 at 5:56 PM, Mark Struberg strub...@yahoo.de wrote: It's a chain. The maven-compiler-plugin sees that there are stale sources or changed dependencies and compiles the sources. The jar plugin sees that there are changed classes which have a newer timestamp than the jar file, so it creates the jar again. The maven-shade-plugin will see the md5 of the jar changed (or detect otherwise that this is not a shaded jar) and trigger the shading. Not good enough. Let me describe the failure mode. No sources have changed. So the compiler plugin doesn't recompile. So, no .class files have changed, so the jar plugin doesn't jar. However, a dependency *has* changed, so the shade plugin needs a clean input. But it doesn't have a clean input, since the only thing around is the jar it shaded last time around. The only fix I can see is if the shade plugin keeps around a copy of the original unshaded jar when it shades, and uses that to redo the shade when all of this happens. Of course, most of those plugins do not yet support this. But there is nothing stopping us from hacking that ;) LieGrue, strub - Original Message - From: Benson Margulies bimargul...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 11:13 PM Subject: Re: improving incremental builds Mark, I don't see how this helps. If the jar plugin hasn't rebuilt the jar, the shade plugin can't rebuild the jar, even if, for other reasons, it needs to. It has knowledge that the jar plugin lacks. On Mon, Aug 27, 2012 at 4:04 PM, Mark Struberg strub...@yahoo.de wrote: the shade plugin would just need to store a local status file with the md5 of the created result. Or add the info to the manifest as you proposed. The main point is that this should be up to the maven-shade-plugin, as it is the only one who knows when to repeat this step. LieGrue, strub From: Stephen Connolly stephen.alan.conno...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Sent: Monday, August 27, 2012 9:53 PM Subject: Re: improving incremental builds The issue (for benson) is when the shade plugin modifies after the class files have changed... the jar plugin will currently skip recreating the jar, and he has no way of knowing if the jar file has been shaded or not the solution is to have the manifest.mf include an entry that indicates created by shade, so that shade can safely skip On 27 August 2012 19:39, Mark Struberg strub...@yahoo.de wrote: Not sure if I get the problem. The timestamp of the input files is still than the timestamp of the jar file. Regardless if it later got modified by the shade-plugin or not. LieGrue, strub From: Stephen Connolly stephen.alan.conno...@gmail.com To: Maven Developers List dev@maven.apache.org Cc: Mark Struberg strub...@yahoo.de Sent: Monday, August 27, 2012 8:11 PM Subject: Re: improving incremental builds On 27 August 2012 19:10, Stephen Connolly stephen.alan.conno...@gmail.com wrote: On 27 August 2012 18:09, Benson Margulies bimargul...@gmail.com wrote: On Mon, Aug 27, 2012 at 12:07 PM, Mark Struberg strub...@yahoo.de wrote: build1 and build 2 are different modules or different mvn invocations with some time inbetween? Of course we will need to test all plugins to behave incremental like! Means the maven-shade-plugin should check itself whether it needs shading or
Re: improving incremental builds
Oh I think I'm slowly catching what you meant. You were not talking about dependency but the configurationartifactSet inside the maven-shade-plugin declaration, right? In that case A.) doesn't kick in, true. Guess the easiest way would be to keep the original jar somewhere as you already proposed. LieGrue, strub - Original Message - From: Mark Struberg strub...@yahoo.de To: Maven Developers List dev@maven.apache.org Cc: Sent: Tuesday, August 28, 2012 12:10 AM Subject: Re: improving incremental builds Hi Benson! However, a dependency *has* changed, Yea, thanks for the example! I think I kind of know what you mean now (I hope). That is why I introduced point A.) in the WiKi description [1]. If any of those tests indicate a change then we force a 'clean' on the module and on all depending downstream modules. This will kind of make an intelligent mvn clean install, where the 'clean' lifecycle only gets invoked if an external change got detected. And only on the modules which are affected, others will remain untouched. I'm even trying to spare running the tests for those unaffected modules. Just need to figure a way to force running the tests manually. Thats why I thought about -ni and -i ... What makes it a bit more complicated is that there is a slightly different behaviour for all phases package and = package in maven. But by combining A.) and B.) we should get this working as well. LieGrue, strub [1] https://cwiki.apache.org/confluence/display/MAVEN/Incremental+Builds - Original Message - From: Benson Margulies bimargul...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 11:58 PM Subject: Re: improving incremental builds On Mon, Aug 27, 2012 at 5:56 PM, Mark Struberg strub...@yahoo.de wrote: It's a chain. The maven-compiler-plugin sees that there are stale sources or changed dependencies and compiles the sources. The jar plugin sees that there are changed classes which have a newer timestamp than the jar file, so it creates the jar again. The maven-shade-plugin will see the md5 of the jar changed (or detect otherwise that this is not a shaded jar) and trigger the shading. Not good enough. Let me describe the failure mode. No sources have changed. So the compiler plugin doesn't recompile. So, no .class files have changed, so the jar plugin doesn't jar. However, a dependency *has* changed, so the shade plugin needs a clean input. But it doesn't have a clean input, since the only thing around is the jar it shaded last time around. The only fix I can see is if the shade plugin keeps around a copy of the original unshaded jar when it shades, and uses that to redo the shade when all of this happens. Of course, most of those plugins do not yet support this. But there is nothing stopping us from hacking that ;) LieGrue, strub - Original Message - From: Benson Margulies bimargul...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Cc: Sent: Monday, August 27, 2012 11:13 PM Subject: Re: improving incremental builds Mark, I don't see how this helps. If the jar plugin hasn't rebuilt the jar, the shade plugin can't rebuild the jar, even if, for other reasons, it needs to. It has knowledge that the jar plugin lacks. On Mon, Aug 27, 2012 at 4:04 PM, Mark Struberg strub...@yahoo.de wrote: the shade plugin would just need to store a local status file with the md5 of the created result. Or add the info to the manifest as you proposed. The main point is that this should be up to the maven-shade-plugin, as it is the only one who knows when to repeat this step. LieGrue, strub From: Stephen Connolly stephen.alan.conno...@gmail.com To: Maven Developers List dev@maven.apache.org; Mark Struberg strub...@yahoo.de Sent: Monday, August 27, 2012 9:53 PM Subject: Re: improving incremental builds The issue (for benson) is when the shade plugin modifies after the class files have changed... the jar plugin will currently skip recreating the jar, and he has no way of knowing if the jar file has been shaded or not the solution is to have the manifest.mf include an entry that indicates created by shade, so that shade can safely skip On 27 August 2012 19:39, Mark Struberg strub...@yahoo.de wrote: Not sure if I get the problem. The timestamp of the input files is still than the timestamp of the jar file. Regardless if it later got modified by the shade-plugin or not. LieGrue, strub From: Stephen Connolly stephen.alan.conno...@gmail.com To: Maven
Re: Why do the core its fail massively when using mirrorOf in settings.xml ?
There are lots of tests that are trying to use file based repositories for certain conditions. This is why in 2.0.9 I had added the external:* : external:* matches all repositories except those using localhost or file based repositories. This is used in conjunction with a repository manager when you want to exclude redirecting repositories that are defined for Integration Testing. Maybe we should update the default setting.xml in the book, but it's pretty rare that people are using file repos for their ITs (read: pretty much only maven ;-) ) On Thu, Aug 23, 2012 at 2:44 PM, Kristian Rosenvold kristian.rosenv...@gmail.com wrote: When I try to run the maven core IT's with a mirrorOf setting in my settings.xml (copy-paste from http://www.sonatype.com/books/nexus-book/reference/maven-sect-single-group.html) I get all sorts of weird wonderful artifact resolution failures for snapshots inside the forked test processes; this happens regardless of which maven 3.x version I try to test. Removing the the setup from settings.xml clearly fixes the problem. Can someone explain what is going on, is this a jira issue or WTF ? Kristian - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Why do the core its fail massively when using mirrorOf in settings.xml ?
I definitely use external:* I didn't even notice the book doesn't. On Aug 27, 2012, at 9:11 PM, Brian Fox wrote: There are lots of tests that are trying to use file based repositories for certain conditions. This is why in 2.0.9 I had added the external:* : external:* matches all repositories except those using localhost or file based repositories. This is used in conjunction with a repository manager when you want to exclude redirecting repositories that are defined for Integration Testing. Maybe we should update the default setting.xml in the book, but it's pretty rare that people are using file repos for their ITs (read: pretty much only maven ;-) ) On Thu, Aug 23, 2012 at 2:44 PM, Kristian Rosenvold kristian.rosenv...@gmail.com wrote: When I try to run the maven core IT's with a mirrorOf setting in my settings.xml (copy-paste from http://www.sonatype.com/books/nexus-book/reference/maven-sect-single-group.html) I get all sorts of weird wonderful artifact resolution failures for snapshots inside the forked test processes; this happens regardless of which maven 3.x version I try to test. Removing the the setup from settings.xml clearly fixes the problem. Can someone explain what is going on, is this a jira issue or WTF ? Kristian - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org Thanks, Jason -- Jason van Zyl Founder CTO, Sonatype Founder, Apache Maven http://twitter.com/jvanzyl - First, the taking in of scattered particulars under one Idea, so that everyone understands what is being talked about ... Second, the separation of the Idea into parts, by dividing it at the joints, as nature directs, not breaking any limb in half as a bad carver might. -- Plato, Phaedrus (Notes on the Synthesis of Form by C. Alexander)
[RESULT][VOTE] Release Maven Surefire version 2.12.3
Hi, The vote has passed with the following result : +1 (binding): Olivier Lamy, Hervé Boutemy, Kristian Rosenvold +1 (non binding): Tony Chemit, Mirko Friedenhagen I will promote the artifacts to the central repo. Kristian - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
