Not sure how I've missed this post. Have you tried this build with the master build of Maven? MNG-6566 [1] should prevent any unnecessary double executions, thus optimizing the buildplan. I'm interested if this will solve the problem at hand.
Martin [1] https://issues.apache.org/jira/browse/MNG-6566 Op di 18 mei 2021 om 14:57 schreef Guillaume Nodet <gno...@apache.org>: > I'm looking a bit at aggregator goals. > At first glance, it seems most of the problem comes from the fact that the > build ordering is done mostly per-project. This causes obvious problems > for aggregators when a mojo needs a given phase to be executed for all > children before it can run (for example the compilation phase before > creating the javadoc). Another problem comes from the clean task which has > obvious side effects. Calling clean with an aggregator goal involved in > the build seems like a recipe for problems... > > Would you see bad side effects to split the build in smaller chunks > (basically down to a mojo execution) so that ordering can be changed in a > more meaningful way ? For example when running an aggregate javadoc, the > top level project would be built first until before the javadoc aggregate > goal. The build would then go to all other projects (like the forked > lifecycle) until all required goals have been run, then resume from the > javadoc goal. > Note that with the current behavior, when you run "clean verify", the > execution is the following > > [INFO] Scanning for projects... > [INFO] > ------------------------------------------------------------------------ > [INFO] Reactor Build Order: > [INFO] > [INFO] forked > [pom] > [INFO] forked-mod1 > [jar] > [INFO] forked-mod2 > [jar] > [INFO] > [INFO] ---------------< org.mvndaemon.mvnd.test.forked:forked > >---------------- > [INFO] Building forked 0.0.1-SNAPSHOT > [1/3] > [INFO] --------------------------------[ pom > ]--------------------------------- > [INFO] > [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ forked --- > [INFO] >>> maven-javadoc-plugin:3.2.0:aggregate-jar (aggregate-jar) > > compile @ forked >>> > [INFO] > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > [INFO] Forking forked-mod1 0.0.1-SNAPSHOT > [INFO] > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ > forked-mod1 --- > [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ > forked-mod1 --- > [INFO] > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > [INFO] Forking forked-mod2 0.0.1-SNAPSHOT > [INFO] > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ > forked-mod2 --- > [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ > forked-mod2 --- > [INFO] <<< maven-javadoc-plugin:3.2.0:aggregate-jar (aggregate-jar) < > compile @ forked <<< > [INFO] --- maven-javadoc-plugin:3.2.0:aggregate-jar (aggregate-jar) @ > forked --- > [INFO] > [INFO] -------------< org.mvndaemon.mvnd.test.forked:forked-mod1 > >------------- > [INFO] Building forked-mod1 0.0.1-SNAPSHOT > [2/3] > [INFO] --------------------------------[ jar > ]--------------------------------- > [INFO] > [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ forked-mod1 --- > [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ > forked-mod1 --- > [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ > forked-mod1 --- > [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) > @ forked-mod1 --- > [INFO] --- maven-compiler-plugin:3.8.0:testCompile (default-testCompile) @ > forked-mod1 --- > [INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ forked-mod1 > --- > [INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ forked-mod1 --- > [INFO] > [INFO] -------------< org.mvndaemon.mvnd.test.forked:forked-mod2 > >------------- > [INFO] Building forked-mod2 0.0.1-SNAPSHOT > [3/3] > [INFO] --------------------------------[ jar > ]--------------------------------- > [INFO] > [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ forked-mod2 --- > [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ > forked-mod2 --- > [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ > forked-mod2 --- > [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) > @ forked-mod2 --- > [INFO] --- maven-compiler-plugin:3.8.0:testCompile (default-testCompile) @ > forked-mod2 --- > [INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ forked-mod2 > --- > [INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ forked-mod2 --- > [INFO] > ------------------------------------------------------------------------ > [INFO] Reactor Summary for forked 0.0.1-SNAPSHOT: > [INFO] > [INFO] forked ............................................. SUCCESS [ > 1.921 s] > [INFO] forked-mod1 ........................................ SUCCESS [ > 0.498 s] > [INFO] forked-mod2 ........................................ SUCCESS [ > 0.046 s] > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 2.535 s > [INFO] Finished at: 2021-05-18T14:32:09+02:00 > [INFO] > ------------------------------------------------------------------------ > > As you can see, the resources and compile goals are run twice for each > project. Furthermore, given the clean goal is in the loop, the compiler > actually compiles things twice. > > The build plan could be rewritten as > > [INFO] Scanning for projects... > [INFO] > ------------------------------------------------------------------------ > [INFO] Reactor Build Order: > [INFO] > [INFO] forked > [pom] > [INFO] forked-mod1 > [jar] > [INFO] forked-mod2 > [jar] > [INFO] > [INFO] ---------------< org.mvndaemon.mvnd.test.forked:forked > >---------------- > [INFO] Building forked 0.0.1-SNAPSHOT > [1/3] > [INFO] --------------------------------[ pom > ]--------------------------------- > [INFO] > [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ forked --- > ... normal lifecycle interrupted because the next goal is the aggregator > goal ... > [INFO] > [INFO] -------------< org.mvndaemon.mvnd.test.forked:forked-mod1 > >------------- > [INFO] Building forked-mod1 0.0.1-SNAPSHOT > [2/3] > [INFO] --------------------------------[ jar > ]--------------------------------- > [INFO] > [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ forked-mod1 --- > [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ > forked-mod1 --- > [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ > forked-mod1 --- > ... interrupting forked-mod1 build as it reached the required point > execution ... > [INFO] > [INFO] -------------< org.mvndaemon.mvnd.test.forked:forked-mod2 > >------------- > [INFO] Building forked-mod2 0.0.1-SNAPSHOT > [3/3] > [INFO] --------------------------------[ jar > ]--------------------------------- > [INFO] > [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ forked-mod2 --- > [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ > forked-mod2 --- > [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ > forked-mod2 --- > ... interrupting forked-mod2 build as it reached the required point > execution ... > ... resuming the top level project... > [INFO] > [INFO] ---------------< org.mvndaemon.mvnd.test.forked:forked > >---------------- > [INFO] Building forked 0.0.1-SNAPSHOT > [1/3] > [INFO] --------------------------------[ pom > ]--------------------------------- > [INFO] > [INFO] --- maven-javadoc-plugin:3.2.0:aggregate-jar (aggregate-jar) @ > forked --- > ... then resuming the forked-mod1 project... > [INFO] > [INFO] -------------< org.mvndaemon.mvnd.test.forked:forked-mod1 > >------------- > [INFO] Building forked-mod1 0.0.1-SNAPSHOT > [2/3] > [INFO] --------------------------------[ jar > ]--------------------------------- > [INFO] > [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) > @ forked-mod1 --- > [INFO] --- maven-compiler-plugin:3.8.0:testCompile (default-testCompile) @ > forked-mod1 --- > [INFO] --- maven-javadoc-plugin:3.2.0:aggregate-jar (aggregate-jar) @ > forked --- > [INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ forked-mod1 > --- > [INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ forked-mod1 --- > ... and resuming the forked-mod2 project... > [INFO] > [INFO] -------------< org.mvndaemon.mvnd.test.forked:forked-mod2 > >------------- > [INFO] Building forked-mod2 0.0.1-SNAPSHOT > [3/3] > [INFO] --------------------------------[ jar > ]--------------------------------- > [INFO] > [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) > @ forked-mod2 --- > [INFO] --- maven-compiler-plugin:3.8.0:testCompile (default-testCompile) @ > forked-mod2 --- > [INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ forked-mod2 > --- > [INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ forked-mod2 --- > [INFO] > ------------------------------------------------------------------------ > [INFO] Reactor Summary for forked 0.0.1-SNAPSHOT: > [INFO] > [INFO] forked ............................................. SUCCESS [ > 1.921 s] > [INFO] forked-mod1 ........................................ SUCCESS [ > 0.498 s] > [INFO] forked-mod2 ........................................ SUCCESS [ > 0.046 s] > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 2.535 s > [INFO] Finished at: 2021-05-18T14:32:09+02:00 > [INFO] > ------------------------------------------------------------------------ > > This would also solve the parallel build problem because each project would > not be able to be both "cleaned" and "compiled" concurrently, which is > currently possible in the current state. > > > Le lun. 17 mai 2021 à 14:06, Guillaume Nodet <gno...@apache.org> a écrit : > > > I've raised https://issues.apache.org/jira/browse/MNG-7156 > > > > Le mer. 12 mai 2021 à 17:57, Falko Modler <f.mod...@gmx.net> a écrit : > > > >> Hi Guillaume, > >> > >> aggregation goals and parallel builds in combination are a bit of a > >> mess, e.g.: > >> > >> - https://issues.apache.org/jira/browse/MNG-6843 > >> - https://github.com/apache/maven/pull/413 > >> - https://www.mail-archive.com/dev@maven.apache.org/msg123439.html > >> > >> Cheers, > >> > >> Falko > >> > >> Am 12.05.2021 um 17:25 schrieb Guillaume Nodet: > >> > Hi > >> > > >> > I've analyzed a bug reported on mvnd this afternoon ( > >> > https://github.com/mvndaemon/mvnd/issues/408). It appears that the > >> parent > >> > pom executes the javadoc aggregate goal, which forks the lifecycle of > >> the > >> > children modules in order to compile the sources. In a traditional > >> build, > >> > this does not cause any real problem, but in a parallel build, a > clean > >> > verify can definitely cause issues if the forked lifecycle and the > >> normal > >> > project build (and especially the clean) are run concurrently. > >> > This definitely looks like an issue to me. Any idea where I should > >> look at > >> > how to solve the problem ? I wonder if the MojoExecutor should > somehow > >> > delegate to the Builder which is responsible for synchronizing the > >> > executions in the case of a multithreaded build... > >> > Thoughts ? > >> > > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > >> For additional commands, e-mail: dev-h...@maven.apache.org > >> > >> > > > > -- > > ------------------------ > > Guillaume Nodet > > > > > > -- > ------------------------ > Guillaume Nodet >
