Hi Jospeh, I didn’t necessarily expect that enabling the extension would solve/avoid the issue you described, but I mentioned it, because it would allow you to not have to specify an argument like '-pl TestSupportModule1,TestSupportModule2,ModuleB,App’ in the first place, because the Build Cache Extension should automatically determine which modules need to be built and for which ones previously cached artifacts can be used.
Nils. > Op 6 feb 2024, om 15:11 heeft Joseph Leonard <joseph.leon...@alfasystems.com> > het volgende geschreven: > > Thanks Nils, > maven-build-cache-extension looks very interesting generally – I will have a > play with it. > With regards to this issue, the maven-build-cache-extension overview > references “Subtree support for multimodule projects builds part of the > codebase in isolation” which sounded similar to the solution I am proposing > for this issue. Unfortunately, after enabling the maven-build-cache-extension > I still hit the same issue. > Joe > > On 2024/02/06 12:54:59 Nils Breunese wrote: >> I can’t comment on your question directly, but I just wanted to say that >> your use case sounds like it could benefit from the Maven Build Cache >> Extension (https://maven.apache.org/extensions/maven-build-cache-extension/). > >> >> Just my 2 cents. >> >> Nils. >> >>> Op 6 feb 2024 om 11:40 heeft Joseph Leonard <jo...@alfasystems.com> het >>> volgende geschreven: >>> >>> Hi all, >>> >>> It would be great to get any thoughts on whether the following is a defect: >>> >>> >>> Issue details: >>> tl;dr >>> >>> Maven can resolve dependencies either from: >>> >>> * an external repo >>> * a class directory of a module being built within the reactor >>> * a packaged jar of a module being built within the reactor >>> >>> If you run a concurrent multi-module build it is possible to get a race >>> condition whereby the build of module Foo may resolve module Bar from >>> either of the three resolution channels. This inconsistency can result in >>> the Maven war plugin sometimes failing to build a functional war file. I >>> would expect a consistent resolution would always take place. > >>> >>> Full details >>> Scenario >>> >>> Consider you have a repo with the following structure: >>> >>> App >>> >>> / \ >>> >>> / \ >>> >>> (compile scope) (test scope) >>> >>> / \ >>> >>> \/_ _\/ >>> >>> ModuleA TestSupportModule1 >>> >>> / >>> >>> / >>> >>> (compile scope) >>> >>> / >>> >>> \/_ >>> >>> ModuleB >>> >>> / >>> >>> / >>> >>> (test scope) >>> >>> / >>> >>> \/_ >>> >>> TestSupportModule2 >>> >>> If you were to make a src code change to the following test support modules: >>> >>> * TestSupportModule1 >>> * TestSupportModule2 >>> >>> Then the minimum number of modules we need to build to verify the change >>> set is OK is: >>> >>> * TestSupportModule1 >>> * TestSupportModule2 >>> * ModuleB >>> * App >>> >>> i.e. there is no requirement to build ModuleA because we know that none of >>> the src code changes could impact the classpaths used in its maven build. > >>> >>> We know that despite 'App' depending (transitively) on ModuleB there is no >>> need for the 'App' build to wait for ModuleB to complete its build because >>> the src code change to TestSupportModule2 will not impact any of the >>> classpaths used in the App maven build. Therefore to get the most efficient >>> build possible we ideally would invoke Maven to run with 2 threads and with >>> instruction to build two distinct 'dependency graphs': > >>> >>> * TestSupportModule1 followed by ModuleB >>> * TestSupportModule1 followed by App >>> >>> The following Maven command achieves exactly what we want because the >>> reactor build order is based only on the direct (i.e. non-transitive) >>> dependencies of the modules provided to the reactor in the build command. >>> Therefore the absence of ModuleA results in two distinct 'dependency >>> graphs': > >>> >>> mvn clean verify -pl TestSupportModule1,TestSupportModule2,ModuleB,App -T 2 >>> >>> Note: In reality the code base I maintain has a very large monobuild with >>> 100s of modules and this type of build optimisation makes a significant >>> difference to the speed of our monobuild (we use >>> https://github.com/gitflow-incremental-builder/gitflow-incremental-builder >>> to automate the logic of determining which modules to include in the >>> reactor based on our change set). > >>> >>> Issue >>> >>> We have encountered an issue in the above scenario because the 'App' build >>> has a race condition with the ModuleB build which will result in one of the >>> following three outcomes: > >>> >>> * If the 'App' build starts before the ModuleB build has compiled its src >>> classes then the 'App' build will resolve ModuleB from the external repo >>> (i.e. equivalent to ModuleB not being in the reactor at all) > >>> * If the 'App' build starts after ModuleB has compiled its src classes >>> but before it has packaged these classes into a jar then the 'App' build >>> will resolve ModuleB's target/classes directory > >>> * If the 'App' build starts after ModuleB has packaged its jar file then >>> the 'App' build will resolve ModuleB's target/ModuleB.jar file. > >>> >>> In many scenarios this dependency resolution inconsistency doesn't >>> represent a challenge. However, it does cause an issue in our case because >>> the 'App' POM has its Maven packaging stanza configured to war and in the >>> scenario where ModuleB's target/classes directory is resolved by the 'App' >>> then this results in the resultant 'App' war file being packaged with a >>> completely empty ModuleB.jar file. > >>> >>> Proposed solution >>> >>> Ideally we would like the Maven reactor to retain isolation between the two >>> distinct 'dependency graphs' it constructs at instantiation throughout the >>> entire Maven build. This would mean, in the simple example above, that the >>> 'App' would always resolves ModuleB from the external repo (regardless of >>> whether the reactor has built ModuleB or not in a separate 'dependency >>> graph' in the reactor). > >>> >>> >>> >>> Joseph Leonard >>> Manager >>> >>> Alfa >>> ________________________________ >>> e: joseph.leon...@alfasystems.com | w: >>> alfasystems.com<https://www.alfasystems.com> >>> t: +44 (0)20 7588 1800 | Moor Place, 1 Fore Street Avenue, London, EC2Y >>> 9DT, GB >>> ________________________________ >>> >>> The contents of this communication are not intended to be binding or >>> constitute any form of offer or acceptance or give rise to any legal >>> obligations on behalf of the sender or Alfa. The views or opinions >>> expressed represent those of the author and not necessarily those of Alfa. >>> This email and any attachments are strictly confidential and are intended >>> solely for use by the individual or entity to whom it is addressed. If you >>> are not the addressee (or responsible for delivery of the message to the >>> addressee) you may not copy, forward, disclose or use any part of the >>> message or its attachments. At present the integrity of email across the >>> internet cannot be guaranteed and messages sent via this medium are >>> potentially at risk. All liability is excluded to the extent permitted by >>> law for any claims arising as a result of the use of this medium to >>> transmit information by or to Alfa or its affiliates. > >>> >>> Alfa Financial Software Ltd >>> Reg. in England No: 0248 2325 >> --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
