RE: Re: Re: Re: Re: Inconsistent dependency resolution behaviour for concurrent multi-module build can cause failures
Hi Tamás, Is there an Maven issue out there to track this issue (and your fix for it)? I haven’t been able to get access to the Maven Jira project so have not been able to raise the issue myself. Thanks, Joe On 2024/02/08 10:09:31 Tamás Cservenák wrote: > Seems we are on track with this. To prove my last-night theory I created a > "hack" (is really just that) and guess what? > It makes reproducer behave "as expected": > https://github.com/apache/maven/pull/1406 > > T > > On Wed, Feb 7, 2024 at 10:05 PM Tamás Cservenák wrote: > > > Howdy, > > > > Thank you very much, the reproducer works. Did not dig thru it fully, but > > here are some related issues: > > > > https://issues.apache.org/jira/browse/MNG-8028 (funny thing, I created > > this few weeks ago) > > https://issues.apache.org/jira/browse/MNG-6300 > > > > Will report back tomorrow (EU TZ) > > T > > > > > > On Wed, Feb 7, 2024 at 7:48 PM Joseph Leonard < > > joseph.leon...@alfasystems.com> wrote: > > > >> Hi Tamás, > >> I have created a simple example here: > >> https://github.com/josple/mvn-multibuild-issue > >> Hopefully the README is clear enough – let me know if I can clarify > >> anything. > >> Thanks, > >> Joe > >> > >> On 2024/02/07 17:33:08 Tamás Cservenák wrote: > >> > Howdy, > >> > > >> > In that case, there is something fishy with the project, my blind guess > >> > would be some "hidden" inter-module dependency maybe? > >> > > >> > Can you provide access to source, or, if not feasible, could you provide > >> > some reproducer and publish it on Github/Gitlab/whatever (maybe even > >> just > >> > send it privately as ML strips off attachments and images) for us to see > >> > this in action? > >> > > >> > Thanks > >> > T > >> > > >> > On Wed, Feb 7, 2024 at 6:29 PM Joseph Leonard < > >> > joseph.leon...@alfasystems.com> wrote: > >> > > >> > > Hi Tamás, > >> > > We have previously played around a bit with mvnd but not takari > >> directly – > >> > > I will have a play with it. With regards to this issue, using the > >> takari > >> > > smart builder unfortunately doesn’t resolve the issue. > >> > > Joe > >> > > > >> > > On 2024/02/07 11:41:22 Tamás Cservenák wrote: > >> > > > Can you please try smart builder instead? > >> > > > https://github.com/takari/takari-smart-builder > >> > > > > >> > > > (note: smart builder is used by mvnd as well) > >> > > > > >> > > > The difference between the two can be seen here: > >> > > > http://takari.io/book/30-team-maven.html#takari-smart-builder > >> > > > > >> > > > On Wed, Feb 7, 2024 at 11:50 AM Joseph Leonard < > >> > > > joseph.leon...@alfasystems.com> wrote: > >> > > > > >> > > > > Hi Tamás, > >> > > > > Yeah, this was unexpected to me initially as well. From what I > >> can tell > >> > > > > the Maven reactor only considers direct dependencies (i.e. not > >> > > transitive > >> > > > > dependencies) between the modules in the reactor when working out > >> the > >> > > build > >> > > > > graph. For example if you have a simple linear dependency chain > >> of: > >> > > > > One --> Two --> Three --> Four --> Five > >> > > > > Then invoking “mvn clean verify -pl One,Two,Four,Five -T 2 will > >> result > >> > > in > >> > > > > two ‘graphs’ being built in parallel ([One,Two] and [Four,Five]). > >> I > >> > > assume > >> > > > > this is as designed because it actually offers quite powerful > >> > > functionality > >> > > > > to improve the parallelism in your build. An example of where > >> this is > >> > > legit > >> > > > > is when: > >> > > > > > >> > > > > * “Four” has a test scope dependency on “Five” > >> > > > > * “One” has a test scoped dependency on “Two” > >> > > > > If you made a src code change to “Five” and “Two” then it would be > >> > > safe to > >> > > > > build [One,Two] and [Four,Five] in parallel because you know the > >> > > changes > >> > > > > within these graphs cannot impact each other. > >> > > > > Joe > >> > > > > > >> > > > > On 2024/02/06 21:37:42 Tamás Cservenák wrote: > >> > > > > > Howdy, > >> > > > > > > >> > > > > > To me this looks like Maven is not aware that the App depends on > >> > > > > ModuleB... > >> > > > > > Are they "plain dependency" linked? Or what kind of dependency > >> we > >> > > talk > >> > > > > > about here? > >> > > > > > In short: why would App start while ModuleB (upstream dep) is > >> not > >> > > done? > >> > > > > > Something is fishy here. > >> > > > > > > >> > > > > > T > >> > > > > > > >> > > > > > > >> > > > > > On Tue, Feb 6, 2024 at 11:40 AM Joseph Leonard < > >> > > > > > joseph.leon...@alfasystems.com> wrote: > >> > > > > > > >> > > > > > > Hi all, > >> > > > > > > > >> > > > > > > It would be great to get any thoughts on whether the > >> following is a > >> > > > > defect: > >> > > > > > > > >> > > > > > > > >> > > > > > > Issue details: > >> > > > > > > tl;dr > >> > > > > > > > >> > > > > > > Maven can resolve dependencies either from: > >> > > > > > > > >> > > > > > > * an external repo > >> > > > > > > * a class
RE: Re: Re: Re: Re: Re: Inconsistent dependency resolution behaviour for concurrent multi-module build can cause failures
Hey, Yeah, exactly: ideally we want module-b to be resolved in different ways within the same reactor. I noticed that this kind of already happens (albeit in a simpler way): In my reproducer testing if you play with the test timeouts to dictate the module build order then with the same modules in the reactor you can get different resolution behaviour: * If "app" builds before "module-b" has started compiling then "app" will resolve "module-b" from the local repo * If "app" builds after "module-b" has compiled but before "module-b" has packaged then "app" will resolve the "module-b" class directory (this is the out-of-the-box scenario the reproducer produces) * If "app" builds after "module-b" has packaged then "app" will resolve the "module-b" jar from the "module-b" target directory So within this logic to perform different types of resolves, it would be neat if the reactor could perform an initial check first: * if the dependency being resolved is within the same 'sub-build graph' of the module requesting the resolve then look within the reactor (current behaviour) * otherwise, only resolve from the local repository I don't know whether the code architecture means this is quite easy to do or very difficult. But to be fair, I think this extra parallelism opportunity is a bit of niche edge case and it probably isn't worth it if this is very complex to achieve (or considered a smell generally by the community!) On 2024/02/08 Tamás Cservenák wrote > Howdy, > > Well, sadly, Maven currently has no means to make one module "this and that" > at the same time. > It is either in the reactor or is out of it. > To me, it sounds like you want one reactor, but where module-b is "out" and > 'in", at the same time (within same session) > > > T > > On Thu, Feb 8, 2024 at 1:27 PM Joseph Leonard < > joseph.leon...@alfasystems.com> wrote: > > > Hey, > > We don't need the module-b jar built in the reactor to be included in > > the war if the changes made in module-b won't affect the war. Instead, > > in this scenario, we're fine with using a jar resolved from the local > > repository. > > The requisites for this scenario are: > > > > * Any change made to module-b is only in its test tree > > * The only changes made to the direct and transitive dependencies of > > module-b are to its test scoped dependencies So basically module-b is > > only in the reactor for verification purposes rather than ‘src > > packaging’ purposes. > > Thanks, > > Joe > > > > On 2024/02/08 12:12:03 Tamás Cservenák wrote: > > > Hej, > > > > > > yes, basically the "hack" is well described by you: > > > - it retains "full' reactor project > > > - introduces "needs build" on project > > > - when it comes to building, it skips the project if flag set > > > > > > This results that module-b -> app link is not lost. > > > And as for your parallelization request: the truth is that there IS > > > and edge between app and module-b (via module-a), so despite > > > module-a is "done", its transitive module-b is not done, hence you > > > cannot build these two in parallel, as you do have an edge between app > > > and module-b. > > > > > > Or to invert the question: what module-b JAR you want to have > > > included in app WAR, when building? The one built, or the one from > > > the local repository? If the latter, why are you building it in the first > > > place? > > > > > > On Thu, Feb 8, 2024 at 12:06 PM Joseph Leonard < > > > joseph.leon...@alfasystems.com> wrote: > > > > > > > Hi Tamás, > > > > Thank you for hacking around this so rapidly. I am not familiar > > > > with > > the > > > > maven code base, so let me know if I have misinterpreted the > > > > change: Is your hack to: > > > > > > > > 1. Retain the full multi-module build dependency graph such > > > > that the build graph always remains consistent regardless of whether a > > > > “-pl” > > arg has > > > > been provided > > > > 2. Introduce a skip build option to satisfy the functionality > > > > of the “-pl” arg: The reactor will report the module has been > > > > built but the > > build > > > > is basically a no-op > > > > If so, this does resolve the issue but at the cost of losing the > > > > opportunity to gain extra parallelism in the build. > > > > What I was wondering may be possible (without any idea of how > > > > feasible > > it > > > > would be to implement) would be the opportunity to recognise in > > > > the multi-build distinct build graphs and then keep isolation in > > > > the > > reactor > > > > between these graphs. To elaborate, in the reproducer scenario > > > > there > > would > > > > be two distinct graphs: > > > > > > > > * "testsupport-module-1" followed by "app" > > > > * "testsupport-module-2" followed by "module-b" > > > > Ideally these two graphs build in parallel to each other and even > > > > if ultimately there is a dependency between them they NEVER > > > > resolve that dependency from within reactor
Re: Re: Re: Re: Re: Re: Inconsistent dependency resolution behaviour for concurrent multi-module build can cause failures
Howdy, Well, sadly, Maven currently has no means to make one module "this and that" at the same time. It is either in the reactor or is out of it. To me, it sounds like you want one reactor, but where module-b is "out" and 'in", at the same time (within same session) T On Thu, Feb 8, 2024 at 1:27 PM Joseph Leonard < joseph.leon...@alfasystems.com> wrote: > Hey, > We don't need the module-b jar built in the reactor to be included in the > war if the changes made in module-b won't affect the war. Instead, in this > scenario, we're fine with using a jar resolved from the local repository. > The requisites for this scenario are: > > * Any change made to module-b is only in its test tree > * The only changes made to the direct and transitive dependencies of > module-b are to its test scoped dependencies > So basically module-b is only in the reactor for verification purposes > rather than ‘src packaging’ purposes. > Thanks, > Joe > > On 2024/02/08 12:12:03 Tamás Cservenák wrote: > > Hej, > > > > yes, basically the "hack" is well described by you: > > - it retains "full' reactor project > > - introduces "needs build" on project > > - when it comes to building, it skips the project if flag set > > > > This results that module-b -> app link is not lost. > > And as for your parallelization request: the truth is that there IS and > > edge between app and module-b (via module-a), so despite module-a is > > "done", its transitive module-b is not done, hence you cannot build these > > two in parallel, as you do have an edge between app and module-b. > > > > Or to invert the question: what module-b JAR you want to have included in > > app WAR, when building? The one built, or the one from the local > > repository? If the latter, why are you building it in the first place? > > > > On Thu, Feb 8, 2024 at 12:06 PM Joseph Leonard < > > joseph.leon...@alfasystems.com> wrote: > > > > > Hi Tamás, > > > Thank you for hacking around this so rapidly. I am not familiar with > the > > > maven code base, so let me know if I have misinterpreted the change: Is > > > your hack to: > > > > > > 1. Retain the full multi-module build dependency graph such that the > > > build graph always remains consistent regardless of whether a “-pl” > arg has > > > been provided > > > 2. Introduce a skip build option to satisfy the functionality of the > > > “-pl” arg: The reactor will report the module has been built but the > build > > > is basically a no-op > > > If so, this does resolve the issue but at the cost of losing the > > > opportunity to gain extra parallelism in the build. > > > What I was wondering may be possible (without any idea of how feasible > it > > > would be to implement) would be the opportunity to recognise in the > > > multi-build distinct build graphs and then keep isolation in the > reactor > > > between these graphs. To elaborate, in the reproducer scenario there > would > > > be two distinct graphs: > > > > > > * "testsupport-module-1" followed by "app" > > > * "testsupport-module-2" followed by "module-b" > > > Ideally these two graphs build in parallel to each other and even if > > > ultimately there is a dependency between them they NEVER resolve that > > > dependency from within reactor they instead always resolve from the > > > external Maven cache. The key statement here is that if a user > presents a > > > list of modules to be built then they really need to be sure that the > > > DIRECT dependencies between these modules truly represent the build > > > graph(s) they want. For tools like gitflow-incremental-builder which > are > > > designed to carefully work out change sets and what needs to build this > > > offers a really powerful opportunity to optimise builds. I appreciate > this > > > functionality could also be seen as a gotcha (although tbf this already > > > exists). > > > Thanks, > > > Joe > > > > > > > > > On 2024/02/08 10:09:31 Tamás Cservenák wrote: > > > > Seems we are on track with this. To prove my last-night theory I > created > > > a > > > > "hack" (is really just that) and guess what? > > > > It makes reproducer behave "as expected": > > > > https://github.com/apache/maven/pull/1406 > > > > > > > > T > > > > > > > > On Wed, Feb 7, 2024 at 10:05 PM Tamás Cservenák > > > > wrote: > > > > > > > > > Howdy, > > > > > > > > > > Thank you very much, the reproducer works. Did not dig thru it > fully, > > > but > > > > > here are some related issues: > > > > > > > > > > https://issues.apache.org/jira/browse/MNG-8028 (funny thing, I > created > > > > > this few weeks ago) > > > > > https://issues.apache.org/jira/browse/MNG-6300 > > > > > > > > > > Will report back tomorrow (EU TZ) > > > > > T > > > > > > > > > > > > > > > On Wed, Feb 7, 2024 at 7:48 PM Joseph Leonard < > > > > > joseph.leon...@alfasystems.com> wrote: > > > > > > > > > >> Hi Tamás, > > > > >> I have created a simple example here: > > > > >> https://github.com/josple/mvn-multibuild-issue > > > > >> Hopefully the
RE: Re: Re: Re: Re: Re: Inconsistent dependency resolution behaviour for concurrent multi-module build can cause failures
Hey, We don't need the module-b jar built in the reactor to be included in the war if the changes made in module-b won't affect the war. Instead, in this scenario, we're fine with using a jar resolved from the local repository. The requisites for this scenario are: * Any change made to module-b is only in its test tree * The only changes made to the direct and transitive dependencies of module-b are to its test scoped dependencies So basically module-b is only in the reactor for verification purposes rather than ‘src packaging’ purposes. Thanks, Joe On 2024/02/08 12:12:03 Tamás Cservenák wrote: > Hej, > > yes, basically the "hack" is well described by you: > - it retains "full' reactor project > - introduces "needs build" on project > - when it comes to building, it skips the project if flag set > > This results that module-b -> app link is not lost. > And as for your parallelization request: the truth is that there IS and > edge between app and module-b (via module-a), so despite module-a is > "done", its transitive module-b is not done, hence you cannot build these > two in parallel, as you do have an edge between app and module-b. > > Or to invert the question: what module-b JAR you want to have included in > app WAR, when building? The one built, or the one from the local > repository? If the latter, why are you building it in the first place? > > On Thu, Feb 8, 2024 at 12:06 PM Joseph Leonard < > joseph.leon...@alfasystems.com> wrote: > > > Hi Tamás, > > Thank you for hacking around this so rapidly. I am not familiar with the > > maven code base, so let me know if I have misinterpreted the change: Is > > your hack to: > > > > 1. Retain the full multi-module build dependency graph such that the > > build graph always remains consistent regardless of whether a “-pl” arg has > > been provided > > 2. Introduce a skip build option to satisfy the functionality of the > > “-pl” arg: The reactor will report the module has been built but the build > > is basically a no-op > > If so, this does resolve the issue but at the cost of losing the > > opportunity to gain extra parallelism in the build. > > What I was wondering may be possible (without any idea of how feasible it > > would be to implement) would be the opportunity to recognise in the > > multi-build distinct build graphs and then keep isolation in the reactor > > between these graphs. To elaborate, in the reproducer scenario there would > > be two distinct graphs: > > > > * "testsupport-module-1" followed by "app" > > * "testsupport-module-2" followed by "module-b" > > Ideally these two graphs build in parallel to each other and even if > > ultimately there is a dependency between them they NEVER resolve that > > dependency from within reactor they instead always resolve from the > > external Maven cache. The key statement here is that if a user presents a > > list of modules to be built then they really need to be sure that the > > DIRECT dependencies between these modules truly represent the build > > graph(s) they want. For tools like gitflow-incremental-builder which are > > designed to carefully work out change sets and what needs to build this > > offers a really powerful opportunity to optimise builds. I appreciate this > > functionality could also be seen as a gotcha (although tbf this already > > exists). > > Thanks, > > Joe > > > > > > On 2024/02/08 10:09:31 Tamás Cservenák wrote: > > > Seems we are on track with this. To prove my last-night theory I created > > a > > > "hack" (is really just that) and guess what? > > > It makes reproducer behave "as expected": > > > https://github.com/apache/maven/pull/1406 > > > > > > T > > > > > > On Wed, Feb 7, 2024 at 10:05 PM Tamás Cservenák > > wrote: > > > > > > > Howdy, > > > > > > > > Thank you very much, the reproducer works. Did not dig thru it fully, > > but > > > > here are some related issues: > > > > > > > > https://issues.apache.org/jira/browse/MNG-8028 (funny thing, I created > > > > this few weeks ago) > > > > https://issues.apache.org/jira/browse/MNG-6300 > > > > > > > > Will report back tomorrow (EU TZ) > > > > T > > > > > > > > > > > > On Wed, Feb 7, 2024 at 7:48 PM Joseph Leonard < > > > > joseph.leon...@alfasystems.com> wrote: > > > > > > > >> Hi Tamás, > > > >> I have created a simple example here: > > > >> https://github.com/josple/mvn-multibuild-issue > > > >> Hopefully the README is clear enough – let me know if I can clarify > > > >> anything. > > > >> Thanks, > > > >> Joe > > > >> > > > >> On 2024/02/07 17:33:08 Tamás Cservenák wrote: > > > >> > Howdy, > > > >> > > > > >> > In that case, there is something fishy with the project, my blind > > guess > > > >> > would be some "hidden" inter-module dependency maybe? > > > >> > > > > >> > Can you provide access to source, or, if not feasible, could you > > provide > > > >> > some reproducer and publish it on Github/Gitlab/whatever (maybe even > > > >> just > > > >> > send it privately as ML strips off
Re: Re: Re: Re: Re: Inconsistent dependency resolution behaviour for concurrent multi-module build can cause failures
Hej, yes, basically the "hack" is well described by you: - it retains "full' reactor project - introduces "needs build" on project - when it comes to building, it skips the project if flag set This results that module-b -> app link is not lost. And as for your parallelization request: the truth is that there IS and edge between app and module-b (via module-a), so despite module-a is "done", its transitive module-b is not done, hence you cannot build these two in parallel, as you do have an edge between app and module-b. Or to invert the question: what module-b JAR you want to have included in app WAR, when building? The one built, or the one from the local repository? If the latter, why are you building it in the first place? On Thu, Feb 8, 2024 at 12:06 PM Joseph Leonard < joseph.leon...@alfasystems.com> wrote: > Hi Tamás, > Thank you for hacking around this so rapidly. I am not familiar with the > maven code base, so let me know if I have misinterpreted the change: Is > your hack to: > > 1. Retain the full multi-module build dependency graph such that the > build graph always remains consistent regardless of whether a “-pl” arg has > been provided > 2. Introduce a skip build option to satisfy the functionality of the > “-pl” arg: The reactor will report the module has been built but the build > is basically a no-op > If so, this does resolve the issue but at the cost of losing the > opportunity to gain extra parallelism in the build. > What I was wondering may be possible (without any idea of how feasible it > would be to implement) would be the opportunity to recognise in the > multi-build distinct build graphs and then keep isolation in the reactor > between these graphs. To elaborate, in the reproducer scenario there would > be two distinct graphs: > > * "testsupport-module-1" followed by "app" > * "testsupport-module-2" followed by "module-b" > Ideally these two graphs build in parallel to each other and even if > ultimately there is a dependency between them they NEVER resolve that > dependency from within reactor they instead always resolve from the > external Maven cache. The key statement here is that if a user presents a > list of modules to be built then they really need to be sure that the > DIRECT dependencies between these modules truly represent the build > graph(s) they want. For tools like gitflow-incremental-builder which are > designed to carefully work out change sets and what needs to build this > offers a really powerful opportunity to optimise builds. I appreciate this > functionality could also be seen as a gotcha (although tbf this already > exists). > Thanks, > Joe > > > On 2024/02/08 10:09:31 Tamás Cservenák wrote: > > Seems we are on track with this. To prove my last-night theory I created > a > > "hack" (is really just that) and guess what? > > It makes reproducer behave "as expected": > > https://github.com/apache/maven/pull/1406 > > > > T > > > > On Wed, Feb 7, 2024 at 10:05 PM Tamás Cservenák > wrote: > > > > > Howdy, > > > > > > Thank you very much, the reproducer works. Did not dig thru it fully, > but > > > here are some related issues: > > > > > > https://issues.apache.org/jira/browse/MNG-8028 (funny thing, I created > > > this few weeks ago) > > > https://issues.apache.org/jira/browse/MNG-6300 > > > > > > Will report back tomorrow (EU TZ) > > > T > > > > > > > > > On Wed, Feb 7, 2024 at 7:48 PM Joseph Leonard < > > > joseph.leon...@alfasystems.com> wrote: > > > > > >> Hi Tamás, > > >> I have created a simple example here: > > >> https://github.com/josple/mvn-multibuild-issue > > >> Hopefully the README is clear enough – let me know if I can clarify > > >> anything. > > >> Thanks, > > >> Joe > > >> > > >> On 2024/02/07 17:33:08 Tamás Cservenák wrote: > > >> > Howdy, > > >> > > > >> > In that case, there is something fishy with the project, my blind > guess > > >> > would be some "hidden" inter-module dependency maybe? > > >> > > > >> > Can you provide access to source, or, if not feasible, could you > provide > > >> > some reproducer and publish it on Github/Gitlab/whatever (maybe even > > >> just > > >> > send it privately as ML strips off attachments and images) for us > to see > > >> > this in action? > > >> > > > >> > Thanks > > >> > T > > >> > > > >> > On Wed, Feb 7, 2024 at 6:29 PM Joseph Leonard < > > >> > joseph.leon...@alfasystems.com> wrote: > > >> > > > >> > > Hi Tamás, > > >> > > We have previously played around a bit with mvnd but not takari > > >> directly – > > >> > > I will have a play with it. With regards to this issue, using the > > >> takari > > >> > > smart builder unfortunately doesn’t resolve the issue. > > >> > > Joe > > >> > > > > >> > > On 2024/02/07 11:41:22 Tamás Cservenák wrote: > > >> > > > Can you please try smart builder instead? > > >> > > > https://github.com/takari/takari-smart-builder > > >> > > > > > >> > > > (note: smart builder is used by mvnd as well) > > >> > > > > > >> > > > The difference between the two
RE: Re: Re: Re: Re: Inconsistent dependency resolution behaviour for concurrent multi-module build can cause failures
Hi Tamás, Thank you for hacking around this so rapidly. I am not familiar with the maven code base, so let me know if I have misinterpreted the change: Is your hack to: 1. Retain the full multi-module build dependency graph such that the build graph always remains consistent regardless of whether a “-pl” arg has been provided 2. Introduce a skip build option to satisfy the functionality of the “-pl” arg: The reactor will report the module has been built but the build is basically a no-op If so, this does resolve the issue but at the cost of losing the opportunity to gain extra parallelism in the build. What I was wondering may be possible (without any idea of how feasible it would be to implement) would be the opportunity to recognise in the multi-build distinct build graphs and then keep isolation in the reactor between these graphs. To elaborate, in the reproducer scenario there would be two distinct graphs: * "testsupport-module-1" followed by "app" * "testsupport-module-2" followed by "module-b" Ideally these two graphs build in parallel to each other and even if ultimately there is a dependency between them they NEVER resolve that dependency from within reactor they instead always resolve from the external Maven cache. The key statement here is that if a user presents a list of modules to be built then they really need to be sure that the DIRECT dependencies between these modules truly represent the build graph(s) they want. For tools like gitflow-incremental-builder which are designed to carefully work out change sets and what needs to build this offers a really powerful opportunity to optimise builds. I appreciate this functionality could also be seen as a gotcha (although tbf this already exists). Thanks, Joe On 2024/02/08 10:09:31 Tamás Cservenák wrote: > Seems we are on track with this. To prove my last-night theory I created a > "hack" (is really just that) and guess what? > It makes reproducer behave "as expected": > https://github.com/apache/maven/pull/1406 > > T > > On Wed, Feb 7, 2024 at 10:05 PM Tamás Cservenák wrote: > > > Howdy, > > > > Thank you very much, the reproducer works. Did not dig thru it fully, but > > here are some related issues: > > > > https://issues.apache.org/jira/browse/MNG-8028 (funny thing, I created > > this few weeks ago) > > https://issues.apache.org/jira/browse/MNG-6300 > > > > Will report back tomorrow (EU TZ) > > T > > > > > > On Wed, Feb 7, 2024 at 7:48 PM Joseph Leonard < > > joseph.leon...@alfasystems.com> wrote: > > > >> Hi Tamás, > >> I have created a simple example here: > >> https://github.com/josple/mvn-multibuild-issue > >> Hopefully the README is clear enough – let me know if I can clarify > >> anything. > >> Thanks, > >> Joe > >> > >> On 2024/02/07 17:33:08 Tamás Cservenák wrote: > >> > Howdy, > >> > > >> > In that case, there is something fishy with the project, my blind guess > >> > would be some "hidden" inter-module dependency maybe? > >> > > >> > Can you provide access to source, or, if not feasible, could you provide > >> > some reproducer and publish it on Github/Gitlab/whatever (maybe even > >> just > >> > send it privately as ML strips off attachments and images) for us to see > >> > this in action? > >> > > >> > Thanks > >> > T > >> > > >> > On Wed, Feb 7, 2024 at 6:29 PM Joseph Leonard < > >> > joseph.leon...@alfasystems.com> wrote: > >> > > >> > > Hi Tamás, > >> > > We have previously played around a bit with mvnd but not takari > >> directly – > >> > > I will have a play with it. With regards to this issue, using the > >> takari > >> > > smart builder unfortunately doesn’t resolve the issue. > >> > > Joe > >> > > > >> > > On 2024/02/07 11:41:22 Tamás Cservenák wrote: > >> > > > Can you please try smart builder instead? > >> > > > https://github.com/takari/takari-smart-builder > >> > > > > >> > > > (note: smart builder is used by mvnd as well) > >> > > > > >> > > > The difference between the two can be seen here: > >> > > > http://takari.io/book/30-team-maven.html#takari-smart-builder > >> > > > > >> > > > On Wed, Feb 7, 2024 at 11:50 AM Joseph Leonard < > >> > > > joseph.leon...@alfasystems.com> wrote: > >> > > > > >> > > > > Hi Tamás, > >> > > > > Yeah, this was unexpected to me initially as well. From what I > >> can tell > >> > > > > the Maven reactor only considers direct dependencies (i.e. not > >> > > transitive > >> > > > > dependencies) between the modules in the reactor when working out > >> the > >> > > build > >> > > > > graph. For example if you have a simple linear dependency chain > >> of: > >> > > > > One --> Two --> Three --> Four --> Five > >> > > > > Then invoking “mvn clean verify -pl One,Two,Four,Five -T 2 will > >> result > >> > > in > >> > > > > two ‘graphs’ being built in parallel ([One,Two] and [Four,Five]). > >> I > >> > > assume > >> > > > > this is as designed because it actually offers quite powerful > >> > > functionality > >> > > > > to improve the parallelism in your
