[jira] [Created] (MBUILDCACHE-95) Improve efficiency of cache when adding/removing modules
Marquis Wang created MBUILDCACHE-95: --- Summary: Improve efficiency of cache when adding/removing modules Key: MBUILDCACHE-95 URL: https://issues.apache.org/jira/browse/MBUILDCACHE-95 Project: Maven Build Cache Extension Issue Type: Improvement Reporter: Marquis Wang Right now, if you add or remove a module from your multi-module project, it invalidates your cache for your entire project, since the base pom has changed. (My project adds all modules to the dependency management of the base pom.) This is silly, since * When you remove a module, you should be able to reuse all of the cache * When you add a module, you should be able to build only that one new module -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-94) Provide a build cache report on builds
[
https://issues.apache.org/jira/browse/MBUILDCACHE-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17841906#comment-17841906
]
Marquis Wang commented on MBUILDCACHE-94:
-
Never mind, I see now that a report is put in {{target/maven-incremental/}} of
the base module.
I was saving {{target/incremental-maven}} as build artifacts, so I missed this.
It would be nice if the log message below included the path.
{code}
[INFO] Saving cache report on build completion
{code}
Additionally... I assume the inconsistency between `incremental-maven` and
`maven-incremental` is unintended?
> Provide a build cache report on builds
> --
>
> Key: MBUILDCACHE-94
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-94
> Project: Maven Build Cache Extension
> Issue Type: Improvement
>Reporter: Marquis Wang
>Priority: Major
>
> When building with the build cache, it would be really helpful to provide a
> report of which modules were unable to utilize and cache, as well as
> information about why.
> Right now, when looking at build cache performance, we can use
> {{maven.build.cache.failFast}} and {{maven.build.cache.baselineUrl}} to
> understand which module(s) were unable to utilize the build cache and why.
> However, this process is entirely manual.
> It would be really helpful to provide some base cache statistics on every
> run, even if no baselineUrl is provided. That way, we can gather overall
> statistics and figure out how to change our project to better utilize the
> cache.
> For example, we have a multi-module project with hundreds of modules. I want
> to know if there is one module that is constantly changing and is depended on
> (possibly unnecessarily) by lots of other modules, which is causing cache
> misses.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (MBUILDCACHE-94) Provide a build cache report on builds
Marquis Wang created MBUILDCACHE-94:
---
Summary: Provide a build cache report on builds
Key: MBUILDCACHE-94
URL: https://issues.apache.org/jira/browse/MBUILDCACHE-94
Project: Maven Build Cache Extension
Issue Type: Improvement
Reporter: Marquis Wang
When building with the build cache, it would be really helpful to provide a
report of which modules were unable to utilize and cache, as well as
information about why.
Right now, when looking at build cache performance, we can use
{{maven.build.cache.failFast}} and {{maven.build.cache.baselineUrl}} to
understand which module(s) were unable to utilize the build cache and why.
However, this process is entirely manual.
It would be really helpful to provide some base cache statistics on every run,
even if no baselineUrl is provided. That way, we can gather overall statistics
and figure out how to change our project to better utilize the cache.
For example, we have a multi-module project with hundreds of modules. I want to
know if there is one module that is constantly changing and is depended on
(possibly unnecessarily) by lots of other modules, which is causing cache
misses.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-59) Accommodate Maven CI-friendly versions
[
https://issues.apache.org/jira/browse/MBUILDCACHE-59?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17841177#comment-17841177
]
Marquis Wang commented on MBUILDCACHE-59:
-
Actually [~ralaoui] [~bverp] I've discovered that maven-compiler-plugin 3.12.0
and higher make the version field optional in {{annotationProcessorPaths}}:
https://maven.apache.org/plugins/maven-compiler-plugin/compile-mojo.html#annotationProcessorPaths
{code}
1.2.3
{code}
> Accommodate Maven CI-friendly versions
> --
>
> Key: MBUILDCACHE-59
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-59
> Project: Maven Build Cache Extension
> Issue Type: Improvement
>Affects Versions: 1.0.0
>Reporter: Bas van Erp
>Priority: Major
>
> I would argue that the Maven Build Cache Extension (MBCE) can provide the
> greatest value for users that utilize large multi-module (mono-)repositories,
> since those projects tend to have large amounts of "stable" code which isn't
> modified frequently.
> Speeding up the build of these multi-module projects using the MBCE would be
> totally awesome!
> But for my particular project (200K LOC, 400 modules, mono-repo) our initial
> trials with the MBCE have proven somewhat troublesome.
> It has to do with our use of [https://maven.apache.org/maven-ci-friendly.html]
> h2. Example setup
> {{repo/.mvn/maven.config}}
> {code:java}
> -Drevision=0.0.0-local-SNAPSHOT {code}
> {{repo/pom.xml}}
> {code:java}
> com.corp
> parent
> ${revision}
> pom {code}
> {{repo/moduleX/pom.xml}}
> {code:java}
>
> com.corp
> parent
> ${revision}
> ../pom.xml
>
> moduleX
> jar{code}
> {{repo/moduleY/pom.xml}}
> {code:java}
>
> com.corp
> parent
> ${revision}
> ../pom.xml
>
> moduleY
> jar
>
>
> com.corp
> moduleX
> ${revision}
>
> {code}
> {{repo/Jenkinsfile}}
> {code:java}
> mvn clean deploy -Drevision=$(git describe){code}
> h2. Effect
> This setup effectively means that while all our 150 developers might
> theoretically get decent cache hit % since they are all always using version
> {{0.0.0-local-SNAPSHOT}} for every single POM for every single build, our CI
> pipelines are out of luck.
> Since the effective-POMs will change radically with each new Git commit, the
> hashes will virtually never collide.
> h2. Solution?
> I'm very new to the MBCE so I might be way off the mark, but I think
> accommodating this setup will not be trivial to implement. It would probably
> require parsing the effective-POM for every module picked-up by the reactor
> and then removing all clauses of the module, the parent, and each
> dependency and plugin which are part of the reactor build.
> Doesn't sound easy to me. But perhaps I'm missing something.
>
> p.s. Congrats on the v1.0.0 release :). I'm enjoying experimenting with it.
> But the documentation could use some love. It's pretty hard to read. Some
> paragraphs are confusing, terse or lacking in detail, are clearly written by
> non-English-native writers (like myself), or even stop mid-sentence.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-59) Accommodate Maven CI-friendly versions
[
https://issues.apache.org/jira/browse/MBUILDCACHE-59?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17840386#comment-17840386
]
Marquis Wang commented on MBUILDCACHE-59:
-
I am observing the same thing - it seems to work correctly for dependencies,
but not with annotationProcessorPaths.
> Accommodate Maven CI-friendly versions
> --
>
> Key: MBUILDCACHE-59
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-59
> Project: Maven Build Cache Extension
> Issue Type: Improvement
>Affects Versions: 1.0.0
>Reporter: Bas van Erp
>Priority: Major
>
> I would argue that the Maven Build Cache Extension (MBCE) can provide the
> greatest value for users that utilize large multi-module (mono-)repositories,
> since those projects tend to have large amounts of "stable" code which isn't
> modified frequently.
> Speeding up the build of these multi-module projects using the MBCE would be
> totally awesome!
> But for my particular project (200K LOC, 400 modules, mono-repo) our initial
> trials with the MBCE have proven somewhat troublesome.
> It has to do with our use of [https://maven.apache.org/maven-ci-friendly.html]
> h2. Example setup
> {{repo/.mvn/maven.config}}
> {code:java}
> -Drevision=0.0.0-local-SNAPSHOT {code}
> {{repo/pom.xml}}
> {code:java}
> com.corp
> parent
> ${revision}
> pom {code}
> {{repo/moduleX/pom.xml}}
> {code:java}
>
> com.corp
> parent
> ${revision}
> ../pom.xml
>
> moduleX
> jar{code}
> {{repo/moduleY/pom.xml}}
> {code:java}
>
> com.corp
> parent
> ${revision}
> ../pom.xml
>
> moduleY
> jar
>
>
> com.corp
> moduleX
> ${revision}
>
> {code}
> {{repo/Jenkinsfile}}
> {code:java}
> mvn clean deploy -Drevision=$(git describe){code}
> h2. Effect
> This setup effectively means that while all our 150 developers might
> theoretically get decent cache hit % since they are all always using version
> {{0.0.0-local-SNAPSHOT}} for every single POM for every single build, our CI
> pipelines are out of luck.
> Since the effective-POMs will change radically with each new Git commit, the
> hashes will virtually never collide.
> h2. Solution?
> I'm very new to the MBCE so I might be way off the mark, but I think
> accommodating this setup will not be trivial to implement. It would probably
> require parsing the effective-POM for every module picked-up by the reactor
> and then removing all clauses of the module, the parent, and each
> dependency and plugin which are part of the reactor build.
> Doesn't sound easy to me. But perhaps I'm missing something.
>
> p.s. Congrats on the v1.0.0 release :). I'm enjoying experimenting with it.
> But the documentation could use some love. It's pretty hard to read. Some
> paragraphs are confusing, terse or lacking in detail, are clearly written by
> non-English-native writers (like myself), or even stop mid-sentence.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MCOMPILER-333) Incremental compilation causes "mvn clean compile compile" to fail
[ https://issues.apache.org/jira/browse/MCOMPILER-333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17714763#comment-17714763 ] Marquis Wang commented on MCOMPILER-333: I have an example project that shows this: https://github.com/marquiswang/incremental-compile-dagger-test > Incremental compilation causes "mvn clean compile compile" to fail > -- > > Key: MCOMPILER-333 > URL: https://issues.apache.org/jira/browse/MCOMPILER-333 > Project: Maven Compiler Plugin > Issue Type: Bug >Affects Versions: 3.7.0 >Reporter: Anthony Vanelverdinghe >Priority: Major > > With true, my build > fails if I do "mvn clean compile compile". From the > [comment|https://issues.apache.org/jira/browse/MCOMPILER-205?focusedCommentId=16390065=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16390065] > in MCOMPILER-205, I understand that incremental compilation deletes the > "classes" directory. But it seems that it doesn't delete the > "generated-sources" directory. > When I do "mvn clean compile" and "mvn compile", the second build fails. > However, if I delete the "generated-sources" directory between the 2 builds, > they both succeed. > So I guess the bug here is that incremental compilation doesn't delete the > "generated-sources" directory. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MCOMPILER-532) compile doesn't work without first clean with Dagger annotation processor
Marquis Wang created MCOMPILER-532: -- Summary: compile doesn't work without first clean with Dagger annotation processor Key: MCOMPILER-532 URL: https://issues.apache.org/jira/browse/MCOMPILER-532 Project: Maven Compiler Plugin Issue Type: Bug Reporter: Marquis Wang I have a reproducing example here: https://github.com/marquiswang/incremental-compile-dagger-test. The README there shows how it goes wrong. Long story short, if you use mvn compile on a project with Dagger, touch the @Injected files, then do mvn compile again, then it will not correctly compile the Dagger-generated factory methods. You need to do a mvn clean compile. I am pretty sure that this is because of how maven-compiler-plugin and javac interact with Dagger. maven-compiler-plugin will, if it detects that a file has changed, delete compiled class files and recompile every class in the module. However, it does not delete target/generated-sources/annotations/. It calls javac on the Foo.java. Dagger chooses not to re-generate the Foo_Factory.java file, presumably because it already exists. javac detects that no new files were created by annotation processing, so it doesn't compile anything. According to https://docs.oracle.com/en/java/javase/17/docs/specs/man/javac.html#annotation-processing , javac only compiles "newly generated source files", so it doesn't compile the existing Foo_Factory.java. I think the solution is, when maven-compiler-plugin detects that sources have changed it and it needs to re-compile, it should delete target/generated-sources/annotations as well as target/classes before compiling. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MENFORCER-325) Add binary compatibility check to dependencyConvergence rule
Marquis Wang created MENFORCER-325: -- Summary: Add binary compatibility check to dependencyConvergence rule Key: MENFORCER-325 URL: https://issues.apache.org/jira/browse/MENFORCER-325 Project: Maven Enforcer Plugin Issue Type: New Feature Reporter: Marquis Wang The dependencyConvergence rule is too strict to use in practice most of the time. If you have a project with even a medium number of dependencies, they are pretty much guaranteed to have transitive dependencies that do not converge. For some very common dependencies like Guava, the maintainers of the project are very careful to maintain binary compatibility with new releases. For these dependencies, a dependency divergence is okay if the included node is binary compatible with all of the diverging nodes. For example, if you have two dependencies, one if which depends on Guava 27 and the other depends on Guava 26, you can safely run with Guava 27 because you know that it is compatible with 26. It would be really helpful if the dependencyConvergence rule were able to check for binary compatibility and take it into account. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
