[jira] [Commented] (MNG-7852) Use all the versions for dependency resolution rather than "nearest first" or "declared first"
[
https://issues.apache.org/jira/browse/MNG-7852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17749174#comment-17749174
]
Vladimir Sitnikov commented on MNG-7852:
{quote} Best would be to move this discussion to ML instead{quote}
Would you please clarify why do you think the issue belongs to ML rather than
JIRA?
I do not understand the reason, and I do not want to get a "file JIRA instead"
response on the ML.
{quote}Also, you mix runtime and compile time things ("nobody ensure that
people can compile code")
{quote}
Correct me if I am wrong, but the full cite is "nobody ensures that people can
compile code with the recent version {*}and supply an old version in the
execution{*}". "and supply an old version in the execution" is important as
otherwise, the bug won't trigger.
Correct me if I am wrong, but Java requires compile code before one can proceed
with the execution, and it does matter which third-party libraries were present
on the compilation classpath as it affects the resulting bytecode.
My example says that projects rarely ensure their jars can be safely downgraded.
{quote} You should do your homework{quote}
Sure thing.
Would you please respond to
https://issues.apache.org/jira/browse/MNG-7852?focusedCommentId=17748173=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17748173
or
https://issues.apache.org/jira/browse/MNG-7852?focusedCommentId=17748217=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17748217
?
In my examples, the problematic transitive dependencies come from **other**
dependencies, and I do not use the problematic ones directly.
I do not understand which of them you suggest to "declare as direct".
If you suggest declaring all the possible transitive dependencies (including
transitives of transitives) as direct, then, well, the re-declaring
dependencies approach is prone to errors I described in
https://issues.apache.org/jira/browse/MNG-7852?focusedCommentId=17748677=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17748677.
> Use all the versions for dependency resolution rather than "nearest first" or
> "declared first"
> --
>
> Key: MNG-7852
> URL: https://issues.apache.org/jira/browse/MNG-7852
> Project: Maven
> Issue Type: Improvement
> Components: Dependencies
>Reporter: Vladimir Sitnikov
>Priority: Major
>
> Currently, Maven uses "nearest first", "declared first" rules for conflict
> resolution:
> https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html
> I suggest those rules are removed since they produce hard to reason
> resolutions for transitive dependencies.
> Below I list reasons why both "nearest wins" and "declared first" yield
> hard-to-predict behaviours, and they are likely to produce dependency
> downgrades and the associated runtime errors.
> Here are some examples:
> 1) "Nearest first". Even though the rule sounds easy, it is not something the
> users can control. For instance, if the project does not use Guava library,
> some of the transitive dependencies could add a dependency on Guava. The user
> has no control which dependency would be "the nearest" to declare Guava, so
> user has literally no way to tell which Guava version will be used.
> The only workaround I see for the users is to declare Guava dependency
> explicitly even though the project does not need it directly. It sounds like
> Maven requires users to re-declare all the possible dependencies, including
> the runtime-only ones.
> 2) "declared first". Of course, dependency order matters for classpath order,
> however, it is not predictable in practice, and it might result in
> downgrading dependencies. Imagine the project does not use Guice. However,
> transitive dependencies might use Guice. At the same time, they might start
> using Guice and stop using Guice, so the user can never tell which will be
> the first project that uses Guice. Unfortunately, in Maven, the first project
> that declares dependency wins, so it might easily be the case that the first
> mention of Guice will reference outdated version that would be incompatible
> with the newer one required in another dependency.
> 3) Here's a real-life case: Maven downgrades protobuf-java dependency causing
> something like NoSuchMethodError at the runtime. The step to reproduce is to
> add dependency on dev.sigstore:sigstore-java:0.4.0. See [~hboutemy] analysis
> in https://github.com/hboutemy/sigstore-maven-plugin/blob/import/analysis.md
> Long story short, sigstore-java does not depend on protobuf-java directly,
> however, sigstore-java depends on several third-party libraries that
> eventually depend on protobuf-java. Maven's "the first
[jira] [Commented] (MNG-7852) Use all the versions for dependency resolution rather than "nearest first" or "declared first"
[
https://issues.apache.org/jira/browse/MNG-7852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17749139#comment-17749139
]
Tamas Cservenak commented on MNG-7852:
--
You are very wrong by putting statements like these, I'd really say this is
FUD. Maven chooses "nearest" version and is predictable and even reproducible.
And while blaming something else instead of yourself, due lack of knowledge
about it, is really in human nature, don't fall for it.
Also, you mix runtime and compile time things ("nobody ensure that people can
compile code"). Once you mention guava as runtime dependency, then you draw
conclusion about code compilation. You should do your homework: if you compile
against something, best practice is to declare it as direct dependency (and not
rely on some 3rd level transitive dependency). Tools like Takari Lifecycle even
ensure this: if you even touch a class that comes from any other dependency
that first level ones (those in POM), your build will fail.
So please, stop bombing JIRA, as am still by my original statement: "Best would
be to move this discussion to ML instead, preferably the users mailing list
https://maven.apache.org/mailing-lists.html;
> Use all the versions for dependency resolution rather than "nearest first" or
> "declared first"
> --
>
> Key: MNG-7852
> URL: https://issues.apache.org/jira/browse/MNG-7852
> Project: Maven
> Issue Type: Improvement
> Components: Dependencies
>Reporter: Vladimir Sitnikov
>Priority: Major
>
> Currently, Maven uses "nearest first", "declared first" rules for conflict
> resolution:
> https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html
> I suggest those rules are removed since they produce hard to reason
> resolutions for transitive dependencies.
> Below I list reasons why both "nearest wins" and "declared first" yield
> hard-to-predict behaviours, and they are likely to produce dependency
> downgrades and the associated runtime errors.
> Here are some examples:
> 1) "Nearest first". Even though the rule sounds easy, it is not something the
> users can control. For instance, if the project does not use Guava library,
> some of the transitive dependencies could add a dependency on Guava. The user
> has no control which dependency would be "the nearest" to declare Guava, so
> user has literally no way to tell which Guava version will be used.
> The only workaround I see for the users is to declare Guava dependency
> explicitly even though the project does not need it directly. It sounds like
> Maven requires users to re-declare all the possible dependencies, including
> the runtime-only ones.
> 2) "declared first". Of course, dependency order matters for classpath order,
> however, it is not predictable in practice, and it might result in
> downgrading dependencies. Imagine the project does not use Guice. However,
> transitive dependencies might use Guice. At the same time, they might start
> using Guice and stop using Guice, so the user can never tell which will be
> the first project that uses Guice. Unfortunately, in Maven, the first project
> that declares dependency wins, so it might easily be the case that the first
> mention of Guice will reference outdated version that would be incompatible
> with the newer one required in another dependency.
> 3) Here's a real-life case: Maven downgrades protobuf-java dependency causing
> something like NoSuchMethodError at the runtime. The step to reproduce is to
> add dependency on dev.sigstore:sigstore-java:0.4.0. See [~hboutemy] analysis
> in https://github.com/hboutemy/sigstore-maven-plugin/blob/import/analysis.md
> Long story short, sigstore-java does not depend on protobuf-java directly,
> however, sigstore-java depends on several third-party libraries that
> eventually depend on protobuf-java. Maven's "the first wins" behaviour
> results in incoherent set of protobuf dependencies on the classpath.
> 4) see "unexpected" in MNG-5988
> To my best understanding, when it comes to transitive dependencies, both
> "nearest first" and "declared first" are random variables which user can't
> control unless they re-declare all the dependencies in their local pom. I
> suggest Maven should not use random variables like "dependency depth" or
> "dependency order" to drive conflict resolution.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MNG-7852) Use all the versions for dependency resolution rather than "nearest first" or "declared first"
[
https://issues.apache.org/jira/browse/MNG-7852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17749133#comment-17749133
]
Vladimir Sitnikov commented on MNG-7852:
{quote}There is no proper versions nevermind which version we choose during
resolving transitive dependencies, alway we can have an incompatibilities
between two versions.
{quote}
The current Maven's algorithm results in *{*}downgrading{*}* versions.
Downgrading dependency versions is very often an incompatible change.
On the other hand, upgrading a version is more likely to produce a workable
classpath because many libraries maintain backward compatibility (== further
versions are compatible with previous versions).
Let us quantify this.
Imagine lib-a with Guava 21 (see
https://issues.apache.org/jira/browse/MNG-7852?focusedCommentId=17748217=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17748217)
Imagine I use lib-a in my project.
Imagine Maven selects Guava 19 for the runtime (e.g., another library
introduced guava-19 a little bit earlier in the dependency tree).
Of course, the thing will fail at the runtime because lib-a might easily depend
on a method added in guava-21.
Who will be to blame? Of course, {*}Maven would be to blame{*}, as it
downgraded guava from 21 to 19.
Imagine an alternative case.
Imagine I use lib-a in my project.
Imagine Maven selects Guava 30 for the runtime.
Of course the thing might fail at the runtime (e.g. if Guava 30 accidentally
fails to keep backward compatibility).
Who will be to blame in such case? Of course, {*}it would be Guava to blame{*},
and it is unlikely to break as Guava team tests backward compatibility.
In other words, almost no library tests for "forward compatibility". In other
words, almost nobody ensures that people can compile code with the recent
version and supply an old version in the execution.
At the same time, many teams verify backward compatibility. That means,
upgrading dependencies is much safer than downgrading them.
The current Maven's algorithm seems to select semi-random transitive versions
which are both unpredictable and might cause downgrades.
> Use all the versions for dependency resolution rather than "nearest first" or
> "declared first"
> --
>
> Key: MNG-7852
> URL: https://issues.apache.org/jira/browse/MNG-7852
> Project: Maven
> Issue Type: Improvement
> Components: Dependencies
>Reporter: Vladimir Sitnikov
>Priority: Major
>
> Currently, Maven uses "nearest first", "declared first" rules for conflict
> resolution:
> https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html
> I suggest those rules are removed since they produce hard to reason
> resolutions for transitive dependencies.
> Below I list reasons why both "nearest wins" and "declared first" yield
> hard-to-predict behaviours, and they are likely to produce dependency
> downgrades and the associated runtime errors.
> Here are some examples:
> 1) "Nearest first". Even though the rule sounds easy, it is not something the
> users can control. For instance, if the project does not use Guava library,
> some of the transitive dependencies could add a dependency on Guava. The user
> has no control which dependency would be "the nearest" to declare Guava, so
> user has literally no way to tell which Guava version will be used.
> The only workaround I see for the users is to declare Guava dependency
> explicitly even though the project does not need it directly. It sounds like
> Maven requires users to re-declare all the possible dependencies, including
> the runtime-only ones.
> 2) "declared first". Of course, dependency order matters for classpath order,
> however, it is not predictable in practice, and it might result in
> downgrading dependencies. Imagine the project does not use Guice. However,
> transitive dependencies might use Guice. At the same time, they might start
> using Guice and stop using Guice, so the user can never tell which will be
> the first project that uses Guice. Unfortunately, in Maven, the first project
> that declares dependency wins, so it might easily be the case that the first
> mention of Guice will reference outdated version that would be incompatible
> with the newer one required in another dependency.
> 3) Here's a real-life case: Maven downgrades protobuf-java dependency causing
> something like NoSuchMethodError at the runtime. The step to reproduce is to
> add dependency on dev.sigstore:sigstore-java:0.4.0. See [~hboutemy] analysis
> in https://github.com/hboutemy/sigstore-maven-plugin/blob/import/analysis.md
> Long story short, sigstore-java does not depend on protobuf-java directly,
> however, sigstore-java depends on
[jira] [Commented] (MNG-7852) Use all the versions for dependency resolution rather than "nearest first" or "declared first"
[
https://issues.apache.org/jira/browse/MNG-7852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748682#comment-17748682
]
Slawomir Jaranowski commented on MNG-7852:
--
[~vladimirsitnikov] There is no {{proper versions}} nevermind which version we
choose during resolving transitive dependencies, alway we can have an
incompatibilities between two versions.
Enforcer rule can help to detect such situation, and yes you need redeclared
dependencies in dep or depManagement in your project.
Finally tests unit and integration should confirm that your application is
working.
> Use all the versions for dependency resolution rather than "nearest first" or
> "declared first"
> --
>
> Key: MNG-7852
> URL: https://issues.apache.org/jira/browse/MNG-7852
> Project: Maven
> Issue Type: Improvement
> Components: Dependencies
>Reporter: Vladimir Sitnikov
>Priority: Major
>
> Currently, Maven uses "nearest first", "declared first" rules for conflict
> resolution:
> https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html
> I suggest those rules are removed since they produce hard to reason
> resolutions for transitive dependencies.
> Below I list reasons why both "nearest wins" and "declared first" yield
> hard-to-predict behaviours, and they are likely to produce dependency
> downgrades and the associated runtime errors.
> Here are some examples:
> 1) "Nearest first". Even though the rule sounds easy, it is not something the
> users can control. For instance, if the project does not use Guava library,
> some of the transitive dependencies could add a dependency on Guava. The user
> has no control which dependency would be "the nearest" to declare Guava, so
> user has literally no way to tell which Guava version will be used.
> The only workaround I see for the users is to declare Guava dependency
> explicitly even though the project does not need it directly. It sounds like
> Maven requires users to re-declare all the possible dependencies, including
> the runtime-only ones.
> 2) "declared first". Of course, dependency order matters for classpath order,
> however, it is not predictable in practice, and it might result in
> downgrading dependencies. Imagine the project does not use Guice. However,
> transitive dependencies might use Guice. At the same time, they might start
> using Guice and stop using Guice, so the user can never tell which will be
> the first project that uses Guice. Unfortunately, in Maven, the first project
> that declares dependency wins, so it might easily be the case that the first
> mention of Guice will reference outdated version that would be incompatible
> with the newer one required in another dependency.
> 3) Here's a real-life case: Maven downgrades protobuf-java dependency causing
> something like NoSuchMethodError at the runtime. The step to reproduce is to
> add dependency on dev.sigstore:sigstore-java:0.4.0. See [~hboutemy] analysis
> in https://github.com/hboutemy/sigstore-maven-plugin/blob/import/analysis.md
> Long story short, sigstore-java does not depend on protobuf-java directly,
> however, sigstore-java depends on several third-party libraries that
> eventually depend on protobuf-java. Maven's "the first wins" behaviour
> results in incoherent set of protobuf dependencies on the classpath.
> 4) see "unexpected" in MNG-5988
> To my best understanding, when it comes to transitive dependencies, both
> "nearest first" and "declared first" are random variables which user can't
> control unless they re-declare all the dependencies in their local pom. I
> suggest Maven should not use random variables like "dependency depth" or
> "dependency order" to drive conflict resolution.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MNG-7852) Use all the versions for dependency resolution rather than "nearest first" or "declared first"
[
https://issues.apache.org/jira/browse/MNG-7852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748677#comment-17748677
]
Vladimir Sitnikov commented on MNG-7852:
[~sjaranowski] , thanks for mentioning dependencyConvergence, however, I do not
think it is a generally recommended solution unless you make the said enforcer
rule enabled by default in all Maven installations.
enforcer does not help users "resolve the conflicts properly". I agree enforcer
might make the dependency resolution issue easier to spot, however, it does not
treat the root cause. The idea behind the current issue is to tune Maven's
behaviour so it resolves proper versions in the first place rather than perform
silent downgrades.
{quote}You need resolve conflicts in your project
{quote}
Could you please clarify what you mean by "resolve conflicts"? Correct me if I
am wrong, but it means "re-declaring all (conflicting?) transitive
dependencies" which is a bad idea because re-declared dependencies might get
out of date (e.g. they might become stale, bad versions, and so on).
> Use all the versions for dependency resolution rather than "nearest first" or
> "declared first"
> --
>
> Key: MNG-7852
> URL: https://issues.apache.org/jira/browse/MNG-7852
> Project: Maven
> Issue Type: Improvement
> Components: Dependencies
>Reporter: Vladimir Sitnikov
>Priority: Major
>
> Currently, Maven uses "nearest first", "declared first" rules for conflict
> resolution:
> https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html
> I suggest those rules are removed since they produce hard to reason
> resolutions for transitive dependencies.
> Below I list reasons why both "nearest wins" and "declared first" yield
> hard-to-predict behaviours, and they are likely to produce dependency
> downgrades and the associated runtime errors.
> Here are some examples:
> 1) "Nearest first". Even though the rule sounds easy, it is not something the
> users can control. For instance, if the project does not use Guava library,
> some of the transitive dependencies could add a dependency on Guava. The user
> has no control which dependency would be "the nearest" to declare Guava, so
> user has literally no way to tell which Guava version will be used.
> The only workaround I see for the users is to declare Guava dependency
> explicitly even though the project does not need it directly. It sounds like
> Maven requires users to re-declare all the possible dependencies, including
> the runtime-only ones.
> 2) "declared first". Of course, dependency order matters for classpath order,
> however, it is not predictable in practice, and it might result in
> downgrading dependencies. Imagine the project does not use Guice. However,
> transitive dependencies might use Guice. At the same time, they might start
> using Guice and stop using Guice, so the user can never tell which will be
> the first project that uses Guice. Unfortunately, in Maven, the first project
> that declares dependency wins, so it might easily be the case that the first
> mention of Guice will reference outdated version that would be incompatible
> with the newer one required in another dependency.
> 3) Here's a real-life case: Maven downgrades protobuf-java dependency causing
> something like NoSuchMethodError at the runtime. The step to reproduce is to
> add dependency on dev.sigstore:sigstore-java:0.4.0. See [~hboutemy] analysis
> in https://github.com/hboutemy/sigstore-maven-plugin/blob/import/analysis.md
> Long story short, sigstore-java does not depend on protobuf-java directly,
> however, sigstore-java depends on several third-party libraries that
> eventually depend on protobuf-java. Maven's "the first wins" behaviour
> results in incoherent set of protobuf dependencies on the classpath.
> 4) see "unexpected" in MNG-5988
> To my best understanding, when it comes to transitive dependencies, both
> "nearest first" and "declared first" are random variables which user can't
> control unless they re-declare all the dependencies in their local pom. I
> suggest Maven should not use random variables like "dependency depth" or
> "dependency order" to drive conflict resolution.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MNG-7852) Use all the versions for dependency resolution rather than "nearest first" or "declared first"
[ https://issues.apache.org/jira/browse/MNG-7852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748666#comment-17748666 ] Slawomir Jaranowski commented on MNG-7852: -- Did you consider use https://maven.apache.org/enforcer/enforcer-rules/dependencyConvergence.html Then all artifacts without version conflict will be resolved as is. You need resolve conflicts in your project. There is next rule which can help you: https://maven.apache.org/enforcer/enforcer-rules/requireUpperBoundDeps.html > Use all the versions for dependency resolution rather than "nearest first" or > "declared first" > -- > > Key: MNG-7852 > URL: https://issues.apache.org/jira/browse/MNG-7852 > Project: Maven > Issue Type: Improvement > Components: Dependencies >Reporter: Vladimir Sitnikov >Priority: Major > > Currently, Maven uses "nearest first", "declared first" rules for conflict > resolution: > https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html > I suggest those rules are removed since they produce hard to reason > resolutions for transitive dependencies. > Below I list reasons why both "nearest wins" and "declared first" yield > hard-to-predict behaviours, and they are likely to produce dependency > downgrades and the associated runtime errors. > Here are some examples: > 1) "Nearest first". Even though the rule sounds easy, it is not something the > users can control. For instance, if the project does not use Guava library, > some of the transitive dependencies could add a dependency on Guava. The user > has no control which dependency would be "the nearest" to declare Guava, so > user has literally no way to tell which Guava version will be used. > The only workaround I see for the users is to declare Guava dependency > explicitly even though the project does not need it directly. It sounds like > Maven requires users to re-declare all the possible dependencies, including > the runtime-only ones. > 2) "declared first". Of course, dependency order matters for classpath order, > however, it is not predictable in practice, and it might result in > downgrading dependencies. Imagine the project does not use Guice. However, > transitive dependencies might use Guice. At the same time, they might start > using Guice and stop using Guice, so the user can never tell which will be > the first project that uses Guice. Unfortunately, in Maven, the first project > that declares dependency wins, so it might easily be the case that the first > mention of Guice will reference outdated version that would be incompatible > with the newer one required in another dependency. > 3) Here's a real-life case: Maven downgrades protobuf-java dependency causing > something like NoSuchMethodError at the runtime. The step to reproduce is to > add dependency on dev.sigstore:sigstore-java:0.4.0. See [~hboutemy] analysis > in https://github.com/hboutemy/sigstore-maven-plugin/blob/import/analysis.md > Long story short, sigstore-java does not depend on protobuf-java directly, > however, sigstore-java depends on several third-party libraries that > eventually depend on protobuf-java. Maven's "the first wins" behaviour > results in incoherent set of protobuf dependencies on the classpath. > 4) see "unexpected" in MNG-5988 > To my best understanding, when it comes to transitive dependencies, both > "nearest first" and "declared first" are random variables which user can't > control unless they re-declare all the dependencies in their local pom. I > suggest Maven should not use random variables like "dependency depth" or > "dependency order" to drive conflict resolution. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (MNG-7852) Use all the versions for dependency resolution rather than "nearest first" or "declared first"
[ https://issues.apache.org/jira/browse/MNG-7852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748323#comment-17748323 ] Vladimir Sitnikov commented on MNG-7852: Well, one of the workable algorithms is described at [https://docs.gradle.org/current/userguide/dependency_resolution.html#sub:resolution-strategy] Of course, it would require rich versions (which would be useful in Maven anyway). So it would make sense if Maven could consume and produce Gradle Module Metadata to resolve and use rich version declarations and dependency constraints. With the new algorithm, many projects would just work since users often re-declare dependencies to avoid downgrades. Of course, the ones who intentionally performed downgrade would need some treatment, however, I am sure there is a possibility to make the transition smooth. For instance, Maven could resolve the deps with two algorithms, and issue a warning when the results diverge > Use all the versions for dependency resolution rather than "nearest first" or > "declared first" > -- > > Key: MNG-7852 > URL: https://issues.apache.org/jira/browse/MNG-7852 > Project: Maven > Issue Type: Improvement > Components: Dependencies >Reporter: Vladimir Sitnikov >Priority: Major > > Currently, Maven uses "nearest first", "declared first" rules for conflict > resolution: > https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html > I suggest those rules are removed since they produce hard to reason > resolutions for transitive dependencies. > Below I list reasons why both "nearest wins" and "declared first" yield > hard-to-predict behaviours, and they are likely to produce dependency > downgrades and the associated runtime errors. > Here are some examples: > 1) "Nearest first". Even though the rule sounds easy, it is not something the > users can control. For instance, if the project does not use Guava library, > some of the transitive dependencies could add a dependency on Guava. The user > has no control which dependency would be "the nearest" to declare Guava, so > user has literally no way to tell which Guava version will be used. > The only workaround I see for the users is to declare Guava dependency > explicitly even though the project does not need it directly. It sounds like > Maven requires users to re-declare all the possible dependencies, including > the runtime-only ones. > 2) "declared first". Of course, dependency order matters for classpath order, > however, it is not predictable in practice, and it might result in > downgrading dependencies. Imagine the project does not use Guice. However, > transitive dependencies might use Guice. At the same time, they might start > using Guice and stop using Guice, so the user can never tell which will be > the first project that uses Guice. Unfortunately, in Maven, the first project > that declares dependency wins, so it might easily be the case that the first > mention of Guice will reference outdated version that would be incompatible > with the newer one required in another dependency. > 3) Here's a real-life case: Maven downgrades protobuf-java dependency causing > something like NoSuchMethodError at the runtime. The step to reproduce is to > add dependency on dev.sigstore:sigstore-java:0.4.0. See [~hboutemy] analysis > in https://github.com/hboutemy/sigstore-maven-plugin/blob/import/analysis.md > Long story short, sigstore-java does not depend on protobuf-java directly, > however, sigstore-java depends on several third-party libraries that > eventually depend on protobuf-java. Maven's "the first wins" behaviour > results in incoherent set of protobuf dependencies on the classpath. > 4) see "unexpected" in MNG-5988 > To my best understanding, when it comes to transitive dependencies, both > "nearest first" and "declared first" are random variables which user can't > control unless they re-declare all the dependencies in their local pom. I > suggest Maven should not use random variables like "dependency depth" or > "dependency order" to drive conflict resolution. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (MNG-7852) Use all the versions for dependency resolution rather than "nearest first" or "declared first"
[ https://issues.apache.org/jira/browse/MNG-7852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748243#comment-17748243 ] Lenny Primak commented on MNG-7852: --- Hi, I understand your points, and they are indeed valid. I myself ran into these problems. However, your suggestion to "just remove" them is incomplete at best, as it would "break the world". To make this wish a reality (which I am not sure it's possible) I would try to develop an enhancement of the algorithm that works and doesn't break any current projects. As I said before, not sure that's possible. > Use all the versions for dependency resolution rather than "nearest first" or > "declared first" > -- > > Key: MNG-7852 > URL: https://issues.apache.org/jira/browse/MNG-7852 > Project: Maven > Issue Type: Improvement > Components: Dependencies >Reporter: Vladimir Sitnikov >Priority: Major > > Currently, Maven uses "nearest first", "declared first" rules for conflict > resolution: > https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html > I suggest those rules are removed since they produce hard to reason > resolutions for transitive dependencies. > Below I list reasons why both "nearest wins" and "declared first" yield > hard-to-predict behaviours, and they are likely to produce dependency > downgrades and the associated runtime errors. > Here are some examples: > 1) "Nearest first". Even though the rule sounds easy, it is not something the > users can control. For instance, if the project does not use Guava library, > some of the transitive dependencies could add a dependency on Guava. The user > has no control which dependency would be "the nearest" to declare Guava, so > user has literally no way to tell which Guava version will be used. > The only workaround I see for the users is to declare Guava dependency > explicitly even though the project does not need it directly. It sounds like > Maven requires users to re-declare all the possible dependencies, including > the runtime-only ones. > 2) "declared first". Of course, dependency order matters for classpath order, > however, it is not predictable in practice, and it might result in > downgrading dependencies. Imagine the project does not use Guice. However, > transitive dependencies might use Guice. At the same time, they might start > using Guice and stop using Guice, so the user can never tell which will be > the first project that uses Guice. Unfortunately, in Maven, the first project > that declares dependency wins, so it might easily be the case that the first > mention of Guice will reference outdated version that would be incompatible > with the newer one required in another dependency. > 3) Here's a real-life case: Maven downgrades protobuf-java dependency causing > something like NoSuchMethodError at the runtime. The step to reproduce is to > add dependency on dev.sigstore:sigstore-java:0.4.0. See [~hboutemy] analysis > in https://github.com/hboutemy/sigstore-maven-plugin/blob/import/analysis.md > Long story short, sigstore-java does not depend on protobuf-java directly, > however, sigstore-java depends on several third-party libraries that > eventually depend on protobuf-java. Maven's "the first wins" behaviour > results in incoherent set of protobuf dependencies on the classpath. > 4) see "unexpected" in MNG-5988 > To my best understanding, when it comes to transitive dependencies, both > "nearest first" and "declared first" are random variables which user can't > control unless they re-declare all the dependencies in their local pom. I > suggest Maven should not use random variables like "dependency depth" or > "dependency order" to drive conflict resolution. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (MNG-7852) Use all the versions for dependency resolution rather than "nearest first" or "declared first"
[ https://issues.apache.org/jira/browse/MNG-7852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748217#comment-17748217 ] Vladimir Sitnikov commented on MNG-7852: One more case (5): * I'm the author of my-lib * my-lib does not use guava, and it does not use guice * my-lib uses both lib-a and lib-b which are third-party libraries, and I have no direct control over them * lib-a uses guava 21 (==it depends on classes added in guava 21) * lib-a uses guice 5 (==it depends on classes added in guice 5, and it would fail with guice 3 on the classpath) * lib-b uses guava 23 (== it depends on a class that was added in guava 23, and it would fail with guava 21) * lib-b uses guice 3 (== it depends on a class that was added in guice 3) In that case, Maven is doomed to resolve an incompatible configuration. It will not fail the build, but it would build an artifact that is throws errors in the runtime (either guava or guice would resolve to an too old version). Once again, I understand that the workaround is to re-declare every possible version locally. However, well, what is the point of having "nearest first" strategy if it forces everybody to re-redeclare all the transitive versions? Of course, I understand, that re-declaration is not that needed when there are no conflicts. Unfortunately, I do not understand how Maven users figure out if they have conflicts or not. All these problems would just go away if Maven considered all versions instead of "the first mentioned" ones. > Use all the versions for dependency resolution rather than "nearest first" or > "declared first" > -- > > Key: MNG-7852 > URL: https://issues.apache.org/jira/browse/MNG-7852 > Project: Maven > Issue Type: Improvement > Components: Dependencies >Reporter: Vladimir Sitnikov >Priority: Major > > Currently, Maven uses "nearest first", "declared first" rules for conflict > resolution: > https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html > I suggest those rules are removed since they produce hard to reason > resolutions for transitive dependencies. > Below I list reasons why both "nearest wins" and "declared first" yield > hard-to-predict behaviours, and they are likely to produce dependency > downgrades and the associated runtime errors. > Here are some examples: > 1) "Nearest first". Even though the rule sounds easy, it is not something the > users can control. For instance, if the project does not use Guava library, > some of the transitive dependencies could add a dependency on Guava. The user > has no control which dependency would be "the nearest" to declare Guava, so > user has literally no way to tell which Guava version will be used. > The only workaround I see for the users is to declare Guava dependency > explicitly even though the project does not need it directly. It sounds like > Maven requires users to re-declare all the possible dependencies, including > the runtime-only ones. > 2) "declared first". Of course, dependency order matters for classpath order, > however, it is not predictable in practice, and it might result in > downgrading dependencies. Imagine the project does not use Guice. However, > transitive dependencies might use Guice. At the same time, they might start > using Guice and stop using Guice, so the user can never tell which will be > the first project that uses Guice. Unfortunately, in Maven, the first project > that declares dependency wins, so it might easily be the case that the first > mention of Guice will reference outdated version that would be incompatible > with the newer one required in another dependency. > 3) Here's a real-life case: Maven downgrades protobuf-java dependency causing > something like NoSuchMethodError at the runtime. The step to reproduce is to > add dependency on dev.sigstore:sigstore-java:0.4.0. See [~hboutemy] analysis > in https://github.com/hboutemy/sigstore-maven-plugin/blob/import/analysis.md > Long story short, sigstore-java does not depend on protobuf-java directly, > however, sigstore-java depends on several third-party libraries that > eventually depend on protobuf-java. Maven's "the first wins" behaviour > results in incoherent set of protobuf dependencies on the classpath. > 4) see "unexpected" in MNG-5988 > To my best understanding, when it comes to transitive dependencies, both > "nearest first" and "declared first" are random variables which user can't > control unless they re-declare all the dependencies in their local pom. I > suggest Maven should not use random variables like "dependency depth" or > "dependency order" to drive conflict resolution. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (MNG-7852) Use all the versions for dependency resolution rather than "nearest first" or "declared first"
[
https://issues.apache.org/jira/browse/MNG-7852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748173#comment-17748173
]
Vladimir Sitnikov commented on MNG-7852:
{quote}This "real life example" is totally along the lines of Maven specd
behaviour
{quote}
Tamas, many thanks for your comments, however, I still struggle to figure out
{*}{{*}}why{{*}}{*} Maven behaves like "nearest first".
The behaviour produces hard-to-reason failures, and it is unclear what could be
the benefits of such an algorithm except "dumb-easy to implement".
{quote}As you (I guess) implied, this can be solved by multiple means, ideally
by depMgmt. And yes, Maven makes you able to "not care", unlike Ant and others,
where you need to enlist (and control) all the versions, with Maven you leave
this to resolution. Or in other words, IF you care ("user has no control"),
then use proper means like depMgt to sort out your issues.
{quote}
I am afraid I do not understand the suggestion here.
Imagine
* I'm the author of my-lib
* my-lib does not use guava
* my-lib uses both lib-a and lib-b which are third-party libraries, and I have
no direct control over them
* lib-a uses guava 21 (==it depends on classes added in guava 21, and it would
work fine with guava 23)
* lib-b uses guava 23 (== it depends on a class that was added in guava 23
only, and it would fail with guava 21)
What is your exact suggestion?
Do you suggest that I (as an author of my-lib) should explicitly mention
"my-lib requires guava 23 on the runtime scope"? Of course, it would workaround
the issue, however, it is exactly the duplication I do not like. As an author
of my-lib, I do not really want to manually collect all the third-party
transitive dependencies and figure out the compatible versions for them. I
expect that dependency manager would do that.
At the same time, if I just add both dependencies on lib-a and lib-b, then
Maven would resolve a *{*}random{*}* guava version depending on which of lib-a
or lib-b was first on the list and/or which of them listed guava "closer to the
dependency root". Please, acknowledge that "dependency deepness" is random, and
I don't control which of lib-a or lib-b would put guava deeper.
In other words:
* If I omit guava in my-lib, then Maven resolves random version, and I get
crashes when Maven resolves guava 21 (e.g. lib-a was the first)
* If I list guava in my-lib, then I would have to list every possible
transitive dependency explicitly, and my list might easily get out of date.
That is not much different from using wget and putting jars on the classpath.
If Maven resolved all the versions no matter how deep they were declared, then
Maven could figure out the latest version, and it would avoid runtime failures.
{quote}Given this example is "oddly specific", I'd assume here some integration
happens (DI container is expected at runtime), so yes, here you should either
use "aligned libraries" (not always possible, especially if they are from
multiple vendors) or then it is up to you to sort out the conflicts between
them. Many many tools out there, ranging from Maven plugins to IDEs help you
doing this.
{quote}
I am afraid I do not understand your suggestion here :(
Imagine the case is:
* I'm the author of my-lib
* my-lib does not use guice
* my-lib uses both lib-a and lib-b which are third-party libraries, and I have
no direct control over them
* lib-a does not use guice
* lib-b uses guice 5
* Imagine my-lib declared dependencies in order of "lib-a, then lib-b",
because, well, it was just fine
So far, so good.
Then:
* lib-a started using guice 3
Maven would cause runtime failures, because it would resolve guice 3, while
lib-b required guice 5.
I do not understand why do you suggest resorting to third-party plugins and
IDEs even for such a trivial case like library with a couple of transitive
dependencies.
I understand that I can workaround the case by declaring guice version
explicitly, however, I do not understand why Maven requires duplicating
dependency information. If Maven discovered all the versions (guice 3 and guice
5), then it could resolve the conflicts and come up with guice 5 on the
classpath.
{quote} you mention "no such method" with protobuf-java, so I assumed Maven
"wildly mixed" versions (like major ones), but to me it seems you talk about
this issue{quote}
No. protobuf-java issue is not 11393, but it is like "Maven downgraded
protobuf-java dependency, so the app failed".
Here's the error message: https://youtu.be/2kooPqDguyw?t=7087
In other words: sigstore-java used protobuf compiler 3.22, however, Maven
resolved protobuf-java 3.21 for the runtime. It failed with the dreaded
emptyList error leaving maven users no clue on the way to fix it.
{quote}Best would be to move this discussion to ML instead, preferably the
users mailing list
[jira] [Commented] (MNG-7852) Use all the versions for dependency resolution rather than "nearest first" or "declared first"
[ https://issues.apache.org/jira/browse/MNG-7852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748113#comment-17748113 ] Tamas Cservenak commented on MNG-7852: -- For protobuf issue, here is a reproducer, a PR and example https://github.com/apache/maven/pull/1211 With this in place, you'd at least be warned that 2nd depMgt entry (for protobuf) is completely ignored as 1st depMgt import "deeply" had already imported it. > Use all the versions for dependency resolution rather than "nearest first" or > "declared first" > -- > > Key: MNG-7852 > URL: https://issues.apache.org/jira/browse/MNG-7852 > Project: Maven > Issue Type: Improvement > Components: Dependencies >Reporter: Vladimir Sitnikov >Priority: Major > > Currently, Maven uses "nearest first", "declared first" rules for conflict > resolution: > https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html > I suggest those rules are removed since they produce hard to reason > resolutions for transitive dependencies. > Below I list reasons why both "nearest wins" and "declared first" yield > hard-to-predict behaviours, and they are likely to produce dependency > downgrades and the associated runtime errors. > Here are some examples: > 1) "Nearest first". Even though the rule sounds easy, it is not something the > users can control. For instance, if the project does not use Guava library, > some of the transitive dependencies could add a dependency on Guava. The user > has no control which dependency would be "the nearest" to declare Guava, so > user has literally no way to tell which Guava version will be used. > The only workaround I see for the users is to declare Guava dependency > explicitly even though the project does not need it directly. It sounds like > Maven requires users to re-declare all the possible dependencies, including > the runtime-only ones. > 2) "declared first". Of course, dependency order matters for classpath order, > however, it is not predictable in practice, and it might result in > downgrading dependencies. Imagine the project does not use Guice. However, > transitive dependencies might use Guice. At the same time, they might start > using Guice and stop using Guice, so the user can never tell which will be > the first project that uses Guice. Unfortunately, in Maven, the first project > that declares dependency wins, so it might easily be the case that the first > mention of Guice will reference outdated version that would be incompatible > with the newer one required in another dependency. > 3) Here's a real-life case: Maven downgrades protobuf-java dependency causing > something like NoSuchMethodError at the runtime. The step to reproduce is to > add dependency on dev.sigstore:sigstore-java:0.4.0. See [~hboutemy] analysis > in https://github.com/hboutemy/sigstore-maven-plugin/blob/import/analysis.md > Long story short, sigstore-java does not depend on protobuf-java directly, > however, sigstore-java depends on several third-party libraries that > eventually depend on protobuf-java. Maven's "the first wins" behaviour > results in incoherent set of protobuf dependencies on the classpath. > 4) see "unexpected" in MNG-5988 > To my best understanding, when it comes to transitive dependencies, both > "nearest first" and "declared first" are random variables which user can't > control unless they re-declare all the dependencies in their local pom. I > suggest Maven should not use random variables like "dependency depth" or > "dependency order" to drive conflict resolution. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (MNG-7852) Use all the versions for dependency resolution rather than "nearest first" or "declared first"
[ https://issues.apache.org/jira/browse/MNG-7852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748094#comment-17748094 ] Tamas Cservenak commented on MNG-7852: -- And just FTR: you mention "no such method" with protobuf-java, so I assumed Maven "wildly mixed" versions (like major ones), but to me it sees you talk about this issue https://github.com/protocolbuffers/protobuf/issues/11393 that was simply a "bad release" happening between patch versions. > Use all the versions for dependency resolution rather than "nearest first" or > "declared first" > -- > > Key: MNG-7852 > URL: https://issues.apache.org/jira/browse/MNG-7852 > Project: Maven > Issue Type: Improvement > Components: Dependencies >Reporter: Vladimir Sitnikov >Priority: Major > > Currently, Maven uses "nearest first", "declared first" rules for conflict > resolution: > https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html > I suggest those rules are removed since they produce hard to reason > resolutions for transitive dependencies. > Below I list reasons why both "nearest wins" and "declared first" yield > hard-to-predict behaviours, and they are likely to produce dependency > downgrades and the associated runtime errors. > Here are some examples: > 1) "Nearest first". Even though the rule sounds easy, it is not something the > users can control. For instance, if the project does not use Guava library, > some of the transitive dependencies could add a dependency on Guava. The user > has no control which dependency would be "the nearest" to declare Guava, so > user has literally no way to tell which Guava version will be used. > The only workaround I see for the users is to declare Guava dependency > explicitly even though the project does not need it directly. It sounds like > Maven requires users to re-declare all the possible dependencies, including > the runtime-only ones. > 2) "declared first". Of course, dependency order matters for classpath order, > however, it is not predictable in practice, and it might result in > downgrading dependencies. Imagine the project does not use Guice. However, > transitive dependencies might use Guice. At the same time, they might start > using Guice and stop using Guice, so the user can never tell which will be > the first project that uses Guice. Unfortunately, in Maven, the first project > that declares dependency wins, so it might easily be the case that the first > mention of Guice will reference outdated version that would be incompatible > with the newer one required in another dependency. > 3) Here's a real-life case: Maven downgrades protobuf-java dependency causing > something like NoSuchMethodError at the runtime. The step to reproduce is to > add dependency on dev.sigstore:sigstore-java:0.4.0. See [~hboutemy] analysis > in https://github.com/hboutemy/sigstore-maven-plugin/blob/import/analysis.md > Long story short, sigstore-java does not depend on protobuf-java directly, > however, sigstore-java depends on several third-party libraries that > eventually depend on protobuf-java. Maven's "the first wins" behaviour > results in incoherent set of protobuf dependencies on the classpath. > 4) see "unexpected" in MNG-5988 > To my best understanding, when it comes to transitive dependencies, both > "nearest first" and "declared first" are random variables which user can't > control unless they re-declare all the dependencies in their local pom. I > suggest Maven should not use random variables like "dependency depth" or > "dependency order" to drive conflict resolution. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (MNG-7852) Use all the versions for dependency resolution rather than "nearest first" or "declared first"
[
https://issues.apache.org/jira/browse/MNG-7852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748087#comment-17748087
]
Tamas Cservenak commented on MNG-7852:
--
Maven provides solution to all your enlisted problems, but I still have hard
time to grasp them...
1. you state "project does not use guava", then "but transitive may add guava"
and then "user has no control which guava". As you (I guess) implied, this can
be solved by multiple means, ideally by depMgmt. And yes, Maven makes you able
to "not care", unlike Ant and others, where you need to enlist (and control)
all the versions, with Maven you leave this to resolution. Or in other words,
IF you care ("user has no control"), then use proper means like depMgt to sort
out your issues.
2. Given this example is "oddly specific", I'd assume here some integration
happens (DI container is expected at runtime), so yes, here you should either
use "aligned libraries" (not always possible, especially if they are from
multiple vendors) or then it is up to you to sort out the conflicts between
them. Many many tools out there, ranging from Maven plugins to IDEs help you
doing this.
3. This "real life example" is totally along the lines of Maven specd
behaviour: the (deep) and over use of import scope, and the fact you had import
for protobuf, but you simply was not aware that import above already imported
it. Also, you should be aware that import "flattens" things, hence, if there
would be no "overuse" of BOMs (meaning deep hierarchy), but let's assume you
manually managed depMgt without BOM imports, you'd be immediately aware that
you have TWO conflicting protobuf declarations in depMgt.
Best would be to move this discussion to ML instead, preferably the users
mailing list https://maven.apache.org/mailing-lists.html
> Use all the versions for dependency resolution rather than "nearest first" or
> "declared first"
> --
>
> Key: MNG-7852
> URL: https://issues.apache.org/jira/browse/MNG-7852
> Project: Maven
> Issue Type: Improvement
> Components: Dependencies
>Reporter: Vladimir Sitnikov
>Priority: Major
>
> Currently, Maven uses "nearest first", "declared first" rules for conflict
> resolution:
> https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html
> I suggest those rules are removed since they produce hard to reason
> resolutions for transitive dependencies.
> Below I list reasons why both "nearest wins" and "declared first" yield
> hard-to-predict behaviours, and they are likely to produce dependency
> downgrades and the associated runtime errors.
> Here are some examples:
> 1) "Nearest first". Even though the rule sounds easy, it is not something the
> users can control. For instance, if the project does not use Guava library,
> some of the transitive dependencies could add a dependency on Guava. The user
> has no control which dependency would be "the nearest" to declare Guava, so
> user has literally no way to tell which Guava version will be used.
> The only workaround I see for the users is to declare Guava dependency
> explicitly even though the project does not need it directly. It sounds like
> Maven requires users to re-declare all the possible dependencies, including
> the runtime-only ones.
> 2) "declared first". Of course, dependency order matters for classpath order,
> however, it is not predictable in practice, and it might result in
> downgrading dependencies. Imagine the project does not use Guice. However,
> transitive dependencies might use Guice. At the same time, they might start
> using Guice and stop using Guice, so the user can never tell which will be
> the first project that uses Guice. Unfortunately, in Maven, the first project
> that declares dependency wins, so it might easily be the case that the first
> mention of Guice will reference outdated version that would be incompatible
> with the newer one required in another dependency.
> 3) Here's a real-life case: Maven downgrades protobuf-java dependency causing
> something like NoSuchMethodError at the runtime. The step to reproduce is to
> add dependency on dev.sigstore:sigstore-java:0.4.0. See [~hboutemy] analysis
> in https://github.com/hboutemy/sigstore-maven-plugin/blob/import/analysis.md
> Long story short, sigstore-java does not depend on protobuf-java directly,
> however, sigstore-java depends on several third-party libraries that
> eventually depend on protobuf-java. Maven's "the first wins" behaviour
> results in incoherent set of protobuf dependencies on the classpath.
> 4) see "unexpected" in MNG-5988
> To my best understanding, when it comes to transitive dependencies, both
> "nearest first" and "declared first" are random variables which
[jira] [Commented] (MNG-7852) Use all the versions for dependency resolution rather than "nearest first" or "declared first"
[
https://issues.apache.org/jira/browse/MNG-7852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747891#comment-17747891
]
Vladimir Sitnikov commented on MNG-7852:
{quote}Am really unsure what you want to see as solution here
{quote}
It would really help me if you could suggest how Maven solves the issues in the
description: 1, 2, 3.
The only workaround I can imagine for now is to re-declare all the dependencies
manually, including the runtime dependencies of all the transitives. Of course,
if dependencies are re-declared, they might get out of date, so it doesn't
sound like a reasonable solution to me.
I suggest that Maven expand the complete dependency (regular dependency, and
dependencyManagement) tree, and then perform conflict resolution. When dealing
with conflict resolution, "newest wins" makes more sense than "oldest wins".
There's an option of "fail on conflict", however, it is not that practical.
Of course, it would be nice if Maven supported rich version constraints like
"requires", "prefers", "rejects", "strictly", however, it is a different story.
> Use all the versions for dependency resolution rather than "nearest first" or
> "declared first"
> --
>
> Key: MNG-7852
> URL: https://issues.apache.org/jira/browse/MNG-7852
> Project: Maven
> Issue Type: Improvement
> Components: Dependencies
>Reporter: Vladimir Sitnikov
>Priority: Major
>
> Currently, Maven uses "nearest first", "declared first" rules for conflict
> resolution:
> https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html
> I suggest those rules are removed since they produce hard to reason
> resolutions for transitive dependencies.
> Below I list reasons why both "nearest wins" and "declared first" yield
> hard-to-predict behaviours, and they are likely to produce dependency
> downgrades and the associated runtime errors.
> Here are some examples:
> 1) "Nearest first". Even though the rule sounds easy, it is not something the
> users can control. For instance, if the project does not use Guava library,
> some of the transitive dependencies could add a dependency on Guava. The user
> has no control which dependency would be "the nearest" to declare Guava, so
> user has literally no way to tell which Guava version will be used.
> The only workaround I see for the users is to declare Guava dependency
> explicitly even though the project does not need it directly. It sounds like
> Maven requires users to re-declare all the possible dependencies, including
> the runtime-only ones.
> 2) "declared first". Of course, dependency order matters for classpath order,
> however, it is not predictable in practice, and it might result in
> downgrading dependencies. Imagine the project does not use Guice. However,
> transitive dependencies might use Guice. At the same time, they might start
> using Guice and stop using Guice, so the user can never tell which will be
> the first project that uses Guice. Unfortunately, in Maven, the first project
> that declares dependency wins, so it might easily be the case that the first
> mention of Guice will reference outdated version that would be incompatible
> with the newer one required in another dependency.
> 3) Here's a real-life case: Maven downgrades protobuf-java dependency causing
> something like NoSuchMethodError at the runtime. The step to reproduce is to
> add dependency on dev.sigstore:sigstore-java:0.4.0. See [~hboutemy] analysis
> in https://github.com/hboutemy/sigstore-maven-plugin/blob/import/analysis.md
> Long story short, sigstore-java does not depend on protobuf-java directly,
> however, sigstore-java depends on several third-party libraries that
> eventually depend on protobuf-java. Maven's "the first wins" behaviour
> results in incoherent set of protobuf dependencies on the classpath.
> 4) see "unexpected" in MNG-5988
> To my best understanding, when it comes to transitive dependencies, both
> "nearest first" and "declared first" are random variables which user can't
> control unless they re-declare all the dependencies in their local pom. I
> suggest Maven should not use random variables like "dependency depth" or
> "dependency order" to drive conflict resolution.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MNG-7852) Use all the versions for dependency resolution rather than "nearest first" or "declared first"
[
https://issues.apache.org/jira/browse/MNG-7852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747888#comment-17747888
]
Vladimir Sitnikov commented on MNG-7852:
{quote}all the rest of issues were closed as "works as expected".{quote}
Correct me if I am wrong, however, the only justification I saw was "works as
expected since Maven uses nearest first algorithm".
I suggest reconsidering "nearest first" as it is not really manageable from the
end-user point of view.
{quote} Maven never worked (unlike as npm) by "highest version" nor will.{quote}
Can you clarify why "highest wins" is not an option for Maven?
> Use all the versions for dependency resolution rather than "nearest first" or
> "declared first"
> --
>
> Key: MNG-7852
> URL: https://issues.apache.org/jira/browse/MNG-7852
> Project: Maven
> Issue Type: Improvement
> Components: Dependencies
>Reporter: Vladimir Sitnikov
>Priority: Major
>
> Currently, Maven uses "nearest first", "declared first" rules for conflict
> resolution:
> https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html
> I suggest those rules are removed since they produce hard to reason
> resolutions for transitive dependencies.
> Below I list reasons why both "nearest wins" and "declared first" yield
> hard-to-predict behaviours, and they are likely to produce dependency
> downgrades and the associated runtime errors.
> Here are some examples:
> 1) "Nearest first". Even though the rule sounds easy, it is not something the
> users can control. For instance, if the project does not use Guava library,
> some of the transitive dependencies could add a dependency on Guava. The user
> has no control which dependency would be "the nearest" to declare Guava, so
> user has literally no way to tell which Guava version will be used.
> The only workaround I see for the users is to declare Guava dependency
> explicitly even though the project does not need it directly. It sounds like
> Maven requires users to re-declare all the possible dependencies, including
> the runtime-only ones.
> 2) "declared first". Of course, dependency order matters for classpath order,
> however, it is not predictable in practice, and it might result in
> downgrading dependencies. Imagine the project does not use Guice. However,
> transitive dependencies might use Guice. At the same time, they might start
> using Guice and stop using Guice, so the user can never tell which will be
> the first project that uses Guice. Unfortunately, in Maven, the first project
> that declares dependency wins, so it might easily be the case that the first
> mention of Guice will reference outdated version that would be incompatible
> with the newer one required in another dependency.
> 3) Here's a real-life case: Maven downgrades protobuf-java dependency causing
> something like NoSuchMethodError at the runtime. The step to reproduce is to
> add dependency on dev.sigstore:sigstore-java:0.4.0. See [~hboutemy] analysis
> in https://github.com/hboutemy/sigstore-maven-plugin/blob/import/analysis.md
> Long story short, sigstore-java does not depend on protobuf-java directly,
> however, sigstore-java depends on several third-party libraries that
> eventually depend on protobuf-java. Maven's "the first wins" behaviour
> results in incoherent set of protobuf dependencies on the classpath.
> 4) see "unexpected" in MNG-5988
> To my best understanding, when it comes to transitive dependencies, both
> "nearest first" and "declared first" are random variables which user can't
> control unless they re-declare all the dependencies in their local pom. I
> suggest Maven should not use random variables like "dependency depth" or
> "dependency order" to drive conflict resolution.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MNG-7852) Use all the versions for dependency resolution rather than "nearest first" or "declared first"
[ https://issues.apache.org/jira/browse/MNG-7852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747883#comment-17747883 ] Tamas Cservenak commented on MNG-7852: -- Aside of the "unexpected" (which is known issue, the runtime vs test scope), all the rest of issues were closed as "works as expected". Am really unsure what you want to see as solution here. Maven never worked (unlike as npm) by "highest version" nor will. It uses "dirty tree" (graph) to navigate, and uses properties like "distance" (from root) and "order" to make decisions, seems you argue against both. What would you propose instead? > Use all the versions for dependency resolution rather than "nearest first" or > "declared first" > -- > > Key: MNG-7852 > URL: https://issues.apache.org/jira/browse/MNG-7852 > Project: Maven > Issue Type: Improvement > Components: Dependencies >Reporter: Vladimir Sitnikov >Priority: Major > > Currently, Maven uses "nearest first", "declared first" rules for conflict > resolution: > https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html > I suggest those rules are removed since they produce hard to reason > resolutions for transitive dependencies. > Below I list reasons why both "nearest wins" and "declared first" yield > hard-to-predict behaviours, and they are likely to produce dependency > downgrades and the associated runtime errors. > Here are some examples: > 1) "Nearest first". Even though the rule sounds easy, it is not something the > users can control. For instance, if the project does not use Guava library, > some of the transitive dependencies could add a dependency on Guava. The user > has no control which dependency would be "the nearest" to declare Guava, so > user has literally no way to tell which Guava version will be used. > The only workaround I see for the users is to declare Guava dependency > explicitly even though the project does not need it directly. It sounds like > Maven requires users to re-declare all the possible dependencies, including > the runtime-only ones. > 2) "declared first". Of course, dependency order matters for classpath order, > however, it is not predictable in practice, and it might result in > downgrading dependencies. Imagine the project does not use Guice. However, > transitive dependencies might use Guice. At the same time, they might start > using Guice and stop using Guice, so the user can never tell which will be > the first project that uses Guice. Unfortunately, in Maven, the first project > that declares dependency wins, so it might easily be the case that the first > mention of Guice will reference outdated version that would be incompatible > with the newer one required in another dependency. > 3) Here's a real-life case: Maven downgrades protobuf-java dependency causing > something like NoSuchMethodError at the runtime. The step to reproduce is to > add dependency on dev.sigstore:sigstore-java:0.4.0. See [~hboutemy] analysis > in https://github.com/hboutemy/sigstore-maven-plugin/blob/import/analysis.md > Long story short, sigstore-java does not depend on protobuf-java directly, > however, sigstore-java depends on several third-party libraries that > eventually depend on protobuf-java. Maven's "the first wins" behaviour > results in incoherent set of protobuf dependencies on the classpath. > 4) see "unexpected" in MNG-5988 > To my best understanding, when it comes to transitive dependencies, both > "nearest first" and "declared first" are random variables which user can't > control unless they re-declare all the dependencies in their local pom. I > suggest Maven should not use random variables like "dependency depth" or > "dependency order" to drive conflict resolution. -- This message was sent by Atlassian Jira (v8.20.10#820010)
