[jira] [Comment Edited] (MNG-7852) Use all the versions for dependency resolution rather than "nearest first" or "declared first"
[
https://issues.apache.org/jira/browse/MNG-7852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17749139#comment-17749139
]
Tamas Cservenak edited comment on MNG-7852 at 7/31/23 10:50 AM:
You are very wrong by putting statements like these, I'd really say this is
FUD. Maven chooses "nearest" version and is predictable and even reproducible.
And while blaming something else instead of yourself, due lack of knowledge
about it, is really in human nature, don't fall for it.
Also, you mix runtime and compile time things ("nobody ensure that people can
compile code"). Once you mention guava as runtime dependency, then you draw
conclusion about code compilation. You should do your homework: if you compile
against something, best practice is to declare it as direct dependency (and not
rely on some 3rd level transitive dependency). Tools like Takari Lifecycle even
ensure this: if you even touch a class that comes from any other dependency
than first level ones (those in POM), your build will fail.
So please, stop bombing JIRA, as am still by my original statement: "Best would
be to move this discussion to ML instead, preferably the users mailing list
https://maven.apache.org/mailing-lists.html;
was (Author: cstamas):
You are very wrong by putting statements like these, I'd really say this is
FUD. Maven chooses "nearest" version and is predictable and even reproducible.
And while blaming something else instead of yourself, due lack of knowledge
about it, is really in human nature, don't fall for it.
Also, you mix runtime and compile time things ("nobody ensure that people can
compile code"). Once you mention guava as runtime dependency, then you draw
conclusion about code compilation. You should do your homework: if you compile
against something, best practice is to declare it as direct dependency (and not
rely on some 3rd level transitive dependency). Tools like Takari Lifecycle even
ensure this: if you even touch a class that comes from any other dependency
that first level ones (those in POM), your build will fail.
So please, stop bombing JIRA, as am still by my original statement: "Best would
be to move this discussion to ML instead, preferably the users mailing list
https://maven.apache.org/mailing-lists.html;
> Use all the versions for dependency resolution rather than "nearest first" or
> "declared first"
> --
>
> Key: MNG-7852
> URL: https://issues.apache.org/jira/browse/MNG-7852
> Project: Maven
> Issue Type: Improvement
> Components: Dependencies
>Reporter: Vladimir Sitnikov
>Priority: Major
>
> Currently, Maven uses "nearest first", "declared first" rules for conflict
> resolution:
> https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html
> I suggest those rules are removed since they produce hard to reason
> resolutions for transitive dependencies.
> Below I list reasons why both "nearest wins" and "declared first" yield
> hard-to-predict behaviours, and they are likely to produce dependency
> downgrades and the associated runtime errors.
> Here are some examples:
> 1) "Nearest first". Even though the rule sounds easy, it is not something the
> users can control. For instance, if the project does not use Guava library,
> some of the transitive dependencies could add a dependency on Guava. The user
> has no control which dependency would be "the nearest" to declare Guava, so
> user has literally no way to tell which Guava version will be used.
> The only workaround I see for the users is to declare Guava dependency
> explicitly even though the project does not need it directly. It sounds like
> Maven requires users to re-declare all the possible dependencies, including
> the runtime-only ones.
> 2) "declared first". Of course, dependency order matters for classpath order,
> however, it is not predictable in practice, and it might result in
> downgrading dependencies. Imagine the project does not use Guice. However,
> transitive dependencies might use Guice. At the same time, they might start
> using Guice and stop using Guice, so the user can never tell which will be
> the first project that uses Guice. Unfortunately, in Maven, the first project
> that declares dependency wins, so it might easily be the case that the first
> mention of Guice will reference outdated version that would be incompatible
> with the newer one required in another dependency.
> 3) Here's a real-life case: Maven downgrades protobuf-java dependency causing
> something like NoSuchMethodError at the runtime. The step to reproduce is to
> add dependency on dev.sigstore:sigstore-java:0.4.0. See [~hboutemy] analysis
> in
[jira] [Comment Edited] (MNG-7852) Use all the versions for dependency resolution rather than "nearest first" or "declared first"
[ https://issues.apache.org/jira/browse/MNG-7852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748217#comment-17748217 ] Vladimir Sitnikov edited comment on MNG-7852 at 7/27/23 4:02 PM: - One more case (5): * I'm the author of my-lib * my-lib does not use guava, and it does not use guice * my-lib uses both lib-a and lib-b which are third-party libraries, and I have no direct control over them * lib-a uses guava 21 (==it depends on classes added in guava 21) * lib-a uses guice 5 (==it depends on classes added in guice 5, and it would fail with guice 3 on the classpath) * lib-b uses guava 23 (== it depends on a class that was added in guava 23, and it would fail with guava 21) * lib-b uses guice 3 (== it depends on a class that was added in guice 3) In that case, Maven is doomed to resolve an incompatible configuration. It will not fail the build, but it would build an artifact that throws errors at the runtime (either guava or guice would resolve to an too old version). Once again, I understand that the workaround is to re-declare every possible version locally. However, well, what is the point of having "nearest first" strategy if it forces everybody to re-redeclare all the transitive versions? Of course, I understand, that re-declaration is not that needed when there are no conflicts. Unfortunately, I do not understand how Maven users figure out if they have conflicts or not. All these problems would just go away if Maven considered all versions instead of "the first mentioned" ones. was (Author: vladimirsitnikov): One more case (5): * I'm the author of my-lib * my-lib does not use guava, and it does not use guice * my-lib uses both lib-a and lib-b which are third-party libraries, and I have no direct control over them * lib-a uses guava 21 (==it depends on classes added in guava 21) * lib-a uses guice 5 (==it depends on classes added in guice 5, and it would fail with guice 3 on the classpath) * lib-b uses guava 23 (== it depends on a class that was added in guava 23, and it would fail with guava 21) * lib-b uses guice 3 (== it depends on a class that was added in guice 3) In that case, Maven is doomed to resolve an incompatible configuration. It will not fail the build, but it would build an artifact that is throws errors at the runtime (either guava or guice would resolve to an too old version). Once again, I understand that the workaround is to re-declare every possible version locally. However, well, what is the point of having "nearest first" strategy if it forces everybody to re-redeclare all the transitive versions? Of course, I understand, that re-declaration is not that needed when there are no conflicts. Unfortunately, I do not understand how Maven users figure out if they have conflicts or not. All these problems would just go away if Maven considered all versions instead of "the first mentioned" ones. > Use all the versions for dependency resolution rather than "nearest first" or > "declared first" > -- > > Key: MNG-7852 > URL: https://issues.apache.org/jira/browse/MNG-7852 > Project: Maven > Issue Type: Improvement > Components: Dependencies >Reporter: Vladimir Sitnikov >Priority: Major > > Currently, Maven uses "nearest first", "declared first" rules for conflict > resolution: > https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html > I suggest those rules are removed since they produce hard to reason > resolutions for transitive dependencies. > Below I list reasons why both "nearest wins" and "declared first" yield > hard-to-predict behaviours, and they are likely to produce dependency > downgrades and the associated runtime errors. > Here are some examples: > 1) "Nearest first". Even though the rule sounds easy, it is not something the > users can control. For instance, if the project does not use Guava library, > some of the transitive dependencies could add a dependency on Guava. The user > has no control which dependency would be "the nearest" to declare Guava, so > user has literally no way to tell which Guava version will be used. > The only workaround I see for the users is to declare Guava dependency > explicitly even though the project does not need it directly. It sounds like > Maven requires users to re-declare all the possible dependencies, including > the runtime-only ones. > 2) "declared first". Of course, dependency order matters for classpath order, > however, it is not predictable in practice, and it might result in > downgrading dependencies. Imagine the project does not use Guice. However, > transitive dependencies might use Guice. At the same time, they might start > using Guice and stop using Guice, so
[jira] [Comment Edited] (MNG-7852) Use all the versions for dependency resolution rather than "nearest first" or "declared first"
[ https://issues.apache.org/jira/browse/MNG-7852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748217#comment-17748217 ] Vladimir Sitnikov edited comment on MNG-7852 at 7/27/23 3:46 PM: - One more case (5): * I'm the author of my-lib * my-lib does not use guava, and it does not use guice * my-lib uses both lib-a and lib-b which are third-party libraries, and I have no direct control over them * lib-a uses guava 21 (==it depends on classes added in guava 21) * lib-a uses guice 5 (==it depends on classes added in guice 5, and it would fail with guice 3 on the classpath) * lib-b uses guava 23 (== it depends on a class that was added in guava 23, and it would fail with guava 21) * lib-b uses guice 3 (== it depends on a class that was added in guice 3) In that case, Maven is doomed to resolve an incompatible configuration. It will not fail the build, but it would build an artifact that is throws errors at the runtime (either guava or guice would resolve to an too old version). Once again, I understand that the workaround is to re-declare every possible version locally. However, well, what is the point of having "nearest first" strategy if it forces everybody to re-redeclare all the transitive versions? Of course, I understand, that re-declaration is not that needed when there are no conflicts. Unfortunately, I do not understand how Maven users figure out if they have conflicts or not. All these problems would just go away if Maven considered all versions instead of "the first mentioned" ones. was (Author: vladimirsitnikov): One more case (5): * I'm the author of my-lib * my-lib does not use guava, and it does not use guice * my-lib uses both lib-a and lib-b which are third-party libraries, and I have no direct control over them * lib-a uses guava 21 (==it depends on classes added in guava 21) * lib-a uses guice 5 (==it depends on classes added in guice 5, and it would fail with guice 3 on the classpath) * lib-b uses guava 23 (== it depends on a class that was added in guava 23, and it would fail with guava 21) * lib-b uses guice 3 (== it depends on a class that was added in guice 3) In that case, Maven is doomed to resolve an incompatible configuration. It will not fail the build, but it would build an artifact that is throws errors in the runtime (either guava or guice would resolve to an too old version). Once again, I understand that the workaround is to re-declare every possible version locally. However, well, what is the point of having "nearest first" strategy if it forces everybody to re-redeclare all the transitive versions? Of course, I understand, that re-declaration is not that needed when there are no conflicts. Unfortunately, I do not understand how Maven users figure out if they have conflicts or not. All these problems would just go away if Maven considered all versions instead of "the first mentioned" ones. > Use all the versions for dependency resolution rather than "nearest first" or > "declared first" > -- > > Key: MNG-7852 > URL: https://issues.apache.org/jira/browse/MNG-7852 > Project: Maven > Issue Type: Improvement > Components: Dependencies >Reporter: Vladimir Sitnikov >Priority: Major > > Currently, Maven uses "nearest first", "declared first" rules for conflict > resolution: > https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html > I suggest those rules are removed since they produce hard to reason > resolutions for transitive dependencies. > Below I list reasons why both "nearest wins" and "declared first" yield > hard-to-predict behaviours, and they are likely to produce dependency > downgrades and the associated runtime errors. > Here are some examples: > 1) "Nearest first". Even though the rule sounds easy, it is not something the > users can control. For instance, if the project does not use Guava library, > some of the transitive dependencies could add a dependency on Guava. The user > has no control which dependency would be "the nearest" to declare Guava, so > user has literally no way to tell which Guava version will be used. > The only workaround I see for the users is to declare Guava dependency > explicitly even though the project does not need it directly. It sounds like > Maven requires users to re-declare all the possible dependencies, including > the runtime-only ones. > 2) "declared first". Of course, dependency order matters for classpath order, > however, it is not predictable in practice, and it might result in > downgrading dependencies. Imagine the project does not use Guice. However, > transitive dependencies might use Guice. At the same time, they might start > using Guice and stop using Guice,
[jira] [Comment Edited] (MNG-7852) Use all the versions for dependency resolution rather than "nearest first" or "declared first"
[ https://issues.apache.org/jira/browse/MNG-7852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748094#comment-17748094 ] Tamas Cservenak edited comment on MNG-7852 at 7/27/23 11:19 AM: And just FTR: you mention "no such method" with protobuf-java, so I assumed Maven "wildly mixed" versions (like major ones), but to me it seems you talk about this issue https://github.com/protocolbuffers/protobuf/issues/11393 that was simply a "broken release(s)" happening between patch versions when library used on Java 8. was (Author: cstamas): And just FTR: you mention "no such method" with protobuf-java, so I assumed Maven "wildly mixed" versions (like major ones), but to me it seems you talk about this issue https://github.com/protocolbuffers/protobuf/issues/11393 that was simply a "bad release" happening between patch versions. > Use all the versions for dependency resolution rather than "nearest first" or > "declared first" > -- > > Key: MNG-7852 > URL: https://issues.apache.org/jira/browse/MNG-7852 > Project: Maven > Issue Type: Improvement > Components: Dependencies >Reporter: Vladimir Sitnikov >Priority: Major > > Currently, Maven uses "nearest first", "declared first" rules for conflict > resolution: > https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html > I suggest those rules are removed since they produce hard to reason > resolutions for transitive dependencies. > Below I list reasons why both "nearest wins" and "declared first" yield > hard-to-predict behaviours, and they are likely to produce dependency > downgrades and the associated runtime errors. > Here are some examples: > 1) "Nearest first". Even though the rule sounds easy, it is not something the > users can control. For instance, if the project does not use Guava library, > some of the transitive dependencies could add a dependency on Guava. The user > has no control which dependency would be "the nearest" to declare Guava, so > user has literally no way to tell which Guava version will be used. > The only workaround I see for the users is to declare Guava dependency > explicitly even though the project does not need it directly. It sounds like > Maven requires users to re-declare all the possible dependencies, including > the runtime-only ones. > 2) "declared first". Of course, dependency order matters for classpath order, > however, it is not predictable in practice, and it might result in > downgrading dependencies. Imagine the project does not use Guice. However, > transitive dependencies might use Guice. At the same time, they might start > using Guice and stop using Guice, so the user can never tell which will be > the first project that uses Guice. Unfortunately, in Maven, the first project > that declares dependency wins, so it might easily be the case that the first > mention of Guice will reference outdated version that would be incompatible > with the newer one required in another dependency. > 3) Here's a real-life case: Maven downgrades protobuf-java dependency causing > something like NoSuchMethodError at the runtime. The step to reproduce is to > add dependency on dev.sigstore:sigstore-java:0.4.0. See [~hboutemy] analysis > in https://github.com/hboutemy/sigstore-maven-plugin/blob/import/analysis.md > Long story short, sigstore-java does not depend on protobuf-java directly, > however, sigstore-java depends on several third-party libraries that > eventually depend on protobuf-java. Maven's "the first wins" behaviour > results in incoherent set of protobuf dependencies on the classpath. > 4) see "unexpected" in MNG-5988 > To my best understanding, when it comes to transitive dependencies, both > "nearest first" and "declared first" are random variables which user can't > control unless they re-declare all the dependencies in their local pom. I > suggest Maven should not use random variables like "dependency depth" or > "dependency order" to drive conflict resolution. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (MNG-7852) Use all the versions for dependency resolution rather than "nearest first" or "declared first"
[ https://issues.apache.org/jira/browse/MNG-7852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748094#comment-17748094 ] Tamas Cservenak edited comment on MNG-7852 at 7/27/23 11:19 AM: And just FTR: you mention "no such method" with protobuf-java, so I assumed Maven "wildly mixed" versions (like major ones), but to me it seems you talk about this issue https://github.com/protocolbuffers/protobuf/issues/11393 that was simply a "bad release" happening between patch versions. was (Author: cstamas): And just FTR: you mention "no such method" with protobuf-java, so I assumed Maven "wildly mixed" versions (like major ones), but to me it sees you talk about this issue https://github.com/protocolbuffers/protobuf/issues/11393 that was simply a "bad release" happening between patch versions. > Use all the versions for dependency resolution rather than "nearest first" or > "declared first" > -- > > Key: MNG-7852 > URL: https://issues.apache.org/jira/browse/MNG-7852 > Project: Maven > Issue Type: Improvement > Components: Dependencies >Reporter: Vladimir Sitnikov >Priority: Major > > Currently, Maven uses "nearest first", "declared first" rules for conflict > resolution: > https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html > I suggest those rules are removed since they produce hard to reason > resolutions for transitive dependencies. > Below I list reasons why both "nearest wins" and "declared first" yield > hard-to-predict behaviours, and they are likely to produce dependency > downgrades and the associated runtime errors. > Here are some examples: > 1) "Nearest first". Even though the rule sounds easy, it is not something the > users can control. For instance, if the project does not use Guava library, > some of the transitive dependencies could add a dependency on Guava. The user > has no control which dependency would be "the nearest" to declare Guava, so > user has literally no way to tell which Guava version will be used. > The only workaround I see for the users is to declare Guava dependency > explicitly even though the project does not need it directly. It sounds like > Maven requires users to re-declare all the possible dependencies, including > the runtime-only ones. > 2) "declared first". Of course, dependency order matters for classpath order, > however, it is not predictable in practice, and it might result in > downgrading dependencies. Imagine the project does not use Guice. However, > transitive dependencies might use Guice. At the same time, they might start > using Guice and stop using Guice, so the user can never tell which will be > the first project that uses Guice. Unfortunately, in Maven, the first project > that declares dependency wins, so it might easily be the case that the first > mention of Guice will reference outdated version that would be incompatible > with the newer one required in another dependency. > 3) Here's a real-life case: Maven downgrades protobuf-java dependency causing > something like NoSuchMethodError at the runtime. The step to reproduce is to > add dependency on dev.sigstore:sigstore-java:0.4.0. See [~hboutemy] analysis > in https://github.com/hboutemy/sigstore-maven-plugin/blob/import/analysis.md > Long story short, sigstore-java does not depend on protobuf-java directly, > however, sigstore-java depends on several third-party libraries that > eventually depend on protobuf-java. Maven's "the first wins" behaviour > results in incoherent set of protobuf dependencies on the classpath. > 4) see "unexpected" in MNG-5988 > To my best understanding, when it comes to transitive dependencies, both > "nearest first" and "declared first" are random variables which user can't > control unless they re-declare all the dependencies in their local pom. I > suggest Maven should not use random variables like "dependency depth" or > "dependency order" to drive conflict resolution. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (MNG-7852) Use all the versions for dependency resolution rather than "nearest first" or "declared first"
[
https://issues.apache.org/jira/browse/MNG-7852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747891#comment-17747891
]
Vladimir Sitnikov edited comment on MNG-7852 at 7/27/23 10:03 AM:
--
{quote}Am really unsure what you want to see as solution here
{quote}
It would really help me if you could suggest how Maven solves the issues in the
description: 1, 2, 3.
The only workaround I can imagine for now is to re-declare all the dependencies
manually, including the runtime dependencies of all the transitives. Of course,
if dependencies are re-declared, they might get out of date, so it doesn't
sound like a reasonable solution to me.
I suggest that Maven expand the complete dependency (regular dependency, and
dependencyManagement) tree, and then perform conflict resolution. When dealing
with conflict resolution, "newest wins" makes more sense than "oldest wins".
"newest wins unless there are other constraints" is even easier to describe to
the end users, it does not suffer from "unexpected dependency downgrade", and
it does not rely on semi-random (uncontrollable) variables like "dependency
depth" or "dependency order".
There's an option of "fail on conflict", however, it is not that practical.
Of course, it would be nice if Maven supported rich version constraints like
"requires", "prefers", "rejects", "strictly", however, it is a different story.
was (Author: vladimirsitnikov):
{quote}Am really unsure what you want to see as solution here
{quote}
It would really help me if you could suggest how Maven solves the issues in the
description: 1, 2, 3.
The only workaround I can imagine for now is to re-declare all the dependencies
manually, including the runtime dependencies of all the transitives. Of course,
if dependencies are re-declared, they might get out of date, so it doesn't
sound like a reasonable solution to me.
I suggest that Maven expand the complete dependency (regular dependency, and
dependencyManagement) tree, and then perform conflict resolution. When dealing
with conflict resolution, "newest wins" makes more sense than "oldest wins".
There's an option of "fail on conflict", however, it is not that practical.
Of course, it would be nice if Maven supported rich version constraints like
"requires", "prefers", "rejects", "strictly", however, it is a different story.
> Use all the versions for dependency resolution rather than "nearest first" or
> "declared first"
> --
>
> Key: MNG-7852
> URL: https://issues.apache.org/jira/browse/MNG-7852
> Project: Maven
> Issue Type: Improvement
> Components: Dependencies
>Reporter: Vladimir Sitnikov
>Priority: Major
>
> Currently, Maven uses "nearest first", "declared first" rules for conflict
> resolution:
> https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html
> I suggest those rules are removed since they produce hard to reason
> resolutions for transitive dependencies.
> Below I list reasons why both "nearest wins" and "declared first" yield
> hard-to-predict behaviours, and they are likely to produce dependency
> downgrades and the associated runtime errors.
> Here are some examples:
> 1) "Nearest first". Even though the rule sounds easy, it is not something the
> users can control. For instance, if the project does not use Guava library,
> some of the transitive dependencies could add a dependency on Guava. The user
> has no control which dependency would be "the nearest" to declare Guava, so
> user has literally no way to tell which Guava version will be used.
> The only workaround I see for the users is to declare Guava dependency
> explicitly even though the project does not need it directly. It sounds like
> Maven requires users to re-declare all the possible dependencies, including
> the runtime-only ones.
> 2) "declared first". Of course, dependency order matters for classpath order,
> however, it is not predictable in practice, and it might result in
> downgrading dependencies. Imagine the project does not use Guice. However,
> transitive dependencies might use Guice. At the same time, they might start
> using Guice and stop using Guice, so the user can never tell which will be
> the first project that uses Guice. Unfortunately, in Maven, the first project
> that declares dependency wins, so it might easily be the case that the first
> mention of Guice will reference outdated version that would be incompatible
> with the newer one required in another dependency.
> 3) Here's a real-life case: Maven downgrades protobuf-java dependency causing
> something like NoSuchMethodError at the runtime. The step to reproduce is to
> add dependency on dev.sigstore:sigstore-java:0.4.0. See [~hboutemy] analysis
>
