[
https://issues.apache.org/jira/browse/HDDS-7372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-7372:
-----------------------------------
Description:
{{:ozone-dist}} copies all runtime dependencies, including transitive ones,
into {{share/ozone/lib}}. The same dependency may be part of multiple
dependency sub-trees with different versions, e.g.:
{code}
org.apache.ozone:ozone-manager:jar:1.3.0-SNAPSHOT
+- org.apache.ranger:ranger-intg:jar:2.3.0:compile
| \- org.apache.ranger:ranger-plugins-common:jar:2.3.0:compile
| +- org.apache.ranger:ranger-plugins-audit:jar:2.3.0:compile
| | +- joda-time:joda-time:jar:2.10.6:compile
{code}
and
{code}
org.apache.ozone:ozone-tools:jar:1.3.0-SNAPSHOT
+- com.amazonaws:aws-java-sdk-core:jar:1.12.261:compile
| \- joda-time:joda-time:jar:2.8.1:compile
{code}
but {{maven-dependency-plugin}} ends up copying only one of these:
{code}
org.apache.ozone:ozone-dist:jar:1.3.0-SNAPSHOT
+- org.apache.ozone:ozone-tools:jar:1.3.0-SNAPSHOT:compile
| +- com.amazonaws:aws-java-sdk-core:jar:1.12.261:compileL:q
| | \- joda-time:joda-time:jar:2.8.1:compile
{code}
The classpath of the Ozone modules still include their own dependencies:
{code:title=grep -E -o 'joda-time[^:]*.jar'
hadoop-ozone/dist/target/ozone-1.3.0-SNAPSHOT/share/ozone/classpath/ozone-manager.classpath}
joda-time-2.10.6.jar
{code}
thus we could run into missing classes, should these transitive dependencies
be actually used.
We should add dependency convergence check to the build process to flag such
problems.
https://maven.apache.org/enforcer/enforcer-rules/dependencyConvergence.html
Example from Hadoop:
https://github.com/apache/hadoop/blob/c4aa41aa801302b5efefa2a59e590d8c02c72223/hadoop-project/pom.xml#L2291-L2336
was:
{{:ozone-dist}} copies all runtime dependencies, including transitive ones,
into {{share/ozone/lib}}. The same dependency may be part of multiple
dependency sub-trees, e.g.:
{code}
org.apache.ozone:ozone-manager:jar:1.3.0-SNAPSHOT
+- org.apache.ranger:ranger-intg:jar:2.3.0:compile
| \- org.apache.ranger:ranger-plugins-common:jar:2.3.0:compile
| +- org.apache.ranger:ranger-plugins-audit:jar:2.3.0:compile
| | +- joda-time:joda-time:jar:2.10.6:compile
{code}
and
{code}
org.apache.ozone:ozone-tools:jar:1.3.0-SNAPSHOT
+- com.amazonaws:aws-java-sdk-core:jar:1.12.261:compile
| \- joda-time:joda-time:jar:2.8.1:compile
{code}
but {{maven-dependency-plugin}} ends up copying only one of these:
{code}
org.apache.ozone:ozone-dist:jar:1.3.0-SNAPSHOT
+- org.apache.ozone:ozone-tools:jar:1.3.0-SNAPSHOT:compile
| +- com.amazonaws:aws-java-sdk-core:jar:1.12.261:compileL:q
| | \- joda-time:joda-time:jar:2.8.1:compile
{code}
The classpath of the Ozone modules still include their own dependencies:
{code:title=grep -E -o 'joda-time[^:]*.jar'
hadoop-ozone/dist/target/ozone-1.3.0-SNAPSHOT/share/ozone/classpath/ozone-manager.classpath}
joda-time-2.10.6.jar
{code}
thus we could run into missing classes, should these transitive dependencies
be actually used.
We should add dependency convergence check to the build process to flag such
problems.
https://maven.apache.org/enforcer/enforcer-rules/dependencyConvergence.html
Example from Hadoop:
https://github.com/apache/hadoop/blob/c4aa41aa801302b5efefa2a59e590d8c02c72223/hadoop-project/pom.xml#L2291-L2336
> Check dependency convergence
> ----------------------------
>
> Key: HDDS-7372
> URL: https://issues.apache.org/jira/browse/HDDS-7372
> Project: Apache Ozone
> Issue Type: Task
> Components: build
> Reporter: Attila Doroszlai
> Priority: Major
>
> {{:ozone-dist}} copies all runtime dependencies, including transitive ones,
> into {{share/ozone/lib}}. The same dependency may be part of multiple
> dependency sub-trees with different versions, e.g.:
> {code}
> org.apache.ozone:ozone-manager:jar:1.3.0-SNAPSHOT
> +- org.apache.ranger:ranger-intg:jar:2.3.0:compile
> | \- org.apache.ranger:ranger-plugins-common:jar:2.3.0:compile
> | +- org.apache.ranger:ranger-plugins-audit:jar:2.3.0:compile
> | | +- joda-time:joda-time:jar:2.10.6:compile
> {code}
> and
> {code}
> org.apache.ozone:ozone-tools:jar:1.3.0-SNAPSHOT
> +- com.amazonaws:aws-java-sdk-core:jar:1.12.261:compile
> | \- joda-time:joda-time:jar:2.8.1:compile
> {code}
> but {{maven-dependency-plugin}} ends up copying only one of these:
> {code}
> org.apache.ozone:ozone-dist:jar:1.3.0-SNAPSHOT
> +- org.apache.ozone:ozone-tools:jar:1.3.0-SNAPSHOT:compile
> | +- com.amazonaws:aws-java-sdk-core:jar:1.12.261:compileL:q
> | | \- joda-time:joda-time:jar:2.8.1:compile
> {code}
> The classpath of the Ozone modules still include their own dependencies:
> {code:title=grep -E -o 'joda-time[^:]*.jar'
> hadoop-ozone/dist/target/ozone-1.3.0-SNAPSHOT/share/ozone/classpath/ozone-manager.classpath}
> joda-time-2.10.6.jar
> {code}
> thus we could run into missing classes, should these transitive dependencies
> be actually used.
> We should add dependency convergence check to the build process to flag such
> problems.
> https://maven.apache.org/enforcer/enforcer-rules/dependencyConvergence.html
> Example from Hadoop:
> https://github.com/apache/hadoop/blob/c4aa41aa801302b5efefa2a59e590d8c02c72223/hadoop-project/pom.xml#L2291-L2336
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
