[
https://issues.apache.org/jira/browse/HDDS-691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16656966#comment-16656966
]
Elek, Marton commented on HDDS-691:
-----------------------------------
Thank you very much the answers [~szetszwo]. It's much more clean for me now.
1. Personally I prefer to use the <exclude> rules to exclude dependencies and
use scopes like provided/runtime to classify dependencies. I know that in the
hdfs project the provided scope is used as an ultimate tool to exclude
transitive dependencies but I don't like it. For example this is the reason why
it's almost impossible to run hadoop cluster from IDE or with mvn:run. In the
hdds/ozone projects we have provided dependencies only when we have reason for
that (for datanode plugin the hadoop jar files are "provided" by the original
datanode environment) and I can start a full ozone cluster just from the IDE.
2. I tried to understand why is it a problem if we have this kind of
divergence. It seems to be a false positive warning for me as from maven point
of view the 3.3.0-20181017.235840-140 and 3.3.0-SNAPSHOT should be exactly the
same.
3. This behaviour of the enforcer plugin is set by this rule definition:
"<DependencyConvergence<uniqueVersions>true</uniqueVersions></DependencyConvergence>"
4. This line is added by HADOOP-9962 as an answer of HADOOP-9961. There was no
problem with the snapshot dependencies in HADOOP-9961 there were divergences
between fixed versions. I didn't find any reasoning why the uniqueVersion
should be false.
5. The main reason to use the enforcer plugin is in hadoop hdfs/yarn the
classpath jar files are collected per project basis. It's defined by the
hadoop-project-dist and each project creates a standard dist directory
structure to the target/ folder. And the hadoop-dist project (or more precisely
the dist-layout-stitching shell script) copies all the prepared directories to
one place. In case of a dependency divergence, multiple versions will be added
to the classpath from the same jar file after this copy as all the jar files
from the classpath directories will be added to the classpath.
6. This is not a problem for ozone/hdds any more. Since HDDS-447 we don't add
everything to the classpath. Each of the services uses separated classpath.
There shouldn't be any problem in case of different version of dependencies.
(In fact it's possible to use different version from the same dependency, for
example s3gateway uses a newer jaxrs/jersey than the other projects).
7. Therefore I believe it's totally safe to turn off (not the dependency
convergence check, just) the uniqueVersions flag for the DependencyConvergence
rule:
{code}
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-enforcer-plugin</artifactId>
<executions>
<execution>
<id>depcheck</id>
<configuration>
<rules>
<DependencyConvergence>
<uniqueVersions>false</uniqueVersions>
</DependencyConvergence>
</rules>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
{code}
> Dependency convergence error for org.apache.hadoop:hadoop-annotations
> ---------------------------------------------------------------------
>
> Key: HDDS-691
> URL: https://issues.apache.org/jira/browse/HDDS-691
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Reporter: Tsz Wo Nicholas Sze
> Assignee: Tsz Wo Nicholas Sze
> Priority: Major
> Attachments: HDDS-691_20181018.patch
>
>
> {code}
> [WARNING]
> Dependency convergence error for
> org.apache.hadoop:hadoop-annotations:3.3.0-SNAPSHOT paths to dependency are:
> +-org.apache.hadoop:hadoop-hdds-common:0.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:3.3.0-20181017.235917-140
> +-org.apache.hadoop:hadoop-annotations:3.3.0-SNAPSHOT
> and
> +-org.apache.hadoop:hadoop-hdds-common:0.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:3.3.0-20181017.235917-140
> +-org.apache.hadoop:hadoop-annotations:3.3.0-SNAPSHOT
> and
> +-org.apache.hadoop:hadoop-hdds-common:0.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-annotations:3.3.0-20181017.235840-140
> [WARNING] Rule 0: org.apache.maven.plugins.enforcer.DependencyConvergence
> failed with message:
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
