[
https://issues.apache.org/jira/browse/HDDS-12763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HDDS-12763:
-----------------------------------
Description:
https://github.com/apache/ozone/blob/45eee3fa5d29f49eaed0758d1703d446d56e70fb/hadoop-hdds/hadoop-dependency-client/README.md#L15
{quote}
This helper project provides a reduced view to the Hadoop dependencies. Hadoop
common by default depends on a lot of other projects which are not required for
the parts used in Ozone.
The exclude rules should be updated on each version bump. If Hadoop introduce
new dependencies we need to exclude them (unless they are required).
{quote}
The last update to the hadoop-dependency-client pom.xml was 4 years ago and
when it was Hadoop 3.2.1. We are now at 3.4.1 and it needs to be tighten again.
I don't think it's a blocker for Ozone 2.0.0 though.
According to HDDS-3312, the process to make a reduced transitive dependencies:
{quote}
hadoop-ozone/common and hadoop-hdds/common projects are common between client
and server. Therefore we should remove any server side utilities / dependencies
from them as they would be added to the client classpath which makes harder the
Ozone client / ozonefs adoption.
hadoop-hdds should depend on a minimal set of hadoop dependencies (can be
managed a separated technical project / pom.xml)
code shared between server side projects (and not with the client) should be
moved to the framework
OM related code should be moved to the ozone-manager instead of ozone/common
{quote}
Current dependency tree:
{noformat}
[INFO] --- dependency:3.8.1:tree (default-cli) @ hdds-hadoop-dependency-client
---
[INFO] org.apache.ozone:hdds-hadoop-dependency-client:jar:2.1.0-SNAPSHOT
[INFO] +- com.nimbusds:nimbus-jose-jwt:jar:9.40:compile
[INFO] +- org.apache.hadoop:hadoop-annotations:jar:3.4.1:compile
[INFO] +- org.apache.hadoop:hadoop-common:jar:3.4.1:compile
[INFO] | +-
org.apache.hadoop.thirdparty:hadoop-shaded-protobuf_3_25:jar:1.3.0:compile
[INFO] | +- org.apache.hadoop.thirdparty:hadoop-shaded-guava:jar:1.3.0:compile
[INFO] | +- commons-cli:commons-cli:jar:1.8.0:compile
[INFO] | +- org.apache.httpcomponents:httpclient:jar:4.5.14:compile
[INFO] | +- jakarta.activation:jakarta.activation-api:jar:1.2.2:runtime
[INFO] | +- org.codehaus.jettison:jettison:jar:1.5.4:compile
[INFO] | +- org.apache.commons:commons-configuration2:jar:2.11.0:compile
[INFO] | +- org.apache.commons:commons-text:jar:1.12.0:compile
[INFO] | +- com.google.re2j:re2j:jar:1.7:compile
[INFO] | +- org.apache.hadoop:hadoop-auth:jar:3.4.1:compile
[INFO] | | \- org.apache.kerby:kerb-util:jar:1.0.1:compile
[INFO] | | +- org.apache.kerby:kerby-config:jar:1.0.1:compile
[INFO] | | \- org.apache.kerby:kerb-crypto:jar:1.0.1:compile
[INFO] | +- io.netty:netty-handler:jar:4.1.109.Final:compile
[INFO] | | +- io.netty:netty-common:jar:4.1.109.Final:compile
[INFO] | | +- io.netty:netty-resolver:jar:4.1.109.Final:compile
[INFO] | | +- io.netty:netty-buffer:jar:4.1.109.Final:compile
[INFO] | | +- io.netty:netty-transport:jar:4.1.109.Final:compile
[INFO] | | +-
io.netty:netty-transport-native-unix-common:jar:4.1.109.Final:compile
[INFO] | | \- io.netty:netty-codec:jar:4.1.109.Final:compile
[INFO] | +- io.netty:netty-transport-native-epoll:jar:4.1.109.Final:compile
[INFO] | | \- io.netty:netty-transport-classes-epoll:jar:4.1.109.Final:compile
[INFO] | +- io.dropwizard.metrics:metrics-core:jar:3.2.6:compile
[INFO] | +- org.bouncycastle:bcprov-jdk18on:jar:1.80:compile
[INFO] | +- org.apache.kerby:kerb-core:jar:1.0.1:compile
[INFO] | | \- org.apache.kerby:kerby-pkix:jar:1.0.1:compile
[INFO] | | +- org.apache.kerby:kerby-asn1:jar:1.0.1:compile
[INFO] | | \- org.apache.kerby:kerby-util:jar:1.0.1:compile
[INFO] | +- org.codehaus.woodstox:stax2-api:jar:4.2.2:compile
[INFO] | \- com.fasterxml.woodstox:woodstox-core:jar:5.4.0:compile
[INFO] +- org.apache.hadoop:hadoop-hdfs-client:jar:3.4.1:compile
[INFO] | +- org.apache.httpcomponents:httpcore:jar:4.4.16:compile
[INFO] | +- com.fasterxml.jackson.core:jackson-annotations:jar:2.16.2:compile
[INFO] | \- com.fasterxml.jackson.core:jackson-databind:jar:2.16.2:compile
[INFO] | \- com.fasterxml.jackson.core:jackson-core:jar:2.16.2:compile
[INFO] +- org.xerial.snappy:snappy-java:jar:1.1.10.7:compile
[INFO] +- org.assertj:assertj-core:jar:3.27.3:test
[INFO] | \- net.bytebuddy:byte-buddy:jar:1.15.11:test
[INFO] +- org.junit.jupiter:junit-jupiter-api:jar:5.12.1:test
[INFO] | +- org.opentest4j:opentest4j:jar:1.3.0:test
[INFO] | +- org.junit.platform:junit-platform-commons:jar:1.12.1:test
[INFO] | \- org.apiguardian:apiguardian-api:jar:1.1.2:test
[INFO] +- org.junit.jupiter:junit-jupiter-params:jar:5.12.1:test
[INFO] +- org.mockito:mockito-core:jar:4.11.0:test
[INFO] | +- net.bytebuddy:byte-buddy-agent:jar:1.12.19:test
[INFO] | \- org.objenesis:objenesis:jar:3.3:test
[INFO] +- org.mockito:mockito-inline:jar:4.11.0:test
[INFO] \- org.mockito:mockito-junit-jupiter:jar:4.11.0:test
{noformat}
was:
https://github.com/apache/ozone/blob/45eee3fa5d29f49eaed0758d1703d446d56e70fb/hadoop-hdds/hadoop-dependency-client/README.md#L15
{quote}
This helper project provides a reduced view to the Hadoop dependencies. Hadoop
common by default depends on a lot of other projects which are not required for
the parts used in Ozone.
The exclude rules should be updated on each version bump. If Hadoop introduce
new dependencies we need to exclude them (unless they are required).
{quote}
The last update to the hadoop-dependency-client pom.xml was 4 years ago and
when it was Hadoop 3.2.1. We are now at 3.4.1 and it needs to be tighten again.
I don't think it's a blocker for Ozone 2.0.0 though.
According to HDDS-3312, the process to make a reduced transitive dependencies:
{quote}
hadoop-ozone/common and hadoop-hdds/common projects are common between client
and server. Therefore we should remove any server side utilities / dependencies
from them as they would be added to the client classpath which makes harder the
Ozone client / ozonefs adoption.
hadoop-hdds should depend on a minimal set of hadoop dependencies (can be
managed a separated technical project / pom.xml)
code shared between server side projects (and not with the client) should be
moved to the framework
OM related code should be moved to the ozone-manager instead of ozone/common
{quote}
> Update hadoop-dependency-client pom.xml exclude list
> ----------------------------------------------------
>
> Key: HDDS-12763
> URL: https://issues.apache.org/jira/browse/HDDS-12763
> Project: Apache Ozone
> Issue Type: Task
> Affects Versions: 2.0.0
> Reporter: Wei-Chiu Chuang
> Priority: Major
>
> https://github.com/apache/ozone/blob/45eee3fa5d29f49eaed0758d1703d446d56e70fb/hadoop-hdds/hadoop-dependency-client/README.md#L15
> {quote}
> This helper project provides a reduced view to the Hadoop dependencies.
> Hadoop common by default depends on a lot of other projects which are not
> required for the parts used in Ozone.
> The exclude rules should be updated on each version bump. If Hadoop introduce
> new dependencies we need to exclude them (unless they are required).
> {quote}
> The last update to the hadoop-dependency-client pom.xml was 4 years ago and
> when it was Hadoop 3.2.1. We are now at 3.4.1 and it needs to be tighten
> again.
> I don't think it's a blocker for Ozone 2.0.0 though.
> According to HDDS-3312, the process to make a reduced transitive dependencies:
> {quote}
> hadoop-ozone/common and hadoop-hdds/common projects are common between client
> and server. Therefore we should remove any server side utilities /
> dependencies from them as they would be added to the client classpath which
> makes harder the Ozone client / ozonefs adoption.
> hadoop-hdds should depend on a minimal set of hadoop dependencies (can be
> managed a separated technical project / pom.xml)
> code shared between server side projects (and not with the client) should be
> moved to the framework
> OM related code should be moved to the ozone-manager instead of ozone/common
> {quote}
> Current dependency tree:
> {noformat}
> [INFO] --- dependency:3.8.1:tree (default-cli) @
> hdds-hadoop-dependency-client ---
> [INFO] org.apache.ozone:hdds-hadoop-dependency-client:jar:2.1.0-SNAPSHOT
> [INFO] +- com.nimbusds:nimbus-jose-jwt:jar:9.40:compile
> [INFO] +- org.apache.hadoop:hadoop-annotations:jar:3.4.1:compile
> [INFO] +- org.apache.hadoop:hadoop-common:jar:3.4.1:compile
> [INFO] | +-
> org.apache.hadoop.thirdparty:hadoop-shaded-protobuf_3_25:jar:1.3.0:compile
> [INFO] | +-
> org.apache.hadoop.thirdparty:hadoop-shaded-guava:jar:1.3.0:compile
> [INFO] | +- commons-cli:commons-cli:jar:1.8.0:compile
> [INFO] | +- org.apache.httpcomponents:httpclient:jar:4.5.14:compile
> [INFO] | +- jakarta.activation:jakarta.activation-api:jar:1.2.2:runtime
> [INFO] | +- org.codehaus.jettison:jettison:jar:1.5.4:compile
> [INFO] | +- org.apache.commons:commons-configuration2:jar:2.11.0:compile
> [INFO] | +- org.apache.commons:commons-text:jar:1.12.0:compile
> [INFO] | +- com.google.re2j:re2j:jar:1.7:compile
> [INFO] | +- org.apache.hadoop:hadoop-auth:jar:3.4.1:compile
> [INFO] | | \- org.apache.kerby:kerb-util:jar:1.0.1:compile
> [INFO] | | +- org.apache.kerby:kerby-config:jar:1.0.1:compile
> [INFO] | | \- org.apache.kerby:kerb-crypto:jar:1.0.1:compile
> [INFO] | +- io.netty:netty-handler:jar:4.1.109.Final:compile
> [INFO] | | +- io.netty:netty-common:jar:4.1.109.Final:compile
> [INFO] | | +- io.netty:netty-resolver:jar:4.1.109.Final:compile
> [INFO] | | +- io.netty:netty-buffer:jar:4.1.109.Final:compile
> [INFO] | | +- io.netty:netty-transport:jar:4.1.109.Final:compile
> [INFO] | | +-
> io.netty:netty-transport-native-unix-common:jar:4.1.109.Final:compile
> [INFO] | | \- io.netty:netty-codec:jar:4.1.109.Final:compile
> [INFO] | +- io.netty:netty-transport-native-epoll:jar:4.1.109.Final:compile
> [INFO] | | \-
> io.netty:netty-transport-classes-epoll:jar:4.1.109.Final:compile
> [INFO] | +- io.dropwizard.metrics:metrics-core:jar:3.2.6:compile
> [INFO] | +- org.bouncycastle:bcprov-jdk18on:jar:1.80:compile
> [INFO] | +- org.apache.kerby:kerb-core:jar:1.0.1:compile
> [INFO] | | \- org.apache.kerby:kerby-pkix:jar:1.0.1:compile
> [INFO] | | +- org.apache.kerby:kerby-asn1:jar:1.0.1:compile
> [INFO] | | \- org.apache.kerby:kerby-util:jar:1.0.1:compile
> [INFO] | +- org.codehaus.woodstox:stax2-api:jar:4.2.2:compile
> [INFO] | \- com.fasterxml.woodstox:woodstox-core:jar:5.4.0:compile
> [INFO] +- org.apache.hadoop:hadoop-hdfs-client:jar:3.4.1:compile
> [INFO] | +- org.apache.httpcomponents:httpcore:jar:4.4.16:compile
> [INFO] | +- com.fasterxml.jackson.core:jackson-annotations:jar:2.16.2:compile
> [INFO] | \- com.fasterxml.jackson.core:jackson-databind:jar:2.16.2:compile
> [INFO] | \- com.fasterxml.jackson.core:jackson-core:jar:2.16.2:compile
> [INFO] +- org.xerial.snappy:snappy-java:jar:1.1.10.7:compile
> [INFO] +- org.assertj:assertj-core:jar:3.27.3:test
> [INFO] | \- net.bytebuddy:byte-buddy:jar:1.15.11:test
> [INFO] +- org.junit.jupiter:junit-jupiter-api:jar:5.12.1:test
> [INFO] | +- org.opentest4j:opentest4j:jar:1.3.0:test
> [INFO] | +- org.junit.platform:junit-platform-commons:jar:1.12.1:test
> [INFO] | \- org.apiguardian:apiguardian-api:jar:1.1.2:test
> [INFO] +- org.junit.jupiter:junit-jupiter-params:jar:5.12.1:test
> [INFO] +- org.mockito:mockito-core:jar:4.11.0:test
> [INFO] | +- net.bytebuddy:byte-buddy-agent:jar:1.12.19:test
> [INFO] | \- org.objenesis:objenesis:jar:3.3:test
> [INFO] +- org.mockito:mockito-inline:jar:4.11.0:test
> [INFO] \- org.mockito:mockito-junit-jupiter:jar:4.11.0:test
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
