[
https://issues.apache.org/jira/browse/HADOOP-19708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18028393#comment-18028393
]
Steve Loughran commented on HADOOP-19708:
-----------------------------------------
(note the shading is troubled anyway
{code}
[INFO] Dependency-reduced POM written at:
/Users/stevel/Projects/hadoop-trunk/hadoop-cloud-storage-project/hadoop-tos/dependency-reduced-pom.xml
[WARNING] httpclient5-5.3.jar, httpcore5-5.2.4.jar, httpcore5-h2-5.2.4.jar
define 3 overlapping resources:
[WARNING] - META-INF/DEPENDENCIES
[WARNING] - META-INF/LICENSE
[WARNING] - META-INF/NOTICE
[WARNING] hadoop-tos-3.5.0-SNAPSHOT.jar, httpclient5-5.3.jar,
httpcore5-5.2.4.jar, httpcore5-h2-5.2.4.jar, nimbus-jose-jwt-10.4.jar,
ve-tos-java-sdk-hadoop-2.8.9.jar define 1 overlapping resource:
[WARNING] - META-INF/MANIFEST.MF
[WARNING] maven-shade-plugin has detected that some files are
[WARNING] present in two or more JARs. When this happens, only one
[WARNING] single version of the file is copied to the uber jar.
[WARNING] Usually this is not harmful and you can skip these warnings,
[WARNING] otherwise try to manually exclude artifacts based on
[WARNING] mvn dependency:tree -Ddetail=true and the above output.
[WARNING] See https://maven.apache.org/plugins/maven-shade-plugin/
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing
/Users/stevel/Projects/hadoop-trunk/hadoop-cloud-storage-project/hadoop-tos/target/hadoop-tos-3.5.0-SNAPSHOT.jar
with
/Users/stevel/Projects/hadoop-trunk/hadoop-cloud-storage-project/hadoop-tos/target/hadoop-tos-3.5.0-SNAPSHOT-shaded.jar
[INFO]
[INFO] --- cyclonedx:2.9.1:makeBom (default) @ hadoop-tos ---
{code}
> volcano tos: remove shading
> ---------------------------
>
> Key: HADOOP-19708
> URL: https://issues.apache.org/jira/browse/HADOOP-19708
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs/volcano
> Affects Versions: 3.5.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
>
> hadoop-tos is shaded, includes things like httpclient5, which leads to a big
> 4M artifact, with unknown content inside
> {code}
> 92K share/hadoop/common/lib/hadoop-aliyun-3.5.0-20250916.124028-685.jar
> 912K share/hadoop/common/lib/hadoop-aws-3.5.0-20250916.124028-686.jar
> 808K share/hadoop/common/lib/hadoop-azure-3.5.0-20250916.124028-685.jar
> 36K
> share/hadoop/common/lib/hadoop-azure-datalake-3.5.0-20250916.124028-685.jar
> 72K share/hadoop/common/lib/hadoop-cos-3.5.0-20250916.124028-683.jar
> 136K share/hadoop/common/lib/hadoop-gcp-3.5.0-SNAPSHOT.jar
> 140K share/hadoop/common/lib/hadoop-huaweicloud-3.5.0-SNAPSHOT.jar
> 3.8M share/hadoop/common/lib/hadoop-tos-3.5.0-20250916.124028-202.jar
> {code}
> One thing it includes is yet-another mozilla/public-suffix-list.txt. These
> are a recurrent PITA and I don't want
> to find them surfacing again.
> {code}
> 15. Required Resources
> ======================
> resource: mozilla/public-suffix-list.txt
>
> jar:file:/Users/stevel/Projects/Releases/hadoop-3.5.0-SNAPSHOT/share/hadoop/common/lib/hadoop-tos-3.5.0-20250916.124028-202.jar!/mozilla/public-suffix-list.txt
> {code}
> Plan
> * Move the shade stage behind a profile; off for ASF releases. Exclude
> mozilla/public-suffix-list.txt
> * Explicitly declare and manage httpclient5 dependency
> * hadoop-cloud-storage pom to include hadoop-tos but not dependencies in
> build, unless asked.
> * LICENSE-binary to declare optional redist of ve-tos-java-sdk-hadoop and its
> license.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
