xichen01 opened a new pull request, #8008:
URL: https://github.com/apache/ozone/pull/8008
## What changes were proposed in this pull request?
shade third dependency for ozone client release package.
Ozone `ozone-filesystem-shaded` currently uses the include method to
determine which classes are shaded for org.xx and com.xx. This PR changes
org.xx and com.xx to exclude to determine which classes are shaded. After this
PR, newly added org.xx and com.xx dependencies will be shaded as long as they
are not excluded.
After this PR, except for jakarta and javax (jakarta and javax are
jvm-related dependencies, so they are retained), the rest of the third-party
dependencies have been shaded.
This can effectively reduce dependency conflicts with other projects. Ozone
shades all its own dependencies (excluding native provide dependencies).
### Before
```bash
[~/share/ozone/lib/ozone-filesystem-hadoop3-client-2.0.0-SNAPSHOT]$ ls -lh
../ozone-filesystem-hadoop3-client-2.0.0-SNAPSHOT.jar
-rw-r--r-- 1 root root 71M Mar 3 17:27
../ozone-filesystem-hadoop3-client-2.0.0-SNAPSHOT.jar
[~/share/ozone/lib/ozone-filesystem-hadoop3-client-2.0.0-SNAPSHOT]$ jar xvf
../ozone-filesystem-hadoop3-client-2.0.0-SNAPSHOT.jar
[~/share/ozone/lib/ozone-filesystem-hadoop3-client-2.0.0-SNAPSHOT]$
[root@rocky9
~/ozone-2.0.0-SNAPSHOT/share/ozone/lib/ozone-filesystem-hadoop3-client-2.0.0-SNAPSHOT]$
tree -d -L 2
.
├── android
│ └── annotation
├── com
│ ├── google
│ └── nimbusds
├── google
│ ├── api
//...
├── jakarta
│ └── annotation
├── javassist
│ ├── bytecode
//...
├── javax
│ ├── activation
│ ├── annotation
│ └── xml
├── META-INF
│ ├── license
//...
├── mozilla
├── net
│ └── jcip
└── org
├── apache
//...
47 directories
```
### After
```bash
[~/share/ozone/lib/ozone-filesystem-hadoop3-client-2.0.0-SNAPSHOT]$
ls -lh ../ozone-filesystem-hadoop3-client-2.0.0-SNAPSHOT.jar
-rw-r--r-- 1 root root 69M Mar 5 14:55
../ozone-filesystem-hadoop3-client-2.0.0-SNAPSHOT.jar
[~/share/ozone/lib/ozone-filesystem-hadoop3-client-2.0.0-SNAPSHOT]$ jar xvf
../ozone-filesystem-hadoop3-client-2.0.0-SNAPSHOT.jar
[~/share/ozone/lib/ozone-filesystem-hadoop3-client-2.0.0-SNAPSHOT]$ tree -d
-L 2
tree -d -L 2
.
├── jakarta
│ └── annotation
├── javax
│ ├── activation
│ ├── annotation
│ └── xml
├── META-INF
│ ├── license
//...
├── mozilla
└── org
├── apache
└── bouncycastle
17 directories
```
## What is the link to the Apache JIRA
## How was this patch tested?
- CI
https://github.com/xichen01/ozone/actions/runs/13658629724
- Local manual test
Test with hadoop client, the test passed.
```bash
$ env | grep HADOOP
HADOOP_USER_RPCPASSWORD=1
HADOOP_OPTS= -Djava.library.path=/root/hadoop-3.3.4//lib/
HADOOP_CLASSPATH=/root/ozone/hadoop-ozone/dist/target/ozone-2.0.0-SNAPSHOT/share/ozone/lib/ozone-filesystem-hadoop3-*.jar:
HADOOP_CONF_DIR=/root/ozone/hadoop-ozone/dev-support/intellij/
HADOOP_HOME=/root/hadoop-3.3.4/
HADOOP_COMMON_LIB_NATIVE_DIR=/root/hadoop-3.3.4//lib/native/
$ ozone sh bucket create s3v/bucket1
$ pwd
/root/hadoop-3.3.4/bin
$ ./hadoop jar
../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-*-tests.jar nnbench
-baseDir ofs://localhost/s3v/bucket1 -maps 10 -reduces 5 -numberOfFiles 500
-operation create_write
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
