zzcclp opened a new pull request, #5109:
URL: https://github.com/apache/incubator-gluten/pull/5109
## What changes were proposed in this pull request?
Fix the classes in the hadoop-common conflict when running ut local.
Env: Idea + Spark 3.3 + Hadoop 3.3.1
UT: GlutenClickHouseMergeTreeWriteOnS3Suite /
GlutenClickHouseMergeTreeWriteOnHDFSSuite Error:
```
java.lang.NoClassDefFoundError: xxxxx
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:634)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:619)
at
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3469)
at org.apache.hadoop.fs.FileSystem.access$300(FileSystem.java:174)
at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3574)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3521)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:540)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:288)
```
RC:
The jar `hive-hcatalog-core` includes the hadoop-common 2.7.2 and
hadoop-hdfs 2.7.2, which conflict with the hadoop 3.3.1 when running ut with
hadoop-3.3 profile.
Close #5108.
(Fixes: #5108)
## How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration
tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise,
remove this)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
