This is an automated email from the ASF dual-hosted git repository.
roryqi pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-uniffle.git
The following commit(s) were added to refs/heads/master by this push:
new cb1d2311c [#2030][FOLLOWUP] Exclude log4j dependencies from shaded
plugin (#2134)
cb1d2311c is described below
commit cb1d2311cc164731d7831f69a1cdfca2729882f2
Author: maobaolong <[email protected]>
AuthorDate: Sat Sep 21 14:07:33 2024 +0800
[#2030][FOLLOWUP] Exclude log4j dependencies from shaded plugin (#2134)
### What changes were proposed in this pull request?
Do not include the lo4j dependencies to the shaded uber jar.
### Why are the changes needed?
The log4j related dependencies should be provided by spark environment.
Without this PR, the classes included to the shaded jar which is not expected.
<img width="1169" alt="image"
src="https://github.com/user-attachments/assets/54955957-2d84-4c86-b6ce-0ab172d90160";>
And the check shaded script found this issue.
```
[exec]
checkShaded:/home/runner/work/incubator-uniffle/incubator-uniffle/client-spark/spark3-shaded/target/rss-client-spark3-shaded-0.10.0-SNAPSHOT.jar
[exec] unshaded count: 11
[exec] unshaded content:
[exec] org/apache/logging/slf4j/Log4jEventBuilder.class
[exec] org/apache/logging/slf4j/Log4jLogger.class
[exec] org/apache/logging/slf4j/Log4jLoggerFactory.class
[exec] org/apache/logging/slf4j/Log4jMDCAdapter$1.class
[exec]
org/apache/logging/slf4j/Log4jMDCAdapter$ThreadLocalMapOfStacks.class
[exec] org/apache/logging/slf4j/Log4jMDCAdapter.class
[exec] org/apache/logging/slf4j/Log4jMarker.class
[exec] org/apache/logging/slf4j/Log4jMarkerFactory.class
[exec] org/apache/logging/slf4j/SLF4JLoggingException.class
[exec] org/apache/logging/slf4j/SLF4JServiceProvider.class
[exec] org/apache/logging/slf4j/package-info.class
[exec] check failed.
```
The root cause for this issue I guess is here, maybe we should use
`provided` scope for the `log4j-slf4j2-impl` dependency, it should be provided
in the spark env classpath.
<img width="552" alt="image"
src="https://github.com/user-attachments/assets/9af79159-ff65-4ede-9c7b-26d8f8402e5b";>
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
After this PR
<img width="495" alt="image"
src="https://github.com/user-attachments/assets/7179e9d7-65a4-417e-a56f-fdb6e362a21e";>
The log are output as expected.
<img width="976" alt="image"
src="https://github.com/user-attachments/assets/cd736345-df9f-40b1-8fb5-2ea1c329f152";>
---
client-spark/spark2-shaded/pom.xml | 7 ++++---
client-spark/spark3-shaded/pom.xml | 7 ++++---
2 files changed, 8 insertions(+), 6 deletions(-)
diff --git a/client-spark/spark2-shaded/pom.xml
b/client-spark/spark2-shaded/pom.xml
index 703318a27..1fe1180dc 100644
--- a/client-spark/spark2-shaded/pom.xml
+++ b/client-spark/spark2-shaded/pom.xml
@@ -90,9 +90,10 @@
</goals>
<configuration>
<artifactSet>
- <includes>
- <include>*:*</include>
- </includes>
+ <excludes>
+ <exclude>org.apache.logging.log4j:log4j-slf4j-impl</exclude>
+ <exclude>org.apache.logging.log4j:log4j-slf4j2-impl</exclude>
+ </excludes>
</artifactSet>
<finalName>${project.artifactId}-${project.version}</finalName>
<transformers>
diff --git a/client-spark/spark3-shaded/pom.xml
b/client-spark/spark3-shaded/pom.xml
index 0312632ae..e59e48bba 100644
--- a/client-spark/spark3-shaded/pom.xml
+++ b/client-spark/spark3-shaded/pom.xml
@@ -90,9 +90,10 @@
</goals>
<configuration>
<artifactSet>
- <includes>
- <include>*:*</include>
- </includes>
+ <excludes>
+ <exclude>org.apache.logging.log4j:log4j-slf4j-impl</exclude>
+ <exclude>org.apache.logging.log4j:log4j-slf4j2-impl</exclude>
+ </excludes>
</artifactSet>
<finalName>${project.artifactId}-${project.version}</finalName>
<transformers>
