This is an automated email from the ASF dual-hosted git repository.
chengpan pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-celeborn.git
The following commit(s) were added to refs/heads/main by this push:
new 5e3e9e442 [CELEBORN-906][FOLLOWUP] Removal of redundant dependency
`log4j-slf4j2-impl` from SBT profile `spark-3.4`
5e3e9e442 is described below
commit 5e3e9e442aa62c758770f9ed262910f2c4544ae9
Author: Fu Chen <[email protected]>
AuthorDate: Mon Aug 28 12:12:04 2023 +0800
[CELEBORN-906][FOLLOWUP] Removal of redundant dependency
`log4j-slf4j2-impl` from SBT profile `spark-3.4`
### What changes were proposed in this pull request?
As title
### Why are the changes needed?
To address the CI failure introduced in
https://github.com/apache/incubator-celeborn/pull/1831, this pull request
resolves the issue by removing the `log4j-slf4j2-impl` dependency from SBT
profile `spark-3.4`. This change is prompted by the pinning of `slf4j-api` to
version 1.7.36, rendering `log4j-slf4j2-impl` unnecessary.
```
[error] Test org.apache.spark.shuffle.celeborn.SortBasedPusherSuiteJ
failed: java.lang.NoSuchMethodError:
org.apache.logging.slf4j.Log4jLoggerFactory: method <init>()V not found, took
0.0 sec
[error] at
org.slf4j.impl.StaticLoggerBinder.<init>(StaticLoggerBinder.java:53)
[error] at
org.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:41)
[error] at org.slf4j.LoggerFactory.bind(LoggerFactory.java:150)
[error] at
org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)
[error] at
org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:417)
[error] at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:362)
[error] at
org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:3[88](https://github.com/apache/incubator-celeborn/actions/runs/5974971986/job/16210071148#step:4:89))
[error] at
org.apache.spark.shuffle.celeborn.SortBasedPusherSuiteJ.<clinit>(SortBasedPusherSuiteJ.java:51)
[error] ...
[error] Test org.apache.spark.shuffle.celeborn.SortBasedPusherSuiteJ
failed: java.lang.NoClassDefFoundError: Could not initialize class
org.apache.spark.shuffle.celeborn.SortBasedPusherSuiteJ, took 0.0 sec
[error] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error] ...
```
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Pass GA
Closes #1844 from cfmcgrady/celeborn-906-followup.
Authored-by: Fu Chen <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
---
project/CelebornBuild.scala | 24 ------------------------
1 file changed, 24 deletions(-)
diff --git a/project/CelebornBuild.scala b/project/CelebornBuild.scala
index f63dfc745..0eb8b1988 100644
--- a/project/CelebornBuild.scala
+++ b/project/CelebornBuild.scala
@@ -492,30 +492,6 @@ object Spark34 extends SparkClientProjects {
val sparkVersion = "3.4.1"
val zstdJniVersion = "1.5.2-5"
-
- lazy val deps = Seq(
- // Spark Use `log4j-slf4j2-impl` instead of `log4j-slf4j-impl` in
SPARK-40511
- // to fix the error:
- // ```
- // java.lang.NoSuchMethodError:
org.apache.logging.slf4j.Log4jLoggerFactory.<init>(Lorg/apache/logging/slf4j/Log4jMarkerFactory;)V
- // ```
- "org.apache.logging.log4j" % "log4j-slf4j2-impl" % "2.19.0" % "test"
- )
-
- override def sparkCommon: Project = {
- super.sparkCommon
- .settings(libraryDependencies ++= deps)
- }
-
- override def sparkClient: Project = {
- super.sparkClient
- .settings(libraryDependencies ++= deps)
- }
-
- override def sparkIt: Project = {
- super.sparkIt
- .settings(libraryDependencies ++= deps)
- }
}
trait SparkClientProjects {
