[
https://issues.apache.org/jira/browse/SPARK-44646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750891#comment-17750891
]
Yu Tian commented on SPARK-44646:
---------------------------------
Hi [~viirya]
Could you please check this thread? It is a question related to
https://issues.apache.org/jira/browse/SPARK-37814
Thanks.
> Migrate Log4j 2.x in Spark 3.4.1 to Logback
> -------------------------------------------
>
> Key: SPARK-44646
> URL: https://issues.apache.org/jira/browse/SPARK-44646
> Project: Spark
> Issue Type: Brainstorming
> Components: Build
> Affects Versions: 3.4.1
> Reporter: Yu Tian
> Priority: Major
>
> Hi,
> We are working on the spark 3.4.1 upgrade from spark 3.1.3, in our logging
> system, we are using logback framework, it is working with spark 3.1.3 since
> it is using log4j 1.x. However, when we upgrade spark to 3.4.1, based on the
> [release
> notes|https://spark.apache.org/docs/latest/core-migration-guide.html], spark
> is migrating from log4j 2.x from log4j 1.x, the way we are replacing the
> log4j with logback is causing build failures in spark master start process.
> Error: Unable to initialize main class org.apache.spark.deploy.master.Master
> Caused by: java.lang.NoClassDefFoundError:
> org/apache/logging/log4j/core/Filter
> In our current approach, we are using log4j-over-slf4j to replace the
> log4j-core, it is only applicable to log4j 1.x library. And there is no
> log4j-over-slf4j for log4j 2.x out there yet. (please correct me if I am
> wrong).
> I am also curious that why spark choose to use log4j 2.x instead of using
> SPI, which gives the users less flexibility to choose whatever logger
> implementation they want to use.
> I want to share this issue and see if anyone else has been reported this and
> if there is any work-around or alternative solutions for it. Any suggestions
> are appreciated, thanks.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
