This is an automated email from the ASF dual-hosted git repository.
pan3793 pushed a commit to branch branch-4.2
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.2 by this push:
new 7c86d365d572 [SPARK-57774][CORE] DriverLogger should not propagate
log4j internal exception to caller
7c86d365d572 is described below
commit 7c86d365d5727c521a4542879456d408a5a9ab9e
Author: Cheng Pan <[email protected]>
AuthorDate: Wed Jul 1 10:17:04 2026 +0800
[SPARK-57774][CORE] DriverLogger should not propagate log4j internal
exception to caller
### What changes were proposed in this pull request?
Flip the `setIgnoreExceptions` from false to true in `DriverLogger`
### Why are the changes needed?
https://logging.apache.org/log4j/2.x/plugin-reference.html
> ignoreExceptions - If "true" (default) exceptions encountered when
appending events are logged; otherwise they are propagated to the caller.
The recent log4j 2.25.0 ~ 2.25.3 and 2.26.0 have [known internal
bugs](https://github.com/apache/spark/pull/51719#issuecomment-3341344974) and
may fail the Spark caller, this has been observed from the GHA several times.
One example
https://github.com/apache/spark/actions/runs/27894816608/attempts/1
```
[info] - provider reports error after FS leaves safe mode *** FAILED *** (3
seconds, 25 milliseconds)
[info] org.apache.logging.log4j.core.appender.AppenderLoggingException:
An exception occurred processing Appender _DriverLogAppender
[info] at
org.apache.logging.log4j.core.appender.DefaultErrorHandler.error(DefaultErrorHandler.java:95)
[info] at
org.apache.logging.log4j.core.config.AppenderControl.handleAppenderError(AppenderControl.java:169)
[info] at
org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:162)
[info] at
org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:133)
[info] at
org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:124)
[info] at
org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:88)
[info] at
org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:711)
[info] at
org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:669)
[info] at
org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:645)
[info] at
org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:589)
[info] at
org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:92)
[info] at org.apache.logging.log4j.core.Logger.log(Logger.java:187)
[info] at
org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2970)
[info] at
org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2922)
[info] at
org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2904)
[info] at
org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2648)
[info] at
org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:2587)
[info] at org.apache.logging.slf4j.Log4jLogger.warn(Log4jLogger.java:253)
[info] at org.apache.spark.internal.Logging.logWarning(Logging.scala:299)
[info] at org.apache.spark.internal.Logging.logWarning$(Logging.scala:298)
[info] at
org.apache.spark.SparkFunSuite.logWarning(SparkFunSuite.scala:33)
...
```
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Review and monitor CI.
### Was this patch authored or co-authored using generative AI tooling?
GLM 5.2 is used for issue analysis.
Closes #56890 from pan3793/SPARK-driverlog-ignoreexceptions.
Authored-by: Cheng Pan <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
(cherry picked from commit f5de55e3636d2231926f1be2d1f84fe49ac59ada)
Signed-off-by: Cheng Pan <[email protected]>
---
core/src/main/scala/org/apache/spark/util/logging/DriverLogger.scala | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git
a/core/src/main/scala/org/apache/spark/util/logging/DriverLogger.scala
b/core/src/main/scala/org/apache/spark/util/logging/DriverLogger.scala
index 9854c99707b1..a5206f5282fa 100644
--- a/core/src/main/scala/org/apache/spark/util/logging/DriverLogger.scala
+++ b/core/src/main/scala/org/apache/spark/util/logging/DriverLogger.scala
@@ -71,7 +71,9 @@ private[spark] class DriverLogger(conf: SparkConf) extends
Logging {
builder.setBufferedIo(false)
builder.setConfiguration(config)
builder.withFileName(localLogFile)
- builder.setIgnoreExceptions(false)
+ // SPARK-57774: Set to true to avoid propagating log4j internal exception
+ // (e.g., due to log4j bugs) to the caller.
+ builder.setIgnoreExceptions(true)
builder.setLayout(layout)
builder.setName(DriverLogger.APPENDER_NAME)
builder.build()
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
