gengliangwang commented on code in PR #47732:
URL: https://github.com/apache/spark/pull/47732#discussion_r1714639276
##########
common/utils/src/test/scala/org/apache/spark/util/StructuredLoggingSuite.scala:
##########
@@ -438,6 +438,44 @@ class StructuredLoggingSuite extends LoggingSuiteBase {
assert((log"\r" + log"\n" + log"\t" + log"\b").message == "\r\n\t\b")
assert((log"\r${MDC(LogKeys.EXECUTOR_ID, 1)}\n".message == "\r1\n"))
}
+
+ test("disabled structured logging won't log context") {
+ Logging.disableStructuredLogging()
+ val expectedPatternWithoutContext = compactAndToRegexPattern(
+ s"""
+ {
+ "ts": "<timestamp>",
+ "level": "INFO",
+ "msg": "Lost executor 1.",
+ "logger": "$className"
+ }""")
+
+ Seq(
+ () => logInfo(log"Lost executor ${MDC(LogKeys.EXECUTOR_ID, "1")}."),
+ () => logInfo( // blocked when explicitly constructing the
MessageWithContext
+ MessageWithContext(
+ "Lost executor 1.",
+ new java.util.HashMap[String, String] {
put(LogKeys.EXECUTOR_ID.name, "1") }
Review Comment:
```
private[spark] val STRUCTURED_LOGGING_ENABLED =
ConfigBuilder("spark.log.structuredLogging.enabled")
.doc("When true, Spark logs are output as structured JSON lines with
added Spark " +
"Mapped Diagnostic Context (MDC), facilitating easier integration
with log aggregation " +
"and analysis tools. When false, logs are plain text without MDC.
This configuration " +
"does not apply to interactive environments such as spark-shell,
spark-sql, and " +
"PySpark shell.")
.version("4.0.0")
.booleanConf
.createWithDefault(true)
```
Disabling the structured logging will disable the MDCs. This is on purpose
to avoid regression on setting these contexts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
