igreenfield commented on a change in pull request #26624:
URL: https://github.com/apache/spark/pull/26624#discussion_r417901008
##########
File path: core/src/main/scala/org/apache/spark/executor/Executor.scala
##########
@@ -674,6 +677,18 @@ private[spark] class Executor(
}
}
+ private def setMDCForTask(taskDescription: TaskDescription): Unit = {
+ val properties = taskDescription.properties
+
+ org.slf4j.MDC.put("appId", properties.getProperty("spark.app.id"))
+ org.slf4j.MDC.put("appName", properties.getProperty("spark.app.name"))
+
+ properties.asScala.filter(_._1.startsWith("mdc.")).foreach { item =>
+ val key = item._1.substring(4)
+ org.slf4j.MDC.put(key, item._2)
Review comment:
1. I think where possible it is better to use abstract layer - at least
in new code.
2. about ThreadUtils without all that change the MDC will not propagate to
all places only to the scope of that thread and from my tests, it is not
enough, we are using that code internally for more than a year.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
