pan3793 commented on code in PR #58991:
URL: https://github.com/apache/spark/pull/58991#discussion_r4227841243
##########
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala:
##########
@@ -536,6 +537,30 @@ private[spark] class ApplicationMaster(
}
}
+ /**
+ * Log thread stack traces to help diagnose what the user application thread
(driver)
+ * was blocked on when SparkContext initialization timed out. The user
thread is logged
+ * first since it is usually the most relevant one; a full dump of all
threads follows
+ * for extra context. Failures of the dump itself are swallowed so that they
never
+ * mask the original timeout error.
+ */
+ private def dumpThreadStacksOnScInitTimeout(): Unit = {
+ try {
+ val userThreadDump = Option(userClassThread)
+ .flatMap(t => Utils.getThreadDumpForThread(t.getId))
+ .map(_.toString)
+ .getOrElse("user application thread is not available")
+ logError(log"Stack trace of the user application thread:\n" +
+ log"${MDC(LogKeys.THREAD_DUMP, userThreadDump)}")
+ logError(log"Full thread dump on SparkContext initialization timeout:\n"
+
+ log"${MDC(LogKeys.THREAD_DUMP,
Utils.getThreadDump().map(_.toString).mkString("\n"))}")
Review Comment:
1. this prints `userClassThread` stack trace twice, can you call
`Utils.getThreadDump()` once and sort the output so that `userClassThread`
displays first instead?
2. `LogKeys.THREAD_DUMP` => `LogKeys.THREAD` (see `TaskReaper`)
3. add a config similar to `spark.task.reaper.threadDump`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
