pan3793 commented on code in PR #58991:
URL: https://github.com/apache/spark/pull/58991#discussion_r4230277161
##########
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/config/package.scala:
##########
@@ -251,6 +251,16 @@ package object config extends Logging {
.timeConf(TimeUnit.MILLISECONDS)
.createWithDefaultString("100s")
+ private[spark] val AM_SC_INIT_TIMEOUT_THREAD_DUMP =
+ ConfigBuilder("spark.yarn.am.scInitTimeout.threadDump")
Review Comment:
`sc` might be confusing, let's use the full name: `SparkContext`
```suggestion
ConfigBuilder("spark.yarn.am.sparkContextInitTimeout.threadDump")
```
as a reference, there is an `spark.executor.allowSparkContext`
##########
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/config/package.scala:
##########
@@ -251,6 +251,16 @@ package object config extends Logging {
.timeConf(TimeUnit.MILLISECONDS)
.createWithDefaultString("100s")
+ private[spark] val AM_SC_INIT_TIMEOUT_THREAD_DUMP =
+ ConfigBuilder("spark.yarn.am.scInitTimeout.threadDump")
+ .doc("Whether to dump all thread stacks when SparkContext initialization
times out " +
+ "(see spark.yarn.am.waitTime). The dump is logged at error level
before the " +
+ "application fails, with the user application thread sorted to the
front, to help " +
+ "diagnose what the driver thread was blocked on.")
Review Comment:
implementation details should not leak to the user docs.
```suggestion
.doc("Whether to print all thread stacks when SparkContext
initialization times out " +
"(see `${AM_MAX_WAIT_TIME.key}`).")
```
##########
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/config/package.scala:
##########
@@ -251,6 +251,16 @@ package object config extends Logging {
.timeConf(TimeUnit.MILLISECONDS)
.createWithDefaultString("100s")
+ private[spark] val AM_SC_INIT_TIMEOUT_THREAD_DUMP =
+ ConfigBuilder("spark.yarn.am.scInitTimeout.threadDump")
+ .doc("Whether to dump all thread stacks when SparkContext initialization
times out " +
+ "(see spark.yarn.am.waitTime). The dump is logged at error level
before the " +
+ "application fails, with the user application thread sorted to the
front, to help " +
+ "diagnose what the driver thread was blocked on.")
+ .version("5.0.0")
Review Comment:
`master` branch goes to `5.0.0-SNAPSHOT`, but the next version will be cut
from `branch-4.x`, which is currently `4.4.0-SNAPSHOT`
```suggestion
.version("4.4.0")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
