agrawaldevesh commented on a change in pull request #29466:
URL: https://github.com/apache/spark/pull/29466#discussion_r472297949
##########
File path: core/src/main/scala/org/apache/spark/internal/config/package.scala
##########
@@ -1866,6 +1866,19 @@ package object config {
.timeConf(TimeUnit.MILLISECONDS)
.createOptional
+ private[spark] val DECOMMISSION_ENABLED =
+ ConfigBuilder("spark.decommission.enabled")
+ .doc("When decommission enabled, Spark will try its best to shutdown the
executor " +
+ s"gracefully. Spark will try to migrate all the RDD blocks (controlled
by " +
+ s"${STORAGE_DECOMMISSION_RDD_BLOCKS_ENABLED.key}) and shuffle blocks
(controlled by " +
+ s"${STORAGE_DECOMMISSION_SHUFFLE_BLOCKS_ENABLED.key}) from the
decommissioning " +
Review comment:
nit: some 2 spaces here ... before from
##########
File path: core/src/main/scala/org/apache/spark/internal/config/package.scala
##########
@@ -1866,6 +1866,19 @@ package object config {
.timeConf(TimeUnit.MILLISECONDS)
.createOptional
+ private[spark] val DECOMMISSION_ENABLED =
+ ConfigBuilder("spark.decommission.enabled")
+ .doc("When decommission enabled, Spark will try its best to shutdown the
executor " +
+ s"gracefully. Spark will try to migrate all the RDD blocks (controlled
by " +
+ s"${STORAGE_DECOMMISSION_RDD_BLOCKS_ENABLED.key}) and shuffle blocks
(controlled by " +
+ s"${STORAGE_DECOMMISSION_SHUFFLE_BLOCKS_ENABLED.key}) from the
decommissioning " +
+ s"executor to a remote executor when
${STORAGE_DECOMMISSION_ENABLED.key} is enabled. " +
+ s"With decommission enabled, Spark will also decommission an executor
instead of " +
+ s"killing to get better scalable when ${DYN_ALLOCATION_ENABLED.key}
enabled.")
Review comment:
nit: instead of killing when ...
(remove the "to get better scaleable")
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
