dongjoon-hyun commented on code in PR #39825:
URL: https://github.com/apache/spark/pull/39825#discussion_r1099651597
##########
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala:
##########
@@ -737,6 +737,18 @@ private[spark] object Config extends Logging {
.checkValue(value => value > 0, "Maximum number of pending pods should
be a positive integer")
.createWithDefault(Int.MaxValue)
+ val KUBERNETES_ALLOCATION_BLOCK_ON_SNAPSHOT =
+ ConfigBuilder("spark.kubernetes.allocation.blockOnSnapshot")
+ .doc("By default Spark on Kube will not trigger a new allocation until
Kube delivers a new " +
+ "snapshot this allows an implicit feedback from the cluster manager in
that if it is too " +
+ "busy snapshots may be backed up. Settings this to false increases the
speed at which " +
+ "Spark an scale up but does increase the risk of an excessive number
of pending " +
+ s"resources in some environments. See
${KUBERNETES_MAX_PENDING_PODS.key} " +
Review Comment:
Thank you for mentioning the risks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
