HwangDongJun commented on code in PR #58187:
URL: https://github.com/apache/spark/pull/58187#discussion_r4310547100
##########
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesClientApplication.scala:
##########
@@ -131,48 +147,80 @@ private[spark] class Client(
.withName(configMapName)
.endConfigMap()
.endVolume()
+ .addToNodeSelector(
+ pinnedNodeName.map(nodeName => Map("kubernetes.io/hostname" ->
nodeName))
+ .getOrElse(Map.empty[String, String]).asJava)
+ // SPARK-38079: holds the pod unschedulable -- so kubelet cannot
attempt to mount
+ // anything on it -- until its pre-resources exist (see below). Only
removed once
+ // that is true, right before the watch loop; see the "Remove the
pre-resources
+ // scheduling gate" step below for why this makes an
ownerless-resource window,
+ // and its consequences (a shutdown hook), unnecessary.
+ .addNewSchedulingGate(PRE_RESOURCES_SCHEDULING_GATE)
Review Comment:
Confirmed and documented. On Apache YuniKorn 1.6.3 a gated pod got
`PodBindFailure ... has non-empty .spec.schedulingGates`, a `TaskFailed` event,
and stayed `Pending` after the gate was removed; on 1.7.0 (`YUNIKORN-3031`) it
waited for the gate and was bound as soon as the gate was removed.
`docs/running-on-kubernetes.md` says a scheduler that does not honor scheduling
gates may leave the pod pending, names YuniKorn 1.7.0, and points to
`spark.kubernetes.submission.useDriverSchedulingGate`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
