HwangDongJun commented on code in PR #58187:
URL: https://github.com/apache/spark/pull/58187#discussion_r4310505265
##########
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesClientUtils.scala:
##########
@@ -57,8 +57,17 @@ object KubernetesClientUtils extends Logging {
@Since("3.1.0")
val configMapNameExecutor: String =
configMapName(s"spark-exec-${KubernetesUtils.uniqueID()}")
+ // SPARK-38079: a `def`, not a `val` -- it is called exactly once per
submission
+ // (Client.run(), into a local val reused for that submission), so unlike
+ // configMapNameExecutor (set up once by KubernetesClusterSchedulerBackend,
then read
+ // independently by BasicExecutorFeatureStep for every executor, so every
call must agree on
+ // the same name), there is no cross-call consistency requirement to
preserve here. A `val`
+ // would instead share this name across every submission from the same JVM
(e.g. via
+ // InProcessLauncher) for the life of the JVM, which, now that this config
map is a
+ // pre-resource, would let a failed submission's cleanup delete-by-name a
different,
+ // still-running submission's live config map of that same name.
@Since("3.1.0")
- val configMapNameDriver: String =
configMapName(s"spark-drv-${KubernetesUtils.uniqueID()}")
+ def configMapNameDriver: String =
configMapName(s"spark-drv-${KubernetesUtils.uniqueID()}")
Review Comment:
Reverted: `KubernetesClientUtils` is unchanged from `master`. With the new
order a failure only deletes the pod, so the cross-submission delete by name
can no longer happen, and the name sharing can be handled separately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
