HwangDongJun commented on code in PR #58187:
URL: https://github.com/apache/spark/pull/58187#discussion_r4163449095
##########
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/PodTemplateConfigMapStep.scala:
##########
@@ -74,7 +74,9 @@ private[spark] class PodTemplateConfigMapStep(conf:
KubernetesConf)
}
}
- override def getAdditionalKubernetesResources(): Seq[HasMetadata] = {
+ // SPARK-38079: this config map is mounted by the driver pod, so it must be
created as a
+ // pre-resource (before the pod itself) to avoid a "configmap ... not found"
mount race.
+ override def getAdditionalPreKubernetesResources(): Seq[HasMetadata] = {
Review Comment:
You're right that this is a real gap, but it lives in the operator's own
orchestration code, not in anything `Client.run()` controls.
`KubernetesDriverBuilder`'s output (the pre-resource list) hasn't changed
shape — the operator's `AppInitStep`/`AppCleanUpStep` build their own
sequencing around it, independently of `Client.run()`. This PR's fix (creating
the pre-resources with their owner reference already set, in a single CREATE
gated by `spec.schedulingGates` — see [the owner-reference reply
above](https://github.com/apache/spark/pull/58187#discussion_r4129910475)) only
applies to the path that goes through `Client.run()`. The operator doesn't go
through that path, so it isn't covered by this change.
I don't think this is something I can fix from the Spark repo side — the
operator would need to adopt the same
create-pod-first-then-apply-pre-resources-with-owner-reference sequencing
itself. I'll call this out as a known limitation in the PR description rather
than claiming it's addressed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
