dongjoon-hyun commented on a change in pull request #29846:
URL: https://github.com/apache/spark/pull/29846#discussion_r493881157
##########
File path:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/MountVolumesFeatureStep.scala
##########
@@ -84,4 +112,15 @@ private[spark] class MountVolumesFeatureStep(conf:
KubernetesConf)
(volumeMount, volume)
}
}
+
+ override def getAdditionalKubernetesResources(): Seq[HasMetadata] = {
+ additionalResources
+ }
+}
+
+private[spark] object MountVolumesFeatureStep {
+ val PVC_ON_DEMAND = "OnDemand"
+ val PVC = "PersistentVolumeClaim"
+ val PVC_POSTFIX = "-pvc"
+ val PVC_ACCESS_MODE = "ReadWriteOnce"
Review comment:
Ya, it's possible, but I didn't do that in this PR because of the
followings.
- `PVC_ON_DEMAND`: It's a dummy placeholder because the existing code
expects some pre-defined names. We had better recommend a fixed one instead of
making a configurable one in this case.
- `PVC_POSTFIX`: It can be configurable but doesn't give much benefit to
users because this is a part of transient ids and the prefix already guarantees
no conflicts.
- `PVC_ACCESS_MODE`: Ya. I thought like you at the beginning, but I changed
to this form. Although this makes sense a lot, I leave this PR to focus on a
fixed one because this PR aims to generate a new PVC for each executor. In
other words, this PR is not suggesting creating a `ReadWriteMany` PVC and
sharing across in multiple executors.
For `ReadWriteMany` PVC, we don't need to use this PR. The existing Spark
PVC feature can mount a single `ReadWriteMany` PVC into all executors without
any problem and there is no burden to maintain `ReadWriteMany` PVC, because
it's always a single one. In addition, we also support NFS (AWS EFS) mounting,
too.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
