Yikun commented on a change in pull request #35776:
URL: https://github.com/apache/spark/pull/35776#discussion_r822247430
##########
File path:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/VolcanoFeatureStep.scala
##########
@@ -43,19 +44,27 @@ private[spark] class VolcanoFeatureStep extends
KubernetesDriverCustomFeatureCon
}
override def getAdditionalPreKubernetesResources(): Seq[HasMetadata] = {
- val podGroup = new PodGroupBuilder()
- .editOrNewMetadata()
- .withName(podGroupName)
- .withNamespace(namespace)
- .endMetadata()
- .editOrNewSpec()
- .endSpec()
+ val client = new DefaultVolcanoClient
- queue.foreach(podGroup.editOrNewSpec().withQueue(_).endSpec())
+ val template = if (kubernetesConf.isInstanceOf[KubernetesDriverConf]) {
+ kubernetesConf.get(KUBERNETES_DRIVER_PODGROUP_TEMPLATE_FILE)
+ } else {
+ kubernetesConf.get(KUBERNETES_EXECUTOR_PODGROUP_TEMPLATE_FILE)
+ }
+ val pg = template.map(client.podGroups.load(_).get).getOrElse(new
PodGroup())
+ var metadata = pg.getMetadata
+ if (metadata == null) metadata = new ObjectMeta
+ metadata.setName(podGroupName)
+ metadata.setNamespace(namespace)
+ pg.setMetadata(metadata)
-
priorityClassName.foreach(podGroup.editOrNewSpec().withPriorityClassName(_).endSpec())
+ var spec = pg.getSpec
+ if (spec == null) spec = new PodGroupSpec
+ queue.foreach(spec.setQueue(_))
+ priorityClassName.foreach(spec.setPriorityClassName(_))
Review comment:
I have only one left conern in priorityClassName about configuration
overwrite priority.
Compare to pod template this might be a reasonable overwirte sort for me:
- Top 1 configuration value (like queue)
- Top 2 config template value (like driver/executor template)
- Top 3 default value (like priorityClassName which reuse pod priority as
default value)
So do you think we should change it to:
```
// Overwrite if queue configuration specified
queue.foreach(spec.setQueue(_))
// Reuse Pod priority if priority is not set in template
if (spec.getPriorityClassName == null)
priorityClassName.foreach(spec.setPriorityClassName(_))
```
Then users can set `PriorityClassName` by specifying priority in template.
WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
