attilapiros commented on a change in pull request #33270:
URL: https://github.com/apache/spark/pull/33270#discussion_r670908748
##########
File path:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
##########
@@ -181,7 +184,51 @@ private[spark] class KubernetesClusterSchedulerBackend(
super.getExecutorIds()
}
+ private def labelLowPriorityExecs(execIds: Seq[String]) = {
+ // Only kick off the labeling task if we have a label.
+ conf.get(KUBERNETES_EXECUTOR_POD_DECOMMISSION_LABEL).foreach { label =>
+ val labelTask = new Runnable() {
+ override def run(): Unit = Utils.tryLogNonFatalError {
+
+ val podsToLabel = kubernetesClient.pods()
+ .withLabel(SPARK_APP_ID_LABEL, applicationId())
+ .withLabel(SPARK_ROLE_LABEL, SPARK_POD_EXECUTOR_ROLE)
+ .withLabelIn(SPARK_EXECUTOR_ID_LABEL, execIds: _*)
+ .list().getItems().asScala
+
+ podsToLabel.foreach { pod =>
+ kubernetesClient.pods()
+ .inNamespace(pod.getMetadata.getNamespace)
+ .withName(pod.getMetadata.getName)
+ .edit({p: Pod => new PodBuilder(p).editMetadata()
+ .addToLabels(label,
conf.get(KUBERNETES_EXECUTOR_POD_DECOMMISSION_LABEL_VALUE))
+ .endMetadata()
+ .build()})
Review comment:
I woke up during the night and looked into this a bit more and now I see
I was wrong with the suggestion and why.
The `new PodBuilder()... build()` will create a Pod model object (with the
new metadata) but the changes should be communicated back to k8s API server for
example via a PATCH request like it was done at
https://github.com/apache/spark/blob/5f41a2752fba617524c072ee4f0e3a4e5368be13/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsLifecycleManager.scala#L178-L187
I like the ".edit(...)" method a bit better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
