[
https://issues.apache.org/jira/browse/SPARK-27334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
TommyLike updated SPARK-27334:
------------------------------
Description: Currently, there are some external schedulers which bring a
lot a great value into kubernetes scheduling especially for HPC case, take a
look at the *kube-batch* ([https://github.com/kubernetes-sigs/kube-batch]). In
order to support it, we had to use Pod Template which seems cumbersome. It
would be much convenient if this can be configured via option such as
*"spark.kubernetes.executor.schedulerName"* just like others. (was: Now, there
are some external schedulers which bring a lot a great value into kubernetes
scheduling especially for HPC case, take a look at the [link
kube-batch|[https://github.com/kubernetes-sigs/kube-batch]]. Currently, we had
to use Pod Template to support this which seems cumbersome. It would be much
convenient if this can be configured via option such as
*"spark.kubernetes.executor.schedulerName"* just like others.)
> Support specify scheduler name for executor pods when submit
> ------------------------------------------------------------
>
> Key: SPARK-27334
> URL: https://issues.apache.org/jira/browse/SPARK-27334
> Project: Spark
> Issue Type: Improvement
> Components: Kubernetes
> Affects Versions: 2.4.0
> Reporter: TommyLike
> Priority: Major
> Labels: easyfix, features
>
> Currently, there are some external schedulers which bring a lot a great value
> into kubernetes scheduling especially for HPC case, take a look at the
> *kube-batch* ([https://github.com/kubernetes-sigs/kube-batch]). In order to
> support it, we had to use Pod Template which seems cumbersome. It would be
> much convenient if this can be configured via option such as
> *"spark.kubernetes.executor.schedulerName"* just like others.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
