[ https://issues.apache.org/jira/browse/SPARK-24353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stavros Kontopoulos updated SPARK-24353: ---------------------------------------- Description: Spark on K8s allows to place driver/executor pods on specific k8s nodes, using nodeSelector. NodeSelector is a very simple way to constrain pods to nodes with particular labels. The [affinity/anti-affinity|https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity] feature, currently in beta, greatly expands the types of constraints you can express. Aim here is to bring support of this feature to Spark on K8s, in detail: * Node affinity * Pod affinity/anti-affinity Selecting where to run drivers and executors covers many cases like the ones described here: Note that nodeSelector will be deprecated in the future. was: Spark on K8s allows to place driver/executor pods on specific k8s nodes, using nodeSelector. NodeSelector is a very simple way to constrain pods to nodes with particular labels. The [affinity/anti-affinity|https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity] feature, currently in beta, greatly expands the types of constraints you can express. Aim here is to bring support of this feature to Spark on K8s, in detail: * Node affinity * Pod affinity/anti-affinity Note that nodeSelector will be deprecated in the future. > Add support for pod affinity/anti-affinity > ------------------------------------------ > > Key: SPARK-24353 > URL: https://issues.apache.org/jira/browse/SPARK-24353 > Project: Spark > Issue Type: New Feature > Components: Kubernetes > Affects Versions: 2.3.0 > Reporter: Stavros Kontopoulos > Priority: Major > > Spark on K8s allows to place driver/executor pods on specific k8s nodes, > using nodeSelector. NodeSelector is a very simple way to constrain pods to > nodes with particular labels. The > [affinity/anti-affinity|https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity] > feature, currently in beta, greatly expands the types of constraints you can > express. Aim here is to bring support of this feature to Spark on K8s, in > detail: > * Node affinity > * Pod affinity/anti-affinity > Selecting where to run drivers and executors covers many cases like the ones > described here: > Note that nodeSelector will be deprecated in the future. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org