Henry Yu created SPARK-28022:
--------------------------------
Summary: k8s pod affinity achieve cloud native friendly auto
scaling
Key: SPARK-28022
URL: https://issues.apache.org/jira/browse/SPARK-28022
Project: Spark
Issue Type: New Feature
Components: Kubernetes
Affects Versions: 2.4.3
Reporter: Henry Yu
Hi, in order to achieve friendly auto scaling , I propose to add a pod affinity
feature.
Traditionally, when we use spark in fix size yarn cluster, it make sense to
spread container evenly to every node.
But come to cloud native resource manage, we want to release node when we don't
need any more.
pod affinity feature can help when we try to place all pod of certain job to
some node instead of all node.
we can not use pod template to achieve this, for we must add application id to
affinity .
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
