[
https://issues.apache.org/jira/browse/SPARK-28879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Franco updated SPARK-28879:
---------------------------
Description:
Similar to SPARK-25220.
Having to create pod templates is not an answer because:
a) It's not available until Spark 3.0 which is unlikely to be deployed to
platforms like EMR for a while as there will be major impacts to dependent
libraries.
b) It is on overkill for something as fundamental as selecting different nodes
for your workers as your drivers e.g. for Spot instances.
Would request that we add this feature in the 2.4 timeframe.
was:
Similar to SPARK-25220.
Having to create pod templates is not an answer because:
a) It's not available until Spark 3.0 which is unlikely to be deployed to
platforms like EMR for a while as there will be major impacts to dependent
libraries.
b) It is on overkill for something as fundamental as selecting different nodes
for your workers as your drivers i.e. for Spot pricing.
Would request that we add this feature in the 2.4 timeframe.
> Kubernetes node selector should be configurable
> -----------------------------------------------
>
> Key: SPARK-28879
> URL: https://issues.apache.org/jira/browse/SPARK-28879
> Project: Spark
> Issue Type: Improvement
> Components: Kubernetes
> Affects Versions: 2.4.3
> Reporter: Franco
> Priority: Major
>
> Similar to SPARK-25220.
>
> Having to create pod templates is not an answer because:
> a) It's not available until Spark 3.0 which is unlikely to be deployed to
> platforms like EMR for a while as there will be major impacts to dependent
> libraries.
> b) It is on overkill for something as fundamental as selecting different
> nodes for your workers as your drivers e.g. for Spot instances.
>
> Would request that we add this feature in the 2.4 timeframe.
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
