GitHub user jweaver-personal opened a pull request:
https://github.com/apache/spark/pull/22212
[SPARK-25220] Seperate kubernetes node selector config between driver and
executors.
Seperated node selector config option between executors and driver.
This removes the spark.kubernetes.node.selector config option and seperates
it to
spark.kubernetes.driver.selector and spark.kubernetes.executor.selector
to allow seperate node selectors on drivers and executors.
My personal use case for this change is that on AWS we have cheap
spotinstances that can terminate at any moment which is okay for executors but
not the driver.
With a single node selector option I am unable to use the spot instances to
save costs on the executor nodes.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jweaver-personal/spark jweaver-dev
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22212.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22212
----
commit a6c6371f8d0ba5e7ff55e83f7c8c955bf24096aa
Author: ? <jonathan.weaver@...>
Date: 2018-08-24T01:55:06Z
KUBERNETES K8S
Seperated node selector config option between executors and driver.
This removes the spark.kubernetes.node.selector config option and seperates
it to
spark.kubernetes.driver.selector and spark.kubernetes.executor.selector
to allow seperate node selectors on drivers and executors.
My personal use case for this change is that on AWS we have cheap
spotinstances that can terminate at any moment which is okay for executors but
not the driver.
With a single node selector option I am unable to use the spot instances to
save costs on the executor nodes.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
