holdenk created SPARK-20624:
-------------------------------
Summary: Consider adding better handling for node shutdown
Key: SPARK-20624
URL: https://issues.apache.org/jira/browse/SPARK-20624
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 2.2.0, 2.30
Reporter: holdenk
Priority: Minor
While we've done some good work with better handling when Spark is choosing to
decommission nodes (SPARK-7955), it might make sense in environments where we
get preempted without our own choice (e.g. YARN over-commit, EC2 spot
instances, GCE Preemptiable instances, etc.) to do something for the data on
the node (or at least not schedule any new tasks).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
