Aaron Davidson created SPARK-1865:
-------------------------------------
Summary: Improve behavior of cleanup of disk state
Key: SPARK-1865
URL: https://issues.apache.org/jira/browse/SPARK-1865
Project: Spark
Issue Type: Improvement
Components: Deploy, Spark Core
Reporter: Aaron Davidson
Right now the behavior of disk cleanup is centered around the exit hook of the
executor, which attempts to cleanup shuffle files and disk manager blocks, but
may fail. We should make this behavior more predictable, perhaps by letting the
Standalone Worker cleanup the disk state, and adding a flag to disable having
the executor cleanup its own state.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
