Aaron Davidson created SPARK-4236:
-------------------------------------
Summary: External shuffle service must cleanup its shuffle files
Key: SPARK-4236
URL: https://issues.apache.org/jira/browse/SPARK-4236
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 1.2.0
Reporter: Aaron Davidson
Priority: Critical
When external shuffle service is enabled, executors no longer cleanup their own
shuffle files, as the external server can continue to serve them. The external
serer must clean these files up once it knows that they will never be used
again (i.e., when the application terminates or when Spark's ContextCleaner
requests that they be deleted).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
