Aaron Davidson created SPARK-4277:
-------------------------------------
Summary: Support external shuffle service on Worker
Key: SPARK-4277
URL: https://issues.apache.org/jira/browse/SPARK-4277
Project: Spark
Issue Type: Bug
Components: Deploy, Spark Core
Reporter: Aaron Davidson
Assignee: Aaron Davidson
It's less useful to have an external shuffle service on the Spark Standalone
Worker than on YARN or Mesos (as executor allocations tend to be more static),
but it would be good to help test the code path. It would also make Spark more
resilient to particular executor failures.
Cool side-feature: When SPARK-4236 is fixed and integrated, the Worker will
take care of cleaning up executor directories, which will mean executors
terminate more quickly and that we don't leak data if the executor dies
forcefully.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
