GitHub user zsxwing opened a pull request:
https://github.com/apache/spark/pull/11399
[SPARK-13519][Core]Driver should tell Executor to stop itself when cleaning
executor's state
## What changes were proposed in this pull request?
When the driver removes an executor's state, the connection between the
driver and the executor may be still alive so that the executor cannot exit
automatically (E.g., Master will send RemoveExecutor when a work is lost but
the executor is still alive), so the driver should try to tell the executor to
stop itself. Otherwise, we will leak an executor.
This PR modified the driver to send `StopExecutor` to the executor when
it's removed.
## How was this patch tested?
manual test: increase the worker heartbeat interval to force it's always
timeout and the leak executors are gone.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zsxwing/spark SPARK-13519
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/11399.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #11399
----
commit 435f020b3749d0dcff2ec19056ebf82526c79c25
Author: Shixiong Zhu <[email protected]>
Date: 2016-02-26T19:47:26Z
Driver should tell Executor to stop itself when cleaning executor's state
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
