GitHub user zsxwing opened a pull request:
https://github.com/apache/spark/pull/11401
[SPARK-13522][Core]Executor should kill itself when it's unable to
heartbeat to driver more than N times
## What changes were proposed in this pull request?
Sometimes, network disconnection event won't be triggered for other
potential race conditions that we may not have thought of, then the executor
will keep sending heartbeats to driver and won't exit.
This PR adds a new configuration `spark.executor.heartbeat.maxFailures` to
kill Executor when it's unable to heartbeat to the driver more than
`spark.executor.heartbeat.maxFailures` times.
## How was this patch tested?
unit tests
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zsxwing/spark SPARK-13522
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/11401.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #11401
----
commit e8ad9dd729d287623fa201425a833779c5cd6e9c
Author: Shixiong Zhu <[email protected]>
Date: 2016-02-26T20:54:49Z
Executor should kill itself when it's unable to heartbeat to driver more
than N times
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
