GitHub user tsdeng opened a pull request:
https://github.com/apache/incubator-spark/pull/596
add onExecutorsStopped to event listeners for getting final stats
add onExecutorStopped event handler.
So when master command a shutdown, the executors can send the final message
reporting its statuses by replying ExecutorFinalState message(currently only
hdfsRead is reported, more can be added later).
And the master will wait for all response to comeback within the timeout
window.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/apache/incubator-spark on_executors_stopped
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-spark/pull/596.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #596
----
commit 1e6c79e60c969d6af6ca226a0c58e00efeab30bb
Author: Tianshuo Deng <td...@twitter.com>
Date: 2014-01-28T23:51:52Z
add onExecutorsStopped to event listeners for getting final stats from
executors when command a shutting down
----
