GitHub user sadhen reopened a pull request:
https://github.com/apache/spark/pull/21166
[SPARK-11334][CORE] clear idle executors in executorIdToTaskIds keySet
## What changes were proposed in this pull request?
quote from #11205
> Executors may never be idle. Currently we use the executor to tasks
mapping relation to identify the status of executors, in maximum task failure
situation, some TaskEnd events may never be delivered, which makes the related
executor always be busy.
#19580 fix the incorrect number of running tasks.
This PR solves the problem in a similar way on the `fact` that TaskEnd
events may never be delivered but StageCompleted events would be delivered
eventually.
## How was this patch tested?
Existing tests
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sadhen/spark SPARK-11334
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21166.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21166
----
commit 3f32a92352551edf79aca2c508ee575d7e3f2aa1
Author: å¿å¬ <rendong@...>
Date: 2018-04-26T10:46:32Z
clear idle executors in executorIdToTaskIds keySet
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
