Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/10951#issuecomment-194525049
To add some more detail, the executorLost function only calls taskEnded
(which is task failed) if the task was in the list of successful tasks.
handleSuccessfulTask calls the taskEnded and then adds it to the list of
successful tasks. since taskEnded ends up sending the CompletionEvent it is
possible for the events to show up in either order.
But the DAGScheduler.handleTaskCompletion before this PR handled both of
those already and sent SparkListenerTaskEnd in both cases. This PR doesn't
change that behavior for those 2 events.
The messages are success and resubmitted (which is failed). Both of which
DAGScheduler.handleTaskCompletion would send the SparkListenerTaskEnd event for
both with this PR and before this PR.
If there is anything else you think I missed let me know.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
