[
https://issues.apache.org/jira/browse/SPARK-13054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Or resolved SPARK-13054.
-------------------------------
Resolution: Fixed
Assignee: Thomas Graves (was: Andrew Or)
Fix Version/s: 2.0.0
> Always post TaskEnd event for tasks in cancelled stages
> -------------------------------------------------------
>
> Key: SPARK-13054
> URL: https://issues.apache.org/jira/browse/SPARK-13054
> Project: Spark
> Issue Type: Bug
> Components: Scheduler
> Affects Versions: 1.0.0
> Reporter: Andrew Or
> Assignee: Thomas Graves
> Fix For: 2.0.0
>
>
> {code}
> // The success case is dealt with separately below.
> // TODO: Why post it only for failed tasks in cancelled stages? Clarify
> semantics here.
> if (event.reason != Success) {
> val attemptId = task.stageAttemptId
> listenerBus.post(SparkListenerTaskEnd(
> stageId, attemptId, taskType, event.reason, event.taskInfo,
> taskMetrics))
> }
> {code}
> Today we only post task end events for canceled stages if the task failed.
> There is no reason why we shouldn't just post it for all the tasks, including
> the ones that succeeded. If we do that we will be able to simplify another
> branch in the DAGScheduler, which needs a lot of simplification.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
