[
https://issues.apache.org/jira/browse/SPARK-19565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jin xing updated SPARK-19565:
-----------------------------
Description:
This is related to SPARK-19263.
When fetch failed, stage will be resubmitted. There can be running tasks from
both old and new stage attempts. Success of tasks from old stage attempt should
be taken as valid and partitionId should be removed from stage's
pendingPartitions accordingly. When pending partitions is empty, downstream
stage can be scheduled, even though there's still running tasks in the
active(new) stage attempt.
> After fetching failed, success of old attempt of stage should be taken as
> valid.
> --------------------------------------------------------------------------------
>
> Key: SPARK-19565
> URL: https://issues.apache.org/jira/browse/SPARK-19565
> Project: Spark
> Issue Type: Test
> Components: Scheduler
> Affects Versions: 2.1.0
> Reporter: jin xing
>
> This is related to SPARK-19263.
> When fetch failed, stage will be resubmitted. There can be running tasks from
> both old and new stage attempts. Success of tasks from old stage attempt
> should be taken as valid and partitionId should be removed from stage's
> pendingPartitions accordingly. When pending partitions is empty, downstream
> stage can be scheduled, even though there's still running tasks in the
> active(new) stage attempt.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
