[
https://issues.apache.org/jira/browse/SPARK-21301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-21301:
---------------------------------
Labels: bulk-closed (was: )
> Should abort active taskSets or kill all running Tasks when that stage
> success.
> -------------------------------------------------------------------------------
>
> Key: SPARK-21301
> URL: https://issues.apache.org/jira/browse/SPARK-21301
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.0.1
> Reporter: SuYan
> Priority: Minor
> Labels: bulk-closed
>
> I not sure this problem suit for the master branch....and also I may missing
> some logic when I looked into that problem.
> stage 1 -> stage 2-> stage 3
> stage 1.0 done OK
> stage 2.0 , fetch failed, taskset 2.0 was zombie but still has running tasks.
> stage 1.1, done OK
> stage 2.1, running, taskset 2.1 active
> after some time...
> for stage 2, pendingPartition become empty and outputLocs become full,
> but taskset 2.0 and taskset 2.1 was not marked as removed. so 2.1 still
> active, and 2.0 zombie.
> So stage 2 Done OK.
> Stage 3, Fetch failed.
> Resubmit stage 2, taskset 2.2, active conflict with taskset 2.1.
> I had a impression that some jira is about to not resubmit the running tasks
> in previous taskset....? if that jira resolved, this will fixed too....
> and or just do sth to make stage2 related running tasks aborted, to make
> those taskset finished correctly.
> [~irashid][[email protected]]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
