[
https://issues.apache.org/jira/browse/SPARK-45182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-45182.
---------------------------------
Fix Version/s: 3.5.1
4.0.0
Resolution: Fixed
Issue resolved by pull request 42950
[https://github.com/apache/spark/pull/42950]
> Ignore task completion from old stage after retrying indeterminate stages
> -------------------------------------------------------------------------
>
> Key: SPARK-45182
> URL: https://issues.apache.org/jira/browse/SPARK-45182
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 3.3.2
> Reporter: Mayur Bhosale
> Assignee: Mayur Bhosale
> Priority: Minor
> Labels: pull-request-available
> Fix For: 3.5.1, 4.0.0
>
>
> SPARK-25342 Added a support for rolling back shuffle map stage so that all
> tasks of the stage can be retried when the stage output is indeterminate.
> This is done by clearing all map outputs at the time of stage submission.
> This approach workouts well except for this case:
> Assume both Shuffle 1 and 2 are indeterminate
> ShuffleMapStage1 –{-}–{-}> Shuffle 1 ---–> ShuffleMapStage2 ----> Shuffle 2
> ----> ResultStage
> * ShuffleMapStage1 is complete
> * A task from ShuffleMapStage2 fails with FetchFailed. Other tasks are still
> running
> * Both ShuffleMapStage1 and ShuffleMapStage2 are retried
> * ShuffleMapStage1 is retried and completes
> * ShuffleMapStage2 reattempt is scheduled for execution
> * Before all tasks of ShuffleMapStage2 reattempt could finish, one/more
> laggard tasks from the original attempt of ShuffleMapStage2 finish and
> ShuffleMapStage2 also gets marked as complete
> * Result Stage gets scheduled and finishes
> Internally within Uber, we have been using the stage rollback functionality
> even for deterministic stages from Spark 2.4.3 to add fault tolerance from
> server going down in [remote shuffle service
> |https://github.com/uber/RemoteShuffleService]and have faced this scenario
> quite often
> Ideally, such laggard tasks should not be considered towards the partition
> completion.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
