[
https://issues.apache.org/jira/browse/SPARK-54956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tengfei Huang updated SPARK-54956:
----------------------------------
Description:
Currently, we have two version of implementations to deal with indeterminate
shuffle retry.
The shuffle output checksum based solution and the legacy one.
Ideally, the major difference is how we detect the indeterminate shuffle retry,
after that the solution to deal with it should be the same.
See more details:
[https://github.com/apache/spark/pull/53274#discussion_r2610307715]
was:
Currently, we have two version of implementations to deal with indeterminate
shuffle retry.
The shuffle output checksum based solution and the legacy one.
Ideally, the major difference is how we detect the indeterminate shuffle retry,
after that the solution to deal with it should be the same.
> Unify the indeterminate shuffle retry solutions
> -----------------------------------------------
>
> Key: SPARK-54956
> URL: https://issues.apache.org/jira/browse/SPARK-54956
> Project: Spark
> Issue Type: Task
> Components: Spark Core
> Affects Versions: 4.1.0
> Reporter: Tengfei Huang
> Priority: Major
>
> Currently, we have two version of implementations to deal with indeterminate
> shuffle retry.
> The shuffle output checksum based solution and the legacy one.
> Ideally, the major difference is how we detect the indeterminate shuffle
> retry, after that the solution to deal with it should be the same.
> See more details:
> [https://github.com/apache/spark/pull/53274#discussion_r2610307715]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
