[
https://issues.apache.org/jira/browse/SPARK-34534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-34534.
---------------------------------
Fix Version/s: 3.1.2
3.0.3
3.2.0
Resolution: Fixed
Issue resolved by pull request 31643
[https://github.com/apache/spark/pull/31643]
> New protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or
> correctness
> -----------------------------------------------------------------------------------------
>
> Key: SPARK-34534
> URL: https://issues.apache.org/jira/browse/SPARK-34534
> Project: Spark
> Issue Type: Bug
> Components: Shuffle
> Affects Versions: 3.0.0, 3.0.1, 3.0.2
> Reporter: haiyangyu
> Priority: Major
> Labels: Correctness, data-loss
> Fix For: 3.2.0, 3.0.3, 3.1.2
>
> Attachments: image-2021-02-25-11-17-12-714.png,
> image-2021-02-25-11-27-34-429.png, image-2021-02-25-11-28-31-255.png,
> image-2021-02-25-11-30-03-834.png, image-2021-02-25-11-31-59-110.png
>
>
> We will build a new rpc message `FetchShuffleBlocks` when
> `OneForOneBlockFetcher` init in replace of `OpenBlocks` to use adaptive
> feature, this introduce additional problems as follows.
> `OneForOneBlockFetcher` will init a `blockIds` String array to catch chunk
> fetch success, it will use index in `blockIds` to fetch blocks and match
> blockId in `blockIds` when chunk data return. So the `blockIds` 's order must
> be consistent with fetchChunk index, but the new `FetchShuffleBlocks` return
> chunk order is not same as `blockIds`.
> This will lead to the return data not match the blockId, and this can lead
> to data corretness when retry to fetch after fetch block chunk failed.
> Fetch chunk orker code and match blockId when rerun data code as follows:
> !image-2021-02-25-11-27-34-429.png|width=446,height=251!!image-2021-02-25-11-28-31-255.png|width=445,height=159!
> Howerver, the fetch order in shuffle service,
> !image-2021-02-25-11-30-03-834.png|width=510,height=361!
> So, it will fetch some wrong block data when chunk fetch failed beause the
> blocks's wrong order.
> !image-2021-02-25-11-31-59-110.png|width=601,height=204!
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
