[
https://issues.apache.org/jira/browse/SPARK-36855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jinhai updated SPARK-36855:
---------------------------
Description: In the ShuffleBlockFetcherIterator.partitionBlocksByFetchMode
method, both local and host-local execute the
mergeContinuousShuffleBlockIdsIfNeeded method first, but remote blocks executes
the method many times in the createFetchRequests method. Can the method of
merge blocks be executed only once in advance? (was: In the
ShuffleBlockFetcherIterator.partitionBlocksByFetchMode method, both local and
host-local execute the mergeContinuousShuffleBlockIdsIfNeeded method first, but
remote blocks executes the method many times in the createFetchRequests method.
Can merge blocks be executed only once in advance?)
> Uniformly execute the mergeContinuousShuffleBlockIdsIfNeeded method first
> -------------------------------------------------------------------------
>
> Key: SPARK-36855
> URL: https://issues.apache.org/jira/browse/SPARK-36855
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 3.1.0, 3.1.1, 3.1.2
> Reporter: jinhai
> Priority: Major
>
> In the ShuffleBlockFetcherIterator.partitionBlocksByFetchMode method, both
> local and host-local execute the mergeContinuousShuffleBlockIdsIfNeeded
> method first, but remote blocks executes the method many times in the
> createFetchRequests method. Can the method of merge blocks be executed only
> once in advance?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
