xuanyuanking commented on issue #26040: [SPARK-9853][Core] Optimize shuffle fetch of continuous partition IDs URL: https://github.com/apache/spark/pull/26040#issuecomment-538849477 The difference between this PR and the original one is mainly caused by the following proposals from my side: - Instead of considering merged batch fetch as several blocks, this approach treats the combined batch id as a single block, so that we can not only keep the interfaces of FetchResult with no change in all places, but also delete the duplicated code of merge continuous block id in `BlockManager` and `ExternalShuffleBlockHandler`. - Extend the newly added shuffle fetch protocol by reusing the reduceIds for ShuffleBlockBatchId, it keeps both start and end reduce id for the range [startReduceId, endReduceId).
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
