AngersZhuuuu opened a new pull request #27419: [SPARK-30694][SHUFFLE]If exception occured while fetching blocks by ExternalBlockClient, fail early when External Shuffle Service is not alive URL: https://github.com/apache/spark/pull/27419 ### What changes were proposed in this pull request? Like [SPARK-27637](https://github.com/apache/spark/pull/25469), When we fetch data use External Shuffle Service, we use ExternalBlockStoreClient to fetch blocks from External Shuffle Service, in real huge cluster, we always need to offline some bad machine, and remote External Shuffle Service will down and we will can't create TransportClient to connect. It will throw IOException, but in current way, we will only retry until run out of retry times. I think we should throw this error early like [SPARK-27637](https://github.com/apache/spark/pull/25469) ### Why are the changes needed? Old solution not comprehensive. Throw exception early and reduce unnecessary retry. ### Does this PR introduce any user-facing change? NO ### How was this patch tested? WIP
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
