otterc edited a comment on pull request #32140: URL: https://github.com/apache/spark/pull/32140#issuecomment-854796576
> Took an initial pass, yet to look at `ShuffleBlockFetcherIterator` or test suites. > I am wondering, given the volume, whether we want to split between ESS side and client side. Thoughts ? Thanks Mridul for reviewing! My thoughts about splitting this change is that this PR completely encapsulates the fetch-side changes so it is easier to understand how the new messages introduced on the client side are being handled on the server-side. One of the feedbacks we got last year was that we broke things up in a way that made it difficult to understand. On the server side, this PR mostly adds the wiring needed in `ExternalBlockHandler` to server the merged meta/data requests from `RemoteBlockPushResolver`. That being said, I am still okay to break this change into client/sever PRs if that makes the review easier for the reviewers. cc. @mridulm @Ngone51 @Victsm @tgravescs @attilapiros -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
