mridulm commented on PR #2373: URL: https://github.com/apache/celeborn/pull/2373#issuecomment-2076295830
QQ: thinking out aloud, instead of this change - do we want to proactively trigger sort for reducers where we are reading a subset of mapper output (based on `ShufflePartitionsUtil`) ? This will help if we are trying to mitigate the impact of reducer read timeouts, etc. It wont bring down the overall load though. On plus side, it does not suffer from the correctness issues here. Thoughts ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
