wankunde commented on pull request #34234: URL: https://github.com/apache/spark/pull/34234#issuecomment-948549219
Hi, @Ngone51 @JoeyValentine @mridulm I add a parameter to limit the number of reported shuffle blocks if there are too many huge skewed blocks. I think this is also helpful to limit the memory usage for MapStatus object. Could you help me to review this PR again? Many thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
