rmcyang commented on pull request #34158: URL: https://github.com/apache/spark/pull/34158#issuecomment-932612606
Thanks for thorough review @JoshRosen @mridulm. The original change indeed was not really to address the actual problem after I did more tests on our internal branch. To better fix the issue, I further made the val lazy wherever `isPushBasedShuffleEnabled` is invoked when it is a class instance variable so it will happen after the jars/classes are available/downloaded. For the case when `isPushBasedShuffleEnabled` is called during executor startup, the idea is we don't check if serializer if relocatable of not by using a flag `checkSerializer`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
