mridulm commented on pull request #33329: URL: https://github.com/apache/spark/pull/33329#issuecomment-879652090
@HyukjinKwon, @dongjoon-hyun Fixing the pending jira's would require protocol changes - and shuffle protocol does not gracefully handle incompatible changes. More details [here](https://github.com/apache/spark/pull/33118#issuecomment-870084009) and [here](https://github.com/apache/spark/pull/33118#issuecomment-871135005). I asked @otterc to keep this PR ready just in case we want to take the path of disabling push based shuffle entirely in 3.2.x line in case correctness fixes cant be resolved before GA - we should not be merging this unless we are sure the other two fixes wont make it through, and we want to disable it. Thoughts ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
