xuanyuanking commented on issue #24565: [SPARK-27665][Core] Split fetch shuffle blocks protocol from OpenBlocks URL: https://github.com/apache/spark/pull/24565#issuecomment-494031201 Thanks for the advice from all reviewers. I agree with @squito on the suggestion about new spark job talk to old shuffle service, and achieved by newly added config, which is the same idea with @cloud-fan. It has been done in a159f41. Please check. For the versioning, I also think it's necessary to work at this time. Maybe we can add the version in the FetchShuffleBlocks message? Check the version in the server side and choose the different way for decoding multi-version of FetchShuffleBlocks message after Spark 3.0. I think it can be done in a separate PR and JIRA. The added config can achieve the compatibility between <3.0 and 3.0 in this PR. WDYT? @vanzin @attilapiros @cloud-fan @squito.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
