tgravescs commented on issue #23510: [SPARK-26590][CORE] make fetch-block-to-disk backward compatible URL: https://github.com/apache/spark/pull/23510#issuecomment-454412006 What do you mean by "we can just simply turn on this feature in Spark 3.0"? I don't see where this is a feature that only applies to 3.0, at least according to description. I agree it's better to be backwards compatible in general but people should be upgrading as there are other possible issues in older versions - correctness issue, security issues, etc. I realize some people don't or can't upgrade quickly though as well. If it's a general improvement then sounds ok, especially if it gives us more flexibility in the future. Do you have other specific changes in mind that will use this, like the work going on with off node shuffle service? Perhaps we should change the description to emphasize that rather then backwards compatibility with versions we no longer maintain
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
