cloud-fan commented on issue #23510: [SPARK-26590][CORE] make fetch-block-to-disk backward compatible URL: https://github.com/apache/spark/pull/23510#issuecomment-454273051 > According to our docs we no longer maintain anything < 2.2 so why are we adding this to support? Yes you are right, we can just simply turn on this feature in Spark 3.0. But in practice, it's always better if we can be more backward compatible without overhead, even for versions that we don't maintain anymore, as users may still use it. Another reason is, the previous approach is really hacky. It makes `OneForOneStreamManager` and `NettyStreamManager` have totally different semantic about `openStream`, which makes a lot of assumption about the shuffle implementation, and is pretty fragile and confusing. This PR allows us to remove `OneForOneStreamManager.openStream` in the future.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
