xkrogen commented on pull request #30891: URL: https://github.com/apache/spark/pull/30891#issuecomment-755437658
In the past we've simultaneously had profiles for multiple Hadoop versions, such as 2.6 and 2.7. At one point we even had 5 different profiles from the 2.X line. Unless our confidence is very high that we would never need multiple Hadoop profiles from the 3.X line, I'm against this change. I am concerned it will cause us to unnecessarily only support a single 3.X line when multiple simultaneous profiles could allow a broader range of support. @sunchao do you have any thoughts on why things might be different now as opposed to the history of Spark when many Hadoop 2.X profiles were supported? Faster release cadence of Hadoop, stronger compatibility guarantees, etc.? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
