dongjoon-hyun commented on pull request #30891: URL: https://github.com/apache/spark/pull/30891#issuecomment-755461182
Hi, @xkrogen . We had multiple profiles at that age and your concern is still valid. However, for a long time, Apache Spark community has been rejected multiple PRs which aims to add a new profile and insists to keep only two profiles (2.x and 3.x). Due to the maintenance burdens, Apache Spark community only focuses on two profiles and it will be the same in the future. > In the past we've simultaneously had profiles for multiple Hadoop versions, such as 2.6 and 2.7. At one point we even had 5 different profiles from the 2.X line. Unless our confidence is very high that we would never need multiple Hadoop profiles from the 3.X line, I'm against this change. I am concerned it will cause us to unnecessarily only support a single 3.X line when multiple simultaneous profiles could allow a broader range of support. I'm supporting this PR. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
