HeartSaVioR commented on PR #49983: URL: https://github.com/apache/spark/pull/49983#issuecomment-2680925857
@cloud-fan > have we merged this graceful deprecation in branch 3.5? Yes, that is merged. It's still a blocker for Spark 4.0.0 though. @dongjoon-hyun > If you don't mind, I'd like to propose to wait until we finish Apache Spark 3.5.5 release. Would you mind explaining why this has to be coupled with Spark 3.5.5, as discussion for 4.1/4.0 can always happen in parallel? I sincerely disagree that the option 2 is viable for us. This builds a huge limitation of SS users for Spark 3.5.4 on upgrading (they have to touch Spark 3.5.x before upgrading to 4.0.0), to achieve our goal of removing the "string" of incorrect config immediately. Again, this is just a string because it's only for migration logic and users won't be able to set the config manually. Why not just be more open for Spark 3.5.4 users to have a range of upgrade path? If you'd like to hear more voices, I'm happy to post the discussion to dev@. I just don't want this change to be coupled with non-relevant release. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
