DanielLeens commented on issue #11173: URL: https://github.com/apache/seatunnel/issues/11173#issuecomment-4791019258
Thanks for writing this up in such a concrete way. This reads like a valid feature request rather than a bug. I checked the current tree, and Spark support is still split into dedicated Spark 2.4 and 3.3 translation/starter lanes, with the current Spark path still tied to Scala 2.12. So Spark 4.1 support is more than a simple version bump: it needs a new Spark/Scala lane while keeping the existing 2.4 / 3.3 paths stable. For the first implementation, the safest boundary would be: 1. add a dedicated `seatunnel-translation-spark-4.1` module and matching starter module 2. keep `seatunnel-translation-spark-common` / `seatunnel-spark-starter-common` version-neutral unless Spark 4.1 API differences force a split 3. avoid changing current Spark 2.4 / 3.3 behavior in the same PR 4. include docs and one runnable validation path for the new starter If you open a PR for this, please link it here and keep the first patch focused on the new 4.1 lane rather than mixing broader Spark cleanup into the same change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
