DanielLeens commented on issue #11184: URL: https://github.com/apache/seatunnel/issues/11184#issuecomment-4814463302
Thanks, this is a useful direction to raise. I agree that a single Spark-oriented distribution may be worth evaluating later, but I would not make that the first delivery boundary for Spark 4.1 support. The immediate constraint is compatibility: Spark 4.1 raises the runtime/JDK floor to 17, while the current Spark 2.4 / 3.3 release path is still expected to remain stable for existing users. Because of that, the dual-tarball approach is the safer Phase 1 choice for now: 1. existing users do not get an unexpected runtime requirement change 2. Spark 4.1 users get an explicit artifact with a clear JDK 17 expectation 3. the review scope for #11174 stays focused on getting 4.1 usable first After that lands, I think it is reasonable to discuss whether a unified Spark distribution would actually simplify operations enough to justify the packaging change. But that follow-up discussion should come with a concrete compatibility story: which JDK becomes the baseline, whether Spark 2.4 still remains in scope, and how we avoid surprising current release consumers. So from my side, I would treat `single Spark distribution` as a follow-up design option, not as a blocker for the current Phase 0 / Phase 1 delivery. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
