LuciferYang opened a new pull request, #2748: URL: https://github.com/apache/uniffle/pull/2748
## What changes were proposed in this pull request? Add Spark 4.0.2 support with an independent `client-spark/spark4` module. Key decisions: - Independent module (not depending on spark3) to allow future divergence as Spark 4 evolves - Scala 2.13 + Java 17 as required by Spark 4 - SLF4J 2.x (`log4j-slf4j2-impl`) to match Spark 4's logging stack - `extraJavaTestArgs` property in root pom for JDK 17 module opens (all modules benefit) - CI workflow updated with per-profile JDK version matrix ## Why are the changes needed? Spark 4.0.2 is the first stable release of the Spark 4.x line. Users running Spark 4 currently have no official Uniffle client. This builds on #1806 (Scala 2.13 build support) and addresses the same goal as #1814 but with a different approach — see comparison below. ## Differences from #1814 | Aspect | #1814 | This PR | |--------|-------|---------| | Spark version | 4.0.0-preview1 | 4.0.2 (stable) | | Module strategy | spark4 depends on spark3 jar | Independent source copy | | SLF4J | Not handled | 2.x compatible | | commons-lang3 | Not handled | 3.17.0 (fixes `JAVA_21` field) | | CI | Per-profile JDK via matrix | Same approach | ## How was this patch tested? - Unit tests: 34 tests pass (`mvn test -Pspark4`) - Shaded jar builds successfully - Integration tests: `AQERepartitionTest`, `MapSideCombineTest` ## Notes - Kerberos + JDK 17 issue (mentioned in #1814) is not addressed here — Spark 4.0.2 itself does not include `java.security.jgss` opens. Tests not involving Kerberos work fine. - `client-spark/extension` module (Scala code) is excluded from spark4 profile for now; Scala 2.13 compatibility for that module can be a follow-up. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
