LuciferYang opened a new pull request, #2749:
URL: https://github.com/apache/uniffle/pull/2749
## What changes were proposed in this pull request?
Enable `client-spark/extension` (the Uniffle Spark UI plugin module,
published as `rss-client-spark-ui`) under the `spark4` profile.
Before this PR the module was only built under `spark3*` profiles, so Spark
4 users lost the Uniffle Shuffle tab, the History Server plugin, and the
listener-driven status store.
### Core challenge
Spark 4 switched the UI servlet API from `javax.servlet` (Jetty 9) to
`jakarta.servlet` (Jetty 11+). `WebUIPage.render(request)` must override its
parent signature exactly, so the same `.scala` file cannot satisfy both Spark 3
and Spark 4 from a single source root.
### Approach
Dual servlet source roots selected by Maven profile:
- `src/main/scala-javax/org/apache/spark/ui/ServletCompat.scala` — `type
HttpServletRequest = javax.servlet.http.HttpServletRequest`
- `src/main/scala-jakarta/org/apache/spark/ui/ServletCompat.scala` — `type
HttpServletRequest = jakarta.servlet.http.HttpServletRequest`
- `ShufflePage.scala` imports `ServletCompat.HttpServletRequest` and
overrides `render` with that alias.
- `build-helper-maven-plugin` adds one source root based on
`${extension.servlet.source.dir}`.
- Default property in `client-spark/extension/pom.xml` is `scala-javax`; a
same-id `spark4` profile in this module's pom overrides it to `scala-jakarta`.
Maven merges same-id profiles across the reactor, so `-Pspark4` activates both.
### Other changes
- Add `client-spark/extension` to the root pom's `spark4` profile
`<modules>`.
- Add `rss-client-spark-ui` dependency to `client-spark/spark4/pom.xml`,
mirroring `client-spark/spark3/pom.xml`.
- Bump `build-helper-maven-plugin` 1.10 → 3.6.0 (the 1.10 release is from
2014).
- Contributor notes in `extension/pom.xml` describe the steps to add a new
Spark major version, a new servlet variant, IDE re-import on profile switch,
and `~/.m2` hygiene.
`ServletCompat` is `private[ui]` so it remains an internal shim; callers
must reside in `org.apache.spark.ui`.
## Why are the changes needed?
Follow-up to #2748 which added Spark 4 core support but explicitly deferred
`client-spark/extension` as a known limitation. Without this PR, Spark 4 users
have no Uniffle UI integration.
## How was this patch tested?
- Compile-matrix across every profile that includes the module:
- `spark3`, `spark3.0`, `spark3.2`, `spark3.2.0`, `spark3.3`, `spark3.4`,
`spark3.5` (Scala 2.12, javax variant) — ✅
- `spark4` (Scala 2.13, jakarta variant) — ✅
- Verified `mvn -Pspark4 dependency:tree` shows `spark-core_2.13:4.0.2`,
`scala-library:2.13.16`, and `jakarta.servlet-api:5.0.0` on the classpath.
- Verified `object ServletCompat` compiles to the correct underlying servlet
class under each profile.
- Sibling-file symmetry check: the two `ServletCompat.scala` files differ
only in the variant-specific strings.
Runtime validation on a live Spark 4 cluster (UI tab rendering, History
Server plugin discovery, listener event flow) is left for follow-up E2E work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
