yew1eb opened a new pull request, #3870:
URL: https://github.com/apache/celeborn/pull/3870
### Why are the changes needed?
The Celeborn tab in the Spark UI does not show which celeborn-client version
the application is using, which makes version-mismatch debugging between a
Spark job and the cluster unnecessarily hard. See CELEBORN-2478.
### What changes were included?
Mirrors Uniffle's "Uniffle Build Information" pattern — the info is baked
into the client at compile time and rendered by the Spark UI plugin, with **no
reporting to the master**:
- **Compile-time constants** (`org.apache.celeborn.ProjectConstants`,
`VERSION` / `REVISION` / `getGitCommitId()`) generated from both build systems:
- Maven: `templating-maven-plugin` + `buildnumber-maven-plugin` +
`properties-maven-plugin` on the new `common/src/main/java-templates` (same
recipe as Uniffle)
- SBT: the existing version source generator in
`project/CelebornBuild.scala` is extended to also emit the same Java class
using `gitHeadCommit`
- **Persistence**: `CelebornDriverPlugin.init` writes a
`CelebornBuildInfoUIData` entry into the Spark KVStore, so it survives
application end and shows up on History Server replay
- **UI**: a collapsible "Celeborn Client Build Information" table rendered
in `CelebornShufflePage`, ahead of "Celeborn Properties"
- Tests: `CelebornBuildInfoSuite` covering KVStore round-trip and constant
sanity
### How was this patch tested?
- `./build/mvn -Pspark-3.5 -pl common,client-spark/spark-3-ui -am clean
install` — build success; the generated `ProjectConstants.java` contains the
correct version and current git revision
- `./build/mvn -Pspark-3.5 -pl client-spark/spark-3-ui clean test` — 8/8
tests pass (5 existing + 3 new)
- `./build/sbt "celeborn-common/Compile/compile"` — SBT generates identical
constants
- Spotless formatting applied
End-to-end verification against a live cluster / History Server was not run
in this change.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
