yew1eb commented on code in PR #3870:
URL: https://github.com/apache/celeborn/pull/3870#discussion_r4267653792
##########
common/pom.xml:
##########
@@ -192,6 +192,56 @@
<groupId>org.xolstice.maven.plugins</groupId>
<artifactId>protobuf-maven-plugin</artifactId>
</plugin>
+ <plugin>
+ <groupId>org.codehaus.mojo</groupId>
+ <artifactId>buildnumber-maven-plugin</artifactId>
+ <configuration>
+
<outputDirectory>${project.build.directory}/generated/build-metadata</outputDirectory>
+ <outputName>build.properties</outputName>
+ <revisionPropertyName>git.revision</revisionPropertyName>
Review Comment:
Done in ea7162a: `revisionOnScmFailure` set to `N/A` (matching the SBT
fallback), plus a `source-archive` CI job building from a `git archive` export.
##########
client-spark/spark-3-ui/src/main/scala/org/apache/spark/shuffle/celeborn/CelebornPlugin.scala:
##########
@@ -49,10 +50,29 @@ private class CelebornDriverPlugin extends DriverPlugin
with Logging {
logInfo("Initializing CelebornDriverPlugin...")
this.sc = sc
val kvStore = sc.statusStore.store
+ writeBuildInfo(kvStore)
new CelebornListener(kvStore, sc.conf).register(sc)
java.util.Collections.emptyMap[String, String]()
}
+ /**
+ * Persists the build info of the celeborn-client bundled with this
application into the
+ * KVStore so that it survives application end and is available on History
Server replay,
+ * where the Spark UI "Celeborn" tab renders it as "Celeborn Client Build
Information".
+ */
+ private def writeBuildInfo(kvStore: KVStore): Unit = {
+ val buildInfo = new CelebornBuildInfoUIData(Seq(
+ "Version" -> org.apache.celeborn.ProjectConstants.VERSION,
+ "Commit Id" -> org.apache.celeborn.ProjectConstants.getGitCommitId,
+ "Revision" -> org.apache.celeborn.ProjectConstants.REVISION))
+ try {
+ kvStore.write(buildInfo)
Review Comment:
Done in ea7162a: the plugin now posts a `CelebornBuildInfoEvent` (same
pattern as gluten's `GlutenBuildInfoEvent`), and the listener restores the
build info from the same event both live and on replay. Covered in
`CelebornListenerSuite`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
