This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-kubernetes-operator.git
The following commit(s) were added to refs/heads/main by this push:
new 947d8d2 [SPARK-54659] Enhance gradle task 'generateConfPropsDoc' to
account for recent changes
947d8d2 is described below
commit 947d8d2e7d4c3c438795887be0b89072b69c2f6e
Author: Zhou JIANG <[email protected]>
AuthorDate: Wed Dec 10 11:16:24 2025 -0800
[SPARK-54659] Enhance gradle task 'generateConfPropsDoc' to account for
recent changes
### What changes were proposed in this pull request?
Enhance existing task `generateConfPropsDoc` and add new task
`validateConfPropsDocUpToDate`
* Task `generateConfPropsDoc` would no longer automatically overwrite
existing doc file. It would generate file for validation purpose only, and
would perform overwrite only if property flag is enabled.
* Also fixed the file output location to use project rootDir instead of
property value of `user.dir` as the latter may not point to project root in
various cases.
* Added new task `validateConfPropsDocUpToDate` to compare the staged docs
vs desired (generated) docs, and prompts developer to update docs if needed.
### Why are the changes needed?
Although we have `generateConfPropsDoc` chained in the build task, it's not
mandatory and hence docs can become out of date. By adding this patch,
developers can be more mindful for doc updates corresponding to code change.
### Does this PR introduce _any_ user-facing change?
No. Building tasks update only.
### How was this patch tested?
Self-contained upon success & manually tested failure scenario when docs
become out of date:
```
FAILURE: Build failed with an exception.
* Where:
Build file
'/Users/zhou_jiang/work/bench/apache/ipr/apache-spark-kubernetes-operator/build-tools/docs-utils/build.gradle'
line: 59
* What went wrong:
Execution failed for task
':build-tools-docs-utils:validateConfPropsDocUpToDate'.
> Doc
/Users/zhou_jiang/work/bench/apache/ipr/apache-spark-kubernetes-operator/docs/config_properties.md
out of date: if operator conf properties has been updated, please re-generate
properties docs by running ./gradlew
build-tools-docs-utils:generateConfPropsDoc -Poverwrite
```
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #422 from jiangzho/doc_generator.
Authored-by: Zhou JIANG <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
build-tools/docs-utils/build.gradle | 30 +++++++++++++++++++++++++++---
1 file changed, 27 insertions(+), 3 deletions(-)
diff --git a/build-tools/docs-utils/build.gradle
b/build-tools/docs-utils/build.gradle
index bf467e6..ab4d982 100644
--- a/build-tools/docs-utils/build.gradle
+++ b/build-tools/docs-utils/build.gradle
@@ -19,7 +19,9 @@
ext {
javaMainClass =
"org.apache.spark.k8s.operator.utils.ConfOptionDocGenerator"
- docsPath = System.getProperty("user.dir") + "/docs"
+ defaultDocsPath = "${project.rootDir}/docs"
+ docOutputPath = project.hasProperty("overwrite") ? defaultDocsPath :
"build"
+ generatedDocFileName = "config_properties.md"
}
dependencies {
@@ -36,7 +38,29 @@ test {
tasks.register('generateConfPropsDoc', Exec) {
description = "Generate config properties doc for operator"
- commandLine "java", "-classpath",
sourceSets.main.runtimeClasspath.getAsPath(), javaMainClass, docsPath
+ dependsOn jar
+ commandLine "java", "-classpath",
sourceSets.main.runtimeClasspath.getAsPath(), javaMainClass, docOutputPath
+ println("Doc generated at $docOutputPath/$generatedDocFileName")
}
-build.finalizedBy(generateConfPropsDoc)
+tasks.register('validateConfPropsDocUpToDate') {
+ description = "Validate the generate config properties doc is up to date"
+ dependsOn 'generateConfPropsDoc'
+
+ doLast {
+ File generated = file("$docOutputPath/$generatedDocFileName")
+ File staged = file("$defaultDocsPath/$generatedDocFileName")
+ if (!staged.exists()) {
+ throw new GradleException("Staged properties doc file does not exist.")
+ }
+ String generatedText = generated.text.trim()
+ String stagedText = staged.text.trim()
+ if (generatedText != stagedText) {
+ throw new GradleException("Doc $defaultDocsPath/$generatedDocFileName
out of date: " +
+ "if operator conf properties have been updated, please re-generate
properties docs " +
+ "by running ./gradlew build-tools-docs-utils:generateConfPropsDoc
-Poverwrite")
+ }
+ }
+}
+
+build.finalizedBy(validateConfPropsDocUpToDate)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
