This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-kubernetes-operator.git
The following commit(s) were added to refs/heads/main by this push:
new e2b5899e [SPARK-59990] Fix the documented precedence of system
properties in `configuration.md`
e2b5899e is described below
commit e2b5899e4b69b527cf409f555828389e38b894c1
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Tue Oct 6 06:24:37 2026 -0700
[SPARK-59990] Fix the documented precedence of system properties in
`configuration.md`
### What changes were proposed in this pull request?
This PR aims to fix `docs/configuration.md` to document that the values in
the property file, `spark-operator.properties`, take precedence over the system
properties of the operator JVM.
### Why are the changes needed?
The doc says that system properties, e.g. `-D` options to the operator JVM,
override the values in the property file. However,
`SparkOperatorConfManager.initialize()` loads `System.getProperties()` first
and then the property file, so the property file wins for the same key.
```java
private void initialize() {
initialConfig.putAll(System.getProperties());
Properties properties =
getProperties(
System.getProperty(INITIAL_CONFIG_FILE_PATH_PROPS_KEY,
INITIAL_CONFIG_FILE_PATH));
initialConfig.putAll(properties);
```
This has been the behavior since SPARK-48786 and in all releases including
[1.0.0](https://github.com/apache/spark-kubernetes-operator/releases/tag/1.0.0)
(2026-07-23). This PR fixes the doc instead of the behavior. The Helm chart
renders runtime-resolved properties into the property file, e.g.
`spark.kubernetes.operator.health.probePort`, which must match the container
ports and probes of the operator pod. Keeping the property file authoritative
prevents `-D` options in `jvmArgs` fro [...]
### Does this PR introduce _any_ user-facing change?
No behavior change. This fixes the documentation to match the existing
behavior.
### How was this patch tested?
Manually verified by running the operator jar with
`-Dspark.kubernetes.operator.basePropertyFileName=<file>`, where both the file
and a `-D` option set `spark.kubernetes.operator.reconciler.intervalSeconds`.
`SparkOperatorConf.RECONCILER_INTERVAL_SECONDS.getValue()` returned the value
in the file.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 5.5
Closes #936 from dongjoon-hyun/SPARK-59990.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
docs/configuration.md | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/docs/configuration.md b/docs/configuration.md
index d6a6dd2c..79628589 100644
--- a/docs/configuration.md
+++ b/docs/configuration.md
@@ -28,7 +28,8 @@ Spark Operator supports different ways to configure the
behavior:
it is also possible to override or append config properties in helm
[Values files](../build-tools/helm/spark-kubernetes-operator/values.yaml).
* **System Properties** : when provided as system properties (e.g. via -D
options to the
- operator JVM), it overrides the values provided in property file.
+ operator JVM), they apply only to the properties which are not provided in
the property file,
+ i.e. the values in the property file take precedence over system properties.
* **Hot property loading** : when enabled, a
[configmap](https://kubernetes.io/docs/concepts/configuration/configmap/)
would be created with
the operator in the same namespace. Operator can monitor updates performed
on the configmap. Hot
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
