This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-kubernetes-operator.git
The following commit(s) were added to refs/heads/main by this push:
new 948f664 [SPARK-58596] Remove
`spark.dynamicAllocation.shuffleTracking.enabled` from examples and docs
948f664 is described below
commit 948f6648cb8a9b4c8dad46361910559ed9185eee
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Wed Aug 5 23:34:10 2026 -0700
[SPARK-58596] Remove `spark.dynamicAllocation.shuffleTracking.enabled` from
examples and docs
### What changes were proposed in this pull request?
This PR aims to remove the
`spark.dynamicAllocation.shuffleTracking.enabled` setting from the example YAML
files and the documentation.
- `examples/pi.yaml`
- `examples/pi-java25-with-executor-resize-plugin.yaml`
- `docs/spark_custom_resources.md` (the `SparkApplication` example quoting
`examples/pi.yaml`)
### Why are the changes needed?
`spark.dynamicAllocation.shuffleTracking.enabled` is `true` by default
since Apache Spark 3.4.0 via
[SPARK-39846](https://issues.apache.org/jira/browse/SPARK-39846). Since all
examples use Spark 4.2.0, this setting is redundant and can be omitted to keep
the examples minimal.
- https://github.com/apache/spark/pull/37257
### Does this PR introduce _any_ user-facing change?
No. This is an example and documentation-only change with no behavior
difference.
### How was this patch tested?
Manually verified that no occurrence remains in the repository.
```
$ grep -rn "shuffleTracking" examples/ docs/ | wc -l
0
```
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Fable 5
Closes #784 from dongjoon-hyun/SPARK-58596.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
docs/spark_custom_resources.md | 1 -
examples/pi-java25-with-executor-resize-plugin.yaml | 1 -
examples/pi.yaml | 1 -
3 files changed, 3 deletions(-)
diff --git a/docs/spark_custom_resources.md b/docs/spark_custom_resources.md
index 8a1ae69..e3ca480 100644
--- a/docs/spark_custom_resources.md
+++ b/docs/spark_custom_resources.md
@@ -49,7 +49,6 @@ spec:
jars: "local:///opt/spark/examples/jars/spark-examples.jar"
sparkConf:
spark.dynamicAllocation.enabled: "true"
- spark.dynamicAllocation.shuffleTracking.enabled: "true"
spark.dynamicAllocation.maxExecutors: "3"
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
spark.kubernetes.container.image: "apache/spark:4.2.0-scala"
diff --git a/examples/pi-java25-with-executor-resize-plugin.yaml
b/examples/pi-java25-with-executor-resize-plugin.yaml
index f6e8ecd..067d701 100644
--- a/examples/pi-java25-with-executor-resize-plugin.yaml
+++ b/examples/pi-java25-with-executor-resize-plugin.yaml
@@ -23,7 +23,6 @@ spec:
sparkConf:
spark.dynamicAllocation.enabled: "true"
spark.dynamicAllocation.maxExecutors: "1"
- spark.dynamicAllocation.shuffleTracking.enabled: "true"
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
spark.kubernetes.container.image: "apache/spark:4.2.0-scala-java25"
spark.kubernetes.executor.resizeInterval: "1min"
diff --git a/examples/pi.yaml b/examples/pi.yaml
index eeee12a..896a454 100644
--- a/examples/pi.yaml
+++ b/examples/pi.yaml
@@ -21,7 +21,6 @@ spec:
jars: "local:///opt/spark/examples/jars/spark-examples.jar"
sparkConf:
spark.dynamicAllocation.enabled: "true"
- spark.dynamicAllocation.shuffleTracking.enabled: "true"
spark.dynamicAllocation.maxExecutors: "3"
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}-scala"
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
