This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-kubernetes-operator.git
The following commit(s) were added to refs/heads/main by this push:
new 37bf26b [SPARK-53994] Exclude `KerberosConfDriverFeatureStep` during
benchmarking
37bf26b is described below
commit 37bf26b9f2a4757410e68bd559e0879e6da95c65
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Wed Oct 22 16:16:41 2025 -0700
[SPARK-53994] Exclude `KerberosConfDriverFeatureStep` during benchmarking
### What changes were proposed in this pull request?
This PR aims to exclude `KerberosConfDriverFeatureStep` during benchmarking
because this is not used during `SparkApp` submission benchmark.
### Why are the changes needed?
To suppress irrelevant warning messages from Hadoop layer.
### Does this PR introduce _any_ user-facing change?
No. This is a test script change.
### How was this patch tested?
Manual review.
```
$ ./tests/benchmark/sparkapps.sh
CLEAN UP NAMESPACE FOR BENCHMARK
No resources found
START BENCHMARK WITH 1000 JOBS
FINISHED 1000 JOBS IN 193 SECONDS.
DELETED 1000 JOBS IN 254 SECONDS.
```
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #404 from dongjoon-hyun/SPARK-53994.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
tests/benchmark/sparkapps.sh | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/tests/benchmark/sparkapps.sh b/tests/benchmark/sparkapps.sh
index fbc84d8..e9f76e0 100755
--- a/tests/benchmark/sparkapps.sh
+++ b/tests/benchmark/sparkapps.sh
@@ -40,10 +40,11 @@ spec:
sparkConf:
spark.driver.memory: "256m"
spark.driver.memoryOverhead: "0m"
- spark.kubernetes.driver.request.cores: "100m"
- spark.kubernetes.driver.master: "local[1]"
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
spark.kubernetes.container.image: "apache/spark:4.0.1-java21-scala"
+ spark.kubernetes.driver.master: "local[1]"
+ spark.kubernetes.driver.pod.excludedFeatureSteps:
"org.apache.spark.deploy.k8s.features.KerberosConfDriverFeatureStep"
+ spark.kubernetes.driver.request.cores: "100m"
runtimeVersions:
sparkVersion: "4.0.1"
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
