This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark-docker.git
The following commit(s) were added to refs/heads/master by this push:
new 0ed54dc [SPARK-56783][FOLLOWUP] Use `sed` instead because
sbt-pom-reader is unable to disable
0ed54dc is described below
commit 0ed54dca2c0e3a24637de60af026d8b373b6946a
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Thu May 7 12:46:03 2026 -0700
[SPARK-56783][FOLLOWUP] Use `sed` instead because sbt-pom-reader is unable
to disable
### What changes were proposed in this pull request?
This PR is a followup of SPARK-56783 and aims to disable Apache Spark's
`volcano` profile during K8S IT by patching the checked-out `pom.xml` with
`sed` instead of relying on the `-P!volcano` sbt option.
- https://github.com/apache/spark-docker/pull/111
### Why are the changes needed?
The previously added `-P!volcano` is a Maven CLI deactivation syntax that
sbt's `sbt-pom-reader` does not honor. In Apache Spark's
`project/SparkBuild.scala`, the profile parser strips only the `-P` prefix and
then feeds `!volcano` back as a profile name to activate, which is silently
ignored. The `volcano` profile in
`resource-managers/kubernetes/integration-tests/pom.xml` is
`activeByDefault=true` since `v4.2.0-preview5`, so it remained active and
caused `VolcanoSuite` to run and fa [...]
```
[info] VolcanoSuite:
[info] - SPARK-42190: Run SparkPi with local[*] *** FAILED *** (3 minutes,
3 seconds)
```
This change instead patches the checked-out `pom.xml` so the `volcano`
profile is no longer `activeByDefault`, which works regardless of whether sbt
or Maven is invoked. It is a no-op for Spark `<= 4.1.x` where `volcano` is not
`activeByDefault`.
### Does this PR introduce _any_ user-facing change?
No. This is a CI-only change.
### How was this patch tested?
Pass the CIs.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 4.7 (1M context)
Closes #114 from dongjoon-hyun/dongjoon/upbeat-austin-194d1e.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.github/workflows/main.yml | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 46b03cd..0d34497 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -282,9 +282,10 @@ jobs:
kubectl create clusterrolebinding serviceaccounts-cluster-admin
--clusterrole=cluster-admin --group=system:serviceaccounts || true
minikube image load ${{ env.IMAGE_URL }}
+ sed -i '/<id>volcano<\/id>/,/<\/profile>/
s|<activeByDefault>true</activeByDefault>|<activeByDefault>false</activeByDefault>|'
resource-managers/kubernetes/integration-tests/pom.xml
+
eval $(minikube docker-env)
OPTS="-Pkubernetes -Pkubernetes-integration-tests "
- OPTS+='-P!volcano '
OPTS+="-Dspark.kubernetes.test.driverRequestCores=0.5
-Dspark.kubernetes.test.executorRequestCores=0.2 "
OPTS+="-Dspark.kubernetes.test.deployMode=minikube "
OPTS+="-Dspark.kubernetes.test.imageRepo=${TEST_REPO}
-Dspark.kubernetes.test.imageTag=${UNIQUE_IMAGE_TAG} "
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
