This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-kubernetes-operator.git
The following commit(s) were added to refs/heads/main by this push:
new 042047b [SPARK-53723] Support `SparkCluster` Sentinel Resource
042047b is described below
commit 042047b98f67b2fda282f872dd70be0da29194e5
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Thu Sep 25 16:22:14 2025 -0700
[SPARK-53723] Support `SparkCluster` Sentinel Resource
### What changes were proposed in this pull request?
This PR aims to support `SparkCluster` Sentinel Resource like
`SparkApplication` Sentinel Resource.
### Why are the changes needed?
For a feature parity between `SparkApplication` and `SparkCluster` feature.
### Does this PR introduce _any_ user-facing change?
No, this is a new feature.
### How was this patch tested?
Pass the CIs.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #349 from dongjoon-hyun/SPARK-53723.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../templates/workload-rbac.yaml | 23 ++++++++++++++++++++++
.../helm/spark-kubernetes-operator/values.yaml | 10 ++++++++++
docs/operations.md | 3 +++
3 files changed, 36 insertions(+)
diff --git
a/build-tools/helm/spark-kubernetes-operator/templates/workload-rbac.yaml
b/build-tools/helm/spark-kubernetes-operator/templates/workload-rbac.yaml
index 5af3b31..865ead2 100644
--- a/build-tools/helm/spark-kubernetes-operator/templates/workload-rbac.yaml
+++ b/build-tools/helm/spark-kubernetes-operator/templates/workload-rbac.yaml
@@ -177,3 +177,26 @@ spec:
{{- end }}
---
{{- end }}
+{{- if $workloadResources.sparkClusterSentinel.create }}
+{{- range $sentinelNs :=
.Values.workloadResources.sparkClusterSentinel.sentinelNamespaces.data }}
+apiVersion: spark.apache.org/v1
+kind: SparkCluster
+metadata:
+ name: {{ $workloadResources.sparkClusterSentinel.name }}
+ namespace: {{ $sentinelNs }}
+ labels:
+ "spark.operator/sentinel": "true"
+ {{- template "spark-operator.workloadLabels" $ }}
+ annotations:
+ {{- template "spark-operator.workloadAnnotations" $ }}
+spec:
+ runtimeVersions:
+ sparkVersion: "dummy"
+ clusterTolerations:
+ instanceConfig:
+ initWorkers: 0
+ minWorkers: 0
+ maxWorkers: 0
+{{- end }}
+---
+{{- end }}
diff --git a/build-tools/helm/spark-kubernetes-operator/values.yaml
b/build-tools/helm/spark-kubernetes-operator/values.yaml
index 58f1c30..1292241 100644
--- a/build-tools/helm/spark-kubernetes-operator/values.yaml
+++ b/build-tools/helm/spark-kubernetes-operator/values.yaml
@@ -159,6 +159,16 @@ workloadResources:
# When enabled, a sentinel resource of type SparkApp will be deployed to
namespace(s) provided
# Note that sentinelNamespaces list should be a subset of
{workloadResources.namespaces.data}
# - "default"
+ sparkClusterSentinel:
+ create: false
+ name: "spark-cluster-sentinel"
+ sentinelNamespaces:
+ data:
+ # - "spark-1"
+ # - "spark-2"
+ # When enabled, a sentinel resource of type SparkCluster will be
deployed to namespace(s) provided
+ # Note that sentinelNamespaces list should be a subset of
{workloadResources.namespaces.data}
+ # - "default"
# workload resources are by default annotated to avoid workload abort due to
operator upgrade
annotations:
"helm.sh/resource-policy": keep
diff --git a/docs/operations.md b/docs/operations.md
index 18a5b91..7537381 100644
--- a/docs/operations.md
+++ b/docs/operations.md
@@ -114,6 +114,9 @@ following table:
| workloadResources.sparkApplicationSentinel.create | If
enabled, sentinel resources will be created for operator to watch and reconcile
for the health probe purpose.
| false
|
| workloadResources.sparkApplicationSentinel.name | Name for
sentinel resources.
| "spark-app-sentinel"
|
| workloadResources.sparkApplicationSentinel.sentinelNamespaces | A list of
namespaces where sentinel resources will be created in. Note that these
namespaces have to be a subset of `workloadResources.namespaces.data`.
|
|
+| workloadResources.sparkClusterSentinel.create | If
enabled, sentinel resources will be created for operator to watch and reconcile
for the health probe purpose.
| false
|
+| workloadResources.sparkClusterSentinel.name | Name for
sentinel resources.
| "spark-cluster-sentinel"
|
+| workloadResources.sparkClusterSentinel.sentinelNamespaces | A list of
namespaces where sentinel resources will be created in. Note that these
namespaces have to be a subset of `workloadResources.namespaces.data`.
|
|
| operatorConfiguration.append | If set to
true, below conf file & properties would be appended to default conf.
Otherwise, they would override default properties.
| true
|
| operatorConfiguration.log4j2.properties | The
default log4j2 configuration.
| Refer default
[log4j2.properties](../build-tools/helm/spark-kubernetes-operator/conf/log4j2.properties)
|
| operatorConfiguration.spark-operator.properties | The
default operator configuration.
|
|
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
