This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-kubernetes-operator.git
The following commit(s) were added to refs/heads/main by this push:
new 5e7b8f4 [SPARK-49380] Add `cluster-with-template.yaml` example
5e7b8f4 is described below
commit 5e7b8f41f97d3e5ca5772f8ca0b16ca9e56896fa
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Sun Aug 25 19:54:19 2024 -0700
[SPARK-49380] Add `cluster-with-template.yaml` example
### What changes were proposed in this pull request?
This PR aims to add `cluster-with-template.yaml` example.
### Why are the changes needed?
Apache Spark K8s Operator starts to support template-based Spark Cluster
creation via the following.
- https://github.com/apache/spark-kubernetes-operator/pull/80
We had better provide a concrete example like the following use cases.
- Custom annotations and labels.
- Custom full spec like `priorityClassName`, `securityContext`, `sidecar`
containers
### Does this PR introduce _any_ user-facing change?
No. This is an unreleased new feature.
### How was this patch tested?
Manual review.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #101 from dongjoon-hyun/SPARK-49380.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
examples/cluster-with-template.yaml | 78 +++++++++++++++++++++++++++++++++++++
1 file changed, 78 insertions(+)
diff --git a/examples/cluster-with-template.yaml
b/examples/cluster-with-template.yaml
new file mode 100644
index 0000000..6d85a1f
--- /dev/null
+++ b/examples/cluster-with-template.yaml
@@ -0,0 +1,78 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+apiVersion: spark.apache.org/v1alpha1
+kind: SparkCluster
+metadata:
+ name: cluster-with-template
+spec:
+ runtimeVersions:
+ sparkVersion: "4.0.0-preview1"
+ clusterTolerations:
+ instanceConfig:
+ initWorkers: 1
+ minWorkers: 1
+ maxWorkers: 1
+ masterSpec:
+ masterStatefulSetMetadata:
+ annotations:
+ customAnnotation: "annotation"
+ masterStatefulSetSpec:
+ template:
+ spec:
+ priorityClassName: system-cluster-critical
+ securityContext:
+ runAsUser: 0
+ containers:
+ - name: sidecar
+ image: registry.k8s.io/pause
+ resources:
+ requests:
+ cpu: "0.1"
+ memory: "10Mi"
+ limits:
+ cpu: "0.1"
+ memory: "10Mi"
+ masterServiceMetadata:
+ annotations:
+ customAnnotation: "svc1"
+ workerSpec:
+ workerStatefulSetMetadata:
+ annotations:
+ customAnnotation: "annotation"
+ workerStatefulSetSpec:
+ template:
+ spec:
+ priorityClassName: system-cluster-critical
+ securityContext:
+ runAsUser: 0
+ containers:
+ - name: sidecar
+ image: registry.k8s.io/pause
+ resources:
+ requests:
+ cpu: "0.1"
+ memory: "10Mi"
+ limits:
+ cpu: "0.1"
+ memory: "10Mi"
+ workerServiceMetadata:
+ annotations:
+ customAnnotation: "annotation"
+ sparkConf:
+ spark.kubernetes.container.image: "spark:4.0.0-preview1"
+ spark.master.ui.title: "Spark Cluster with Template"
+ spark.master.rest.enabled: "true"
+ spark.master.rest.host: "0.0.0.0"
+ spark.ui.reverseProxy: "true"
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
