Reamer commented on code in PR #4896:
URL: https://github.com/apache/zeppelin/pull/4896#discussion_r1894082589
##########
charts/zeppelin/templates/interpreter/spark/configmap.yaml:
##########
@@ -0,0 +1,141 @@
+{{- /*
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/ -}}
+
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: {{ include "zeppelin.interpreter.spark.configMapName" . }}
+ namespace: {{ .Release.Namespace }}
+ labels:
+ {{- include "zeppelin.labels" . | nindent 4 }}
+data:
+ spark-defaults.conf: |
+ #
+ # Licensed to the Apache Software Foundation (ASF) under one or more
+ # contributor license agreements. See the NOTICE file distributed with
+ # this work for additional information regarding copyright ownership.
+ # The ASF licenses this file to You under the Apache License, Version 2.0
+ # (the "License"); you may not use this file except in compliance with
+ # the License. You may obtain a copy of the License at
+ #
+ # http://www.apache.org/licenses/LICENSE-2.0
+ #
+ # Unless required by applicable law or agreed to in writing, software
+ # distributed under the License is distributed on an "AS IS" BASIS,
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ # See the License for the specific language governing permissions and
+ # limitations under the License.
+ #
+
+ # Default system properties included when running spark-submit.
+ # This is useful for setting default environmental settings.
+
+ {{- range $key, $value := .Values.interpreter.spark.conf }}
+ {{ $key }} {{ $value }}
+ {{- end }}
+
+ driver-pod-template.yaml: |
Review Comment:
The driver pod is created directly by Zeppelin. I think this file has no
effect on an already existing pod. Zeppelin starts the Driver Pod in client
mode.
##########
charts/zeppelin/templates/server/configmap.yaml:
##########
@@ -0,0 +1,72 @@
+{{- /*
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/ -}}
+
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: {{ include "zeppelin.server.configMapName" . }}
+ namespace: {{ .Release.Namespace }}
+ labels:
+ {{- include "zeppelin.server.labels" . | nindent 4 }}
+data:
+ zeppelin-env.sh: |
+ #!/bin/bash
+ #
+ # Licensed to the Apache Software Foundation (ASF) under one or more
+ # contributor license agreements. See the NOTICE file distributed with
+ # this work for additional information regarding copyright ownership.
+ # The ASF licenses this file to You under the Apache License, Version 2.0
+ # (the "License"); you may not use this file except in compliance with
+ # the License. You may obtain a copy of the License at
+ #
+ # http://www.apache.org/licenses/LICENSE-2.0
+ #
+ # Unless required by applicable law or agreed to in writing, software
+ # distributed under the License is distributed on an "AS IS" BASIS,
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ # See the License for the specific language governing permissions and
+ # limitations under the License.
+ #
+
+ export SPARK_MASTER=k8s://https://kubernetes.default.svc
+ export ZEPPELIN_K8S_SERVICE_NAME={{ include "zeppelin.server.serviceName"
. }}
+ export ZEPPELIN_K8S_SPARK_CONTAINER_IMAGE={{ include
"zeppelin.interpreter.spark.image" . }}
+
+ zeppelin-site.xml: |
Review Comment:
Please carry out a configuration on environment variables. This
configuration is easier to understand and does not require an additional xsl
file.
##########
charts/zeppelin/values.yaml:
##########
@@ -0,0 +1,271 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+
+# http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# Default values for zeppelin.
+# This is a YAML-formatted file.
+# Declare variables to be passed into your templates.
+
+# -- String to partially override release name.
+nameOverride: ""
+
+# -- String to fully override release name.
+fullnameOverride: ""
+
+global:
+ image:
+ # -- Global image registry.
+ registry: docker.io
+ # -- Global image pull policy.
+ pullPolicy: IfNotPresent
+ # -- Global image pull secrets for private image registry.
+ pullSecrets: []
+ # - name: <secret-name>
+
+server:
+ image:
+ # -- Zeppelin server image registry.
+ # @default -- If not set, `global.image.registry` will be used.
+ registry: ""
+ # -- Zeppelin server image repository.
+ repository: apache/zeppelin
+ # -- Zeppelin image tag.
+ # @default -- If not set, the chart appVersion will be used.
+ tag: ""
+ # -- Zeppelin server image pull policy.
+ pullPolicy: IfNotPresent
+ # -- Zeppelin server image pull secrets for private image registry.
+ pullSecrets: []
+ # - name: <secret-name>
+
+ # -- Zeppelin configurations.
+ # For detailed information, please refer:
https://zeppelin.apache.org/docs/latest/setup/operation/configuration.html.
+ conf: {}
+
+ # -- Extra labels for Zeppelin server pods.
+ labels: {}
+ # key1: value1
+ # key2: value2
+
+ # -- Extra annotations for Zeppelin server pods.
+ annotations: {}
+ # key1: value1
+ # key2: value2
+
+ # -- Volumes for Zeppelin server pods.
+ volumes: []
+
+ # -- Node selector for Zeppelin server pods.
+ nodeSelector: {}
+
+ # -- Affinity for Zeppelin server pods.
+ affinity: {}
+
+ # -- List of node taints to tolerate for Zeppelin server pods.
+ tolerations: []
+
+ # -- Priority class for Zeppelin server pods.
+ priorityClassName: ""
+
+ # -- Security context for Zeppelin server pods.
+ podSecurityContext: {}
+
+ # -- Environment variables for Zeppelin server containers.
+ env:
+ - name: ZEPPELIN_HOME
+ value: /opt/zeppelin
+ - name: SPARK_HOME
+ value: /opt/spark
+ - name: SERVICE_DOMAIN
+ value: local.zeppelin-project.org:8080
+ - name: ZEPPELIN_PORT
+ value: "8080"
+ - name: ZEPPELIN_SERVER_RPC_PORTRANGE
+ value: 12320:12320
+ - name: ZEPPELIN_K8S_TEMPLATE_DIR
+ value: /opt/zeppelin/k8s
+
+ # -- Environment variable sources for Zeppelin server containers.
+ envFrom: []
+
+ # -- Volume mounts for Zeppelin server containers.
+ volumeMounts: []
+
+ # -- Resource requests and limits for Zeppelin server containers.
+ resources: {}
+ # limits:
+ # cpu: 100m
+ # memory: 300Mi
+ # requests:
+ # cpu: 100m
+ # memory: 300Mi
+
+ # -- Security context for Zeppelin server containers.
+ securityContext:
+ runAsNonRoot: true
+ runAsUser: 1000
+ runAsGroup: 0
+
+ serviceAccount:
+ # -- Specifies whether a service account should be created for the
Zeppelin server.
+ create: true
+ # -- Optional name for the Zeppelin server service account.
+ name: ""
+ # -- Extra annotations for the Zeppelin server service account.
+ annotations: {}
+
+ service:
+ # -- Service type for Zeppelin server.
+ type: ClusterIP
+ # -- Service port for Zeppelin server.
+ port: 8080
+
+ ingress:
+ # -- Enable ingress for Zeppelin server.
+ enable: false
+ # -- Ingress class name for Zeppelin server ingress.
+ className: ""
+ # -- Annotations for the Zeppelin server ingress.
+ annotations: {}
+ # kubernetes.io/ingress.class: nginx
+ # kubernetes.io/tls-acme: "true"
+ # -- Hosts for Zeppelin server ingress.
+ hosts:
+ - host: local.zeppelin-project.org
+ paths:
+ - path: /
+ pathType: ImplementationSpecific
+ # -- TLS configuration for Zeppelin server ingress.
+ tls: []
+ # - secretName: zeppelin-server-tls
+ # hosts:
+ # - local.zeppelin-project.org
+
+interpreter:
+ image:
+ # -- Zeppelin interpreter image registry.
+ # @default -- If not set, `global.image.registry` will be used.
+ registry: ""
+ # -- Zeppelin interpreter image repository.
+ repository: apache/zeppelin-interpreter
+ # -- Zeppelin interpreter image tag.
+ # @default -- If not set, the chart appVersion will be used.
+ tag: ""
+ # -- Zeppelin interpreter image pull policy.
+ pullPolicy: IfNotPresent
+ # -- Zeppelin interpreter image pull secrets for private image registry.
+ pullSecrets: []
+ # - name: <secret-name>
+
+ spark:
+ image:
+ # -- Spark image registry.
+ # @default -- If not set, `global.image.registry` will be used.
+ registry: ""
+ # -- Spark image repository.
+ repository: spark
+ # -- Spark image tag.
+ tag: 3.5.3
+
+ # -- Spark configurations.
+ conf:
+ spark.kubernetes.driver.podTemplateFile:
file:///opt/spark/conf/driver-pod-template.yaml
+ spark.kubernetes.executor.podTemplateFile:
file:///opt/spark/conf/executor-pod-template.yaml
+
+ # -- Extra labels for Spark pods.
+ labels: {}
+
+ # -- Extra annotations for Spark pods.
+ annotations: {}
+
+ # -- Volumes for Spark pods.
+ volumes: []
+
+ # -- Environment variables for Spark containers.
+ env: []
+
+ # -- Environment variable sources for Spark containers.
+ envFrom: []
+
+ # -- Volume mounts for Spark containers.
+ volumeMounts: []
+
+ # -- Extra labels for Zeppelin interpreter pods.
+ labels: {}
+ # key1: value1
+ # key2: value2
+
+ # -- Extra annotations for Zeppelin interpreter pods.
+ annotations: {}
+ # key1: value1
+ # key2: value2
+
+ # -- Volumes for Zeppelin interpreter pods.
+ volumes: []
+
+ # -- Node selector for Zeppelin interpreter pods.
+ nodeSelector: {}
+
+ # -- Affinity for Zeppelin interpreter pods.
+ affinity: {}
+
+ # -- List of node taints to tolerate for Zeppelin interpreter pods.
+ tolerations: []
+
+ # -- Priority class for Zeppelin interpreter pods.
+ priorityClassName: ""
+
+ # -- Security context for Zeppelin interpreter pods.
+ podSecurityContext: {}
+
+ # -- Environment variables for Zeppelin interpreter containers.
+ env:
+ - name: ZEPPELIN_HOME
+ value: /opt/zeppelin
+ - name: SPARK_HOME
+ value: /opt/spark
+ - name: SPARK_CONF_DIR
+ value: /opt/spark/conf
+
+ # -- Environment variable sources for Zeppelin interpreter containers.
+ envFrom: []
+
+ # -- Volume mounts for Zeppelin interpreter containers.
+ volumeMounts: []
+
+ # -- Resource requests and limits for Zeppelin interpreter containers.
+ resources: {}
+ # limits:
+ # cpu: 100m
+ # memory: 300Mi
+ # requests:
+ # cpu: 100m
+ # memory: 300Mi
+
+ # -- Security context for Zeppelin interpreter containers.
+ securityContext:
+ runAsNonRoot: true
+ runAsUser: 1000
Review Comment:
Please do not enter a fixed UID. Openshift, for example, uses a random UID
by default.
##########
charts/zeppelin/templates/interpreter/spark/configmap.yaml:
##########
@@ -0,0 +1,141 @@
+{{- /*
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/ -}}
+
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: {{ include "zeppelin.interpreter.spark.configMapName" . }}
+ namespace: {{ .Release.Namespace }}
+ labels:
+ {{- include "zeppelin.labels" . | nindent 4 }}
+data:
+ spark-defaults.conf: |
+ #
+ # Licensed to the Apache Software Foundation (ASF) under one or more
+ # contributor license agreements. See the NOTICE file distributed with
+ # this work for additional information regarding copyright ownership.
+ # The ASF licenses this file to You under the Apache License, Version 2.0
+ # (the "License"); you may not use this file except in compliance with
+ # the License. You may obtain a copy of the License at
+ #
+ # http://www.apache.org/licenses/LICENSE-2.0
+ #
+ # Unless required by applicable law or agreed to in writing, software
+ # distributed under the License is distributed on an "AS IS" BASIS,
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ # See the License for the specific language governing permissions and
+ # limitations under the License.
+ #
+
+ # Default system properties included when running spark-submit.
+ # This is useful for setting default environmental settings.
+
+ {{- range $key, $value := .Values.interpreter.spark.conf }}
+ {{ $key }} {{ $value }}
+ {{- end }}
+
+ driver-pod-template.yaml: |
+ metadata:
+ {{- with .Values.interpreter.spark.labels }}
+ labels:
+ {{- toYaml . | nindent 8 }}
+ {{- end }}
+ {{- with .Values.interpreter.spark.annotations }}
+ annotations:
+ {{- toYaml . | nindent 8 }}
+ {{- end }}
+ spec:
+ containers:
+ - name: spark-kubernetes-driver
+ {{- with .Values.interpreter.spark.env }}
+ env:
+ {{- toYaml . | nindent 8 }}
+ {{- end }}
+ {{- with .Values.interpreter.spark.envFrom }}
+ envFrom:
+ {{- toYaml . | nindent 8 }}
+ {{- end }}
+ {{- with .Values.interpreter.spark.volumeMounts }}
+ volumeMounts:
+ {{- toYaml . | nindent 8 }}
+ {{- end }}
+ {{- with .Values.interpreter.spark.volumes }}
+ volumes:
+ {{- toYaml . | nindent 8 }}
+ {{- end }}
+ {{- with .Values.interpreter.nodeSelector }}
+ nodeSelector:
+ {{- toYaml . | nindent 8 }}
+ {{- end }}
+ {{- with .Values.interpreter.affinity }}
+ affinity:
+ {{- toYaml . | nindent 8 }}
+ {{- end }}
+ {{- with .Values.interpreter.tolerations }}
+ tolerations:
+ {{- toYaml . | nindent 8 }}
+ {{- end }}
+ {{- with .Values.interpreter.podSecurityContext }}
+ securityContext:
+ {{- toYaml . | nindent 8 }}
+ {{- end }}
+
+ executor-pod-template.yaml: |
Review Comment:
This configuration could be used in the Spark driver. I have not yet tried
such a template. I currently configure the executors and drivers via the
Spark-Config.
##########
charts/zeppelin/templates/interpreter/role.yaml:
##########
@@ -0,0 +1,47 @@
+{{- /*
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/ -}}
+
+apiVersion: rbac.authorization.k8s.io/v1
Review Comment:
Please note that Role, ServiceAccount and Role binding are only required for
the Spark interpreter. How is this implemented?
##########
charts/zeppelin/files/conf/interpreter-list:
##########
@@ -0,0 +1,37 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+#
+# [name] [maven artifact] [description]
Review Comment:
For which functionality is this file required?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
