This is an automated email from the ASF dual-hosted git repository.
SbloodyS pushed a commit to branch dev
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
The following commit(s) were added to refs/heads/dev by this push:
new 5b54085861 [Improvement-16390][Helm Chart] Allow Hadoop configuration
mounts in API (#18639)
5b54085861 is described below
commit 5b54085861fd0931da527b7216a838e285aafd0b
Author: Goutam Adwant <[email protected]>
AuthorDate: Wed Sep 16 19:50:30 2026 -0700
[Improvement-16390][Helm Chart] Allow Hadoop configuration mounts in API
(#18639)
Co-authored-by: xiangzihao <[email protected]>
---
deploy/kubernetes/dolphinscheduler/README.md | 45 ++++++++++++++++++++++
.../kubernetes/dolphinscheduler/README.md.gotmpl | 41 ++++++++++++++++++++
.../templates/deployment-dolphinscheduler-api.yaml | 6 +++
deploy/kubernetes/dolphinscheduler/values.yaml | 8 ++++
4 files changed, 100 insertions(+)
diff --git a/deploy/kubernetes/dolphinscheduler/README.md
b/deploy/kubernetes/dolphinscheduler/README.md
index 7e4bc0411e..eff3bfa053 100644
--- a/deploy/kubernetes/dolphinscheduler/README.md
+++ b/deploy/kubernetes/dolphinscheduler/README.md
@@ -8,6 +8,47 @@ This chart bootstraps all the components needed to run Apache
DolphinScheduler o
Please refer to the [Quick Start in
Kubernetes](../../../docs/docs/en/guide/installation/kubernetes.md)
+## HDFS HA configuration
+
+For deployments configured to use HDFS, mount an existing ConfigMap containing
`core-site.xml` and `hdfs-site.xml` into both the API and Worker containers to
supply HA settings. The ConfigMap must be in the same namespace as
DolphinScheduler.
+
+For example, with a ConfigMap named `hadoop-ha-config`, add the following
values:
+
+```yaml
+api:
+ extraVolumes:
+ - name: hadoop-ha-config
+ configMap:
+ name: hadoop-ha-config
+ extraVolumeMounts:
+ - name: hadoop-ha-config
+ mountPath: /opt/dolphinscheduler/api-server/conf/core-site.xml
+ subPath: core-site.xml
+ readOnly: true
+ - name: hadoop-ha-config
+ mountPath: /opt/dolphinscheduler/api-server/conf/hdfs-site.xml
+ subPath: hdfs-site.xml
+ readOnly: true
+worker:
+ extraVolumes:
+ - name: hadoop-ha-config
+ configMap:
+ name: hadoop-ha-config
+ extraVolumeMounts:
+ - name: hadoop-ha-config
+ mountPath: /opt/dolphinscheduler/worker-server/conf/core-site.xml
+ subPath: core-site.xml
+ readOnly: true
+ - name: hadoop-ha-config
+ mountPath: /opt/dolphinscheduler/worker-server/conf/hdfs-site.xml
+ subPath: hdfs-site.xml
+ readOnly: true
+```
+
+Configure `fs.defaultFS` with the logical nameservice URI (for example,
`hdfs://mycluster`) and include the corresponding HA settings in the XML files.
Keep DolphinScheduler's filesystem configuration consistent with that URI.
+
+The paths above match the API and Worker server classpaths. Check the
configuration classpath of a custom image and adjust the mount paths if needed.
Individual file mounts preserve the other packaged configuration files. Restart
API and Worker pods after changing the ConfigMap: Kubernetes does not update
ConfigMap files mounted using `subPath`.
+
## Values
| Key | Type | Default | Description |
@@ -58,6 +99,8 @@ Please refer to the [Quick Start in
Kubernetes](../../../docs/docs/en/guide/inst
| api.enableCustomizedConfig | bool | `false` | enable configure custom config
|
| api.enabled | bool | `true` | Enable or disable the API-Server component |
| api.env.JAVA_OPTS | string | `"-Xms512m -Xmx512m -Xmn256m"` | The jvm
options for api server |
+| api.extraVolumeMounts | list | `[]` | Additional volume mounts for the API
container |
+| api.extraVolumes | list | `[]` | Additional volumes for the API container |
| api.livenessProbe | object |
`{"enabled":true,"failureThreshold":"3","initialDelaySeconds":"30","periodSeconds":"30","successThreshold":"1","timeoutSeconds":"5"}`
| Periodic probe of container liveness. Container will be restarted if the
probe fails. More info:
[container-probes](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#container-probes)
|
| api.livenessProbe.enabled | bool | `true` | Turn on and off liveness probe |
| api.livenessProbe.failureThreshold | string | `"3"` | Minimum consecutive
failures for the probe |
@@ -335,6 +378,8 @@ Please refer to the [Quick Start in
Kubernetes](../../../docs/docs/en/guide/inst
| worker.env.WORKER_TENANT_CONFIG_AUTO_CREATE_TENANT_ENABLED | bool | `true` |
tenant corresponds to the user of the system, which is used by the worker to
submit the job. If system does not have this user, it will be automatically
created after the parameter worker.tenant.auto.create is true. |
| worker.env.WORKER_TENANT_CONFIG_DEFAULT_TENANT_ENABLED | bool | `false` | If
set true, will use worker bootstrap user as the tenant to execute task when the
tenant is `default`; |
| worker.envFromSecret | string | `""` | Direct Secret Mounting Mount secrets
directly as environment variables Single secret |
+| worker.extraVolumeMounts | list | `[]` | Additional volume mounts for the
Worker container |
+| worker.extraVolumes | list | `[]` | Additional volumes for the Worker
container |
| worker.initContainers | object | `{}` | Init Container for Advanced
Processing Use when you need to transform, validate, or prepare configuration
files |
| worker.keda.advanced | object | `{}` | Specify HPA related options |
| worker.keda.cooldownPeriod | int | `30` | How many seconds KEDA will wait
before scaling to zero. Note that HPA has a separate cooldown period for
scale-downs |
diff --git a/deploy/kubernetes/dolphinscheduler/README.md.gotmpl
b/deploy/kubernetes/dolphinscheduler/README.md.gotmpl
index f82b86d205..34332aa04e 100644
--- a/deploy/kubernetes/dolphinscheduler/README.md.gotmpl
+++ b/deploy/kubernetes/dolphinscheduler/README.md.gotmpl
@@ -8,4 +8,45 @@ This chart bootstraps all the components needed to run Apache
DolphinScheduler o
Please refer to the [Quick Start in
Kubernetes](../../../docs/docs/en/guide/installation/kubernetes.md)
+## HDFS HA configuration
+
+For deployments configured to use HDFS, mount an existing ConfigMap containing
`core-site.xml` and `hdfs-site.xml` into both the API and Worker containers to
supply HA settings. The ConfigMap must be in the same namespace as
DolphinScheduler.
+
+For example, with a ConfigMap named `hadoop-ha-config`, add the following
values:
+
+```yaml
+api:
+ extraVolumes:
+ - name: hadoop-ha-config
+ configMap:
+ name: hadoop-ha-config
+ extraVolumeMounts:
+ - name: hadoop-ha-config
+ mountPath: /opt/dolphinscheduler/api-server/conf/core-site.xml
+ subPath: core-site.xml
+ readOnly: true
+ - name: hadoop-ha-config
+ mountPath: /opt/dolphinscheduler/api-server/conf/hdfs-site.xml
+ subPath: hdfs-site.xml
+ readOnly: true
+worker:
+ extraVolumes:
+ - name: hadoop-ha-config
+ configMap:
+ name: hadoop-ha-config
+ extraVolumeMounts:
+ - name: hadoop-ha-config
+ mountPath: /opt/dolphinscheduler/worker-server/conf/core-site.xml
+ subPath: core-site.xml
+ readOnly: true
+ - name: hadoop-ha-config
+ mountPath: /opt/dolphinscheduler/worker-server/conf/hdfs-site.xml
+ subPath: hdfs-site.xml
+ readOnly: true
+```
+
+Configure `fs.defaultFS` with the logical nameservice URI (for example,
`hdfs://mycluster`) and include the corresponding HA settings in the XML files.
Keep DolphinScheduler's filesystem configuration consistent with that URI.
+
+The paths above match the API and Worker server classpaths. Check the
configuration classpath of a custom image and adjust the mount paths if needed.
Individual file mounts preserve the other packaged configuration files. Restart
API and Worker pods after changing the ConfigMap: Kubernetes does not update
ConfigMap files mounted using `subPath`.
+
{{ template "chart.valuesSection" . }}
diff --git
a/deploy/kubernetes/dolphinscheduler/templates/deployment-dolphinscheduler-api.yaml
b/deploy/kubernetes/dolphinscheduler/templates/deployment-dolphinscheduler-api.yaml
index 84abc380d3..5fc8bac9de 100644
---
a/deploy/kubernetes/dolphinscheduler/templates/deployment-dolphinscheduler-api.yaml
+++
b/deploy/kubernetes/dolphinscheduler/templates/deployment-dolphinscheduler-api.yaml
@@ -178,6 +178,9 @@ spec:
{{- include "dolphinscheduler.fsFileResource.volumeMount" . |
nindent 12 }}
{{- include "dolphinscheduler.ldap.ssl.volumeMount" . | nindent 12
}}
{{- include "dolphinscheduler.etcd.ssl.volumeMount" . | nindent 12
}}
+ {{- if .Values.api.extraVolumeMounts }}
+ {{- toYaml .Values.api.extraVolumeMounts | nindent 12 }}
+ {{- end }}
volumes:
- name: {{ include "dolphinscheduler.fullname" . }}-api
{{- if .Values.api.persistentVolumeClaim.enabled }}
@@ -198,4 +201,7 @@ spec:
{{- include "dolphinscheduler.fsFileResource.volume" . | nindent 8 }}
{{- include "dolphinscheduler.ldap.ssl.volume" . | nindent 8 }}
{{- include "dolphinscheduler.etcd.ssl.volume" . | nindent 8 }}
+ {{- if .Values.api.extraVolumes }}
+ {{- toYaml .Values.api.extraVolumes | nindent 8 }}
+ {{- end }}
{{- end }}
diff --git a/deploy/kubernetes/dolphinscheduler/values.yaml
b/deploy/kubernetes/dolphinscheduler/values.yaml
index c5d0fdc4b3..533da1f2b3 100644
--- a/deploy/kubernetes/dolphinscheduler/values.yaml
+++ b/deploy/kubernetes/dolphinscheduler/values.yaml
@@ -598,6 +598,10 @@ master:
annotations: {}
worker:
+ # -- Additional volumes for the Worker container
+ extraVolumes: []
+ # -- Additional volume mounts for the Worker container
+ extraVolumeMounts: []
# -- Enable or disable the Worker component
enabled: true
# -- PodManagementPolicy controls how pods are created during initial scale
up, when replacing pods on nodes, or when scaling down.
@@ -916,6 +920,10 @@ alert:
api:
+ # -- Additional volumes for the API container
+ extraVolumes: []
+ # -- Additional volume mounts for the API container
+ extraVolumeMounts: []
# -- Enable or disable the API-Server component
enabled: true
# -- Number of desired pods. This is a pointer to distinguish between
explicit zero and not specified. Defaults to 1.
