This is an automated email from the ASF dual-hosted git repository.
nicholasjiang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/celeborn.git
The following commit(s) were added to refs/heads/main by this push:
new 835ee978c [CELEBORN-2202] Add configmap checksum to the statefulset
annotation
835ee978c is described below
commit 835ee978c0acb1168754b1f4821011a77ba9fd80
Author: pingzh <[email protected]>
AuthorDate: Wed Nov 12 10:19:38 2025 +0800
[CELEBORN-2202] Add configmap checksum to the statefulset annotation
### What changes were proposed in this pull request?
- Add the `checksum/celeborn_config` annotation to both master and worker's
statefulset:
```
annotations:
celeborn.apache.org/conf-hash:
0a0d909a0539b19a6b70f6a8df315a22eb12b5b4c3f59457d93b10405ae685f6
```
- sort the keys of celeborn config in the config map to produce consistent
checksum
### Why are the changes needed?
When the configs in `celeborn` are changed, helm can restart the celeborn
service.
### Does this PR resolve a correctness bug?
No.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
1. helm unittest
```
~/Documents/repos/opensource/celeborn/charts/celeborn
❯ helm unittest . --file "tests/**/*_test.yaml" --strict --debug
### Chart [ celeborn ] .
PASS Test Celeborn configmap tests/configmap_test.yaml
PASS Test Celeborn role tests/role_test.yaml
PASS Test Celeborn rolebinding tests/rolebinding_test.yaml
PASS Test Celeborn service account tests/serviceaccount_test.yaml
PASS Test Celeborn master pod monitor tests/master/podmonitor_test.yaml
PASS Test Celeborn master priority class
tests/master/priorityclass_test.yaml
PASS Test Celeborn master service tests/master/service_test.yaml
PASS Test Celeborn master statefulset tests/master/statefulset_test.yaml
PASS Test Celeborn worker pod monitor tests/worker/podmonitor_test.yaml
PASS Test Celeborn worker priority class
tests/worker/priorityclass_test.yaml
PASS Test Celeborn worker service tests/worker/service_test.yaml
PASS Test Celeborn worker statefulset tests/worker/statefulset_test.yaml
Charts: 1 passed, 1 total
Test Suites: 12 passed, 12 total
Tests: 75 passed, 75 total
Snapshot: 0 passed, 0 total
Time: 107.537875ms
```
2. `helm template --debug ../celeborn`
<img width="2316" height="1184" alt="image"
src="https://github.com/user-attachments/assets/84297397-b008-40d2-a6a9-209664313b9b";
/>
<img width="2532" height="1430" alt="image"
src="https://github.com/user-attachments/assets/74f0b495-88a1-404f-ba09-647b1c306a69";
/>
Closes #3535 from pingzh/configmap-sha.
Authored-by: pingzh <[email protected]>
Signed-off-by: SteNicholas <[email protected]>
---
charts/celeborn/templates/_helpers.tpl | 5 +++
charts/celeborn/templates/master/statefulset.yaml | 6 ++--
charts/celeborn/templates/worker/statefulset.yaml | 5 +--
charts/celeborn/tests/master/statefulset_test.yaml | 41 +++++++++++++++++++++
charts/celeborn/tests/worker/statefulset_test.yaml | 42 ++++++++++++++++++++++
5 files changed, 94 insertions(+), 5 deletions(-)
diff --git a/charts/celeborn/templates/_helpers.tpl
b/charts/celeborn/templates/_helpers.tpl
index a14b49f4f..81ccbc60f 100644
--- a/charts/celeborn/templates/_helpers.tpl
+++ b/charts/celeborn/templates/_helpers.tpl
@@ -101,6 +101,11 @@ Create the name of the roleBinding to use
{{ include "celeborn.fullname" . }}-conf
{{- end -}}
+{{/* Checksum of the Celeborn ConfigMap for rollout triggers. */}}
+{{- define "celeborn.configChecksum" -}}
+{{ include (print .Template.BasePath "/configmap.yaml") . | sha256sum }}
+{{- end -}}
+
{{/*
Create the name of the celeborn image to use
*/}}
diff --git a/charts/celeborn/templates/master/statefulset.yaml
b/charts/celeborn/templates/master/statefulset.yaml
index 5b9e876fd..e962b2a5e 100644
--- a/charts/celeborn/templates/master/statefulset.yaml
+++ b/charts/celeborn/templates/master/statefulset.yaml
@@ -37,10 +37,11 @@ spec:
labels:
{{- include "celeborn.master.selectorLabels" . | nindent 8 }}
app.kubernetes.io/tag: {{ .Values.image.tag | quote }}
- {{- with .Values.master.annotations }}
annotations:
+ celeborn.apache.org/conf-hash: {{ include "celeborn.configChecksum" .
}}
+ {{- with .Values.master.annotations }}
{{- toYaml . | nindent 8 }}
- {{- end }}
+ {{- end }}
spec:
{{- /* Add an init container to chown mount paths of Celeborn master
volumes if necessary. */}}
{{- $paths := list }}
@@ -178,4 +179,3 @@ spec:
{{- toYaml . | nindent 8 }}
{{- end }}
terminationGracePeriodSeconds: 30
-
diff --git a/charts/celeborn/templates/worker/statefulset.yaml
b/charts/celeborn/templates/worker/statefulset.yaml
index 0b2d7d9b9..a74571c62 100644
--- a/charts/celeborn/templates/worker/statefulset.yaml
+++ b/charts/celeborn/templates/worker/statefulset.yaml
@@ -37,10 +37,11 @@ spec:
labels:
{{- include "celeborn.worker.selectorLabels" . | nindent 8 }}
app.kubernetes.io/tag: {{ .Values.image.tag | quote }}
- {{- with .Values.worker.annotations }}
annotations:
+ celeborn.apache.org/conf-hash: {{ include "celeborn.configChecksum" .
}}
+ {{- with .Values.worker.annotations }}
{{- toYaml . | nindent 8 }}
- {{- end }}
+ {{- end }}
spec:
{{- /* Add an init container to chown mount paths of Celeborn workers
volume if necessary. */}}
{{- $paths := list }}
diff --git a/charts/celeborn/tests/master/statefulset_test.yaml
b/charts/celeborn/tests/master/statefulset_test.yaml
index 8117698c4..09ef9c1d3 100644
--- a/charts/celeborn/tests/master/statefulset_test.yaml
+++ b/charts/celeborn/tests/master/statefulset_test.yaml
@@ -18,6 +18,7 @@
suite: Test Celeborn master statefulset
templates:
+ - configmap.yaml
- master/statefulset.yaml
release:
@@ -26,6 +27,7 @@ release:
tests:
- it: Should create master statefulset
+ template: master/statefulset.yaml
asserts:
- containsDocument:
apiVersion: apps/v1
@@ -33,7 +35,28 @@ tests:
name: celeborn-master
namespace: celeborn
+ - it: Should add checksum annotation referencing configmap
+ template: master/statefulset.yaml
+ asserts:
+ - equal:
+ path:
spec.template.metadata.annotations["celeborn.apache.org/conf-hash"]
+ value:
cd692d34c71437aec034c5a08728570eba1b8f5f2b20a36b153e1cd92a399481
+
+ - it: Should change checksum annotation when celeborn config changes
+ template: master/statefulset.yaml
+ set:
+ celeborn:
+ "celeborn.rpc.io.serverThreads": 128
+ asserts:
+ - notEqual:
+ path:
spec.template.metadata.annotations["celeborn.apache.org/conf-hash"]
+ value:
cd692d34c71437aec034c5a08728570eba1b8f5f2b20a36b153e1cd92a399481
+ - equal:
+ path:
spec.template.metadata.annotations["celeborn.apache.org/conf-hash"]
+ value:
fa6255908e6daaba8d190cb61ea2fa3f57c554279ae0dc753e56008158080d13
+
- it: Should add extra pod annotations if `master.annotations` is specified
+ template: master/statefulset.yaml
set:
master:
annotations:
@@ -48,6 +71,7 @@ tests:
value: value2
- it: Should use the specified image if `image.registry`, `image.repository`
and `image.tag` is set
+ template: master/statefulset.yaml
set:
image:
registry: test-registry
@@ -59,6 +83,7 @@ tests:
value: test-registry/test-repository:test-tag
- it: Should use the specified image pull policy if `image.pullPolicy` is set
+ template: master/statefulset.yaml
set:
image:
pullPolicy: Always
@@ -68,6 +93,7 @@ tests:
value: Always
- it: Should use the specified replicas if `master.replicas` is set
+ template: master/statefulset.yaml
set:
master:
replicas: 10
@@ -77,6 +103,7 @@ tests:
value: 10
- it: Should add environment variables if `master.env` is set
+ template: master/statefulset.yaml
set:
master:
env:
@@ -120,6 +147,7 @@ tests:
optional: false
- it: Should add environment variable sources if `master.envFrom` is set
+ template: master/statefulset.yaml
set:
master:
envFrom:
@@ -144,6 +172,7 @@ tests:
optional: false
- it: Should add volume mounts if `master.volumeMounts` is set
+ template: master/statefulset.yaml
set:
master:
volumeMounts:
@@ -164,6 +193,7 @@ tests:
mountPath: /mnt/disk1
- it: Should use the specified resources if `master.resources` is set
+ template: master/statefulset.yaml
set:
master:
resources:
@@ -194,6 +224,7 @@ tests:
memory: 128Mi
- it: Should add container securityContext if `master.securityContext` is set
+ template: master/statefulset.yaml
set:
master:
securityContext:
@@ -223,6 +254,7 @@ tests:
privileged: false
- it: Should add secrets if `imagePullSecrets` is set
+ template: master/statefulset.yaml
set:
imagePullSecrets:
- name: test-secret1
@@ -236,6 +268,7 @@ tests:
value: test-secret2
- it: Should add volumes if `master.volumes` is set
+ template: master/statefulset.yaml
set:
master:
volumes:
@@ -262,6 +295,7 @@ tests:
path: /mnt/disk2
- it: Should add node selector if `master.nodeSelector` is set
+ template: master/statefulset.yaml
set:
master:
nodeSelector:
@@ -276,6 +310,7 @@ tests:
value: value2
- it: Should add tolerations if `master.tolerations` is set
+ template: master/statefulset.yaml
set:
master:
tolerations:
@@ -299,6 +334,7 @@ tests:
effect: NoSchedule
- it: Should use the specified affinity if `master.affinity` is set
+ template: master/statefulset.yaml
set:
master:
affinity:
@@ -341,6 +377,7 @@ tests:
- another-node-label-value
- it: Should use the specified priority class name if
`master.priorityClass.name` is set
+ template: master/statefulset.yaml
set:
master:
priorityClass:
@@ -351,6 +388,7 @@ tests:
value: test-priority-class
- it: Should use the specified dns policy if `master.dnsPolicy` is set
+ template: master/statefulset.yaml
set:
master:
dnsPolicy: ClusterFirstWithHostNet
@@ -360,6 +398,7 @@ tests:
value: ClusterFirstWithHostNet
- it: Should enable host network if `master.hostNetwork` is set to true
+ template: master/statefulset.yaml
set:
master:
hostNetwork: true
@@ -369,6 +408,7 @@ tests:
value: true
- it: Should use the specified security context if
`master.podSecurityContext` is set
+ template: master/statefulset.yaml
set:
master:
podSecurityContext:
@@ -387,6 +427,7 @@ tests:
value: 3000
- it: Should add volume claim templates if `master.volumeClaimTemplates` is
set
+ template: master/statefulset.yaml
set:
master:
volumeClaimTemplates:
diff --git a/charts/celeborn/tests/worker/statefulset_test.yaml
b/charts/celeborn/tests/worker/statefulset_test.yaml
index e232335dc..d97b57009 100644
--- a/charts/celeborn/tests/worker/statefulset_test.yaml
+++ b/charts/celeborn/tests/worker/statefulset_test.yaml
@@ -18,6 +18,7 @@
suite: Test Celeborn worker statefulset
templates:
+ - configmap.yaml
- worker/statefulset.yaml
release:
@@ -26,6 +27,7 @@ release:
tests:
- it: Should create worker statefulset
+ template: worker/statefulset.yaml
asserts:
- containsDocument:
apiVersion: apps/v1
@@ -33,7 +35,28 @@ tests:
name: celeborn-worker
namespace: celeborn
+ - it: Should add checksum annotation referencing configmap
+ template: worker/statefulset.yaml
+ asserts:
+ - equal:
+ path:
spec.template.metadata.annotations["celeborn.apache.org/conf-hash"]
+ value:
cd692d34c71437aec034c5a08728570eba1b8f5f2b20a36b153e1cd92a399481
+
+ - it: Should change checksum annotation when celeborn config changes
+ template: worker/statefulset.yaml
+ set:
+ celeborn:
+ "celeborn.rpc.io.serverThreads": 128
+ asserts:
+ - notEqual:
+ path:
spec.template.metadata.annotations["celeborn.apache.org/conf-hash"]
+ value:
cd692d34c71437aec034c5a08728570eba1b8f5f2b20a36b153e1cd92a399481
+ - equal:
+ path:
spec.template.metadata.annotations["celeborn.apache.org/conf-hash"]
+ value:
fa6255908e6daaba8d190cb61ea2fa3f57c554279ae0dc753e56008158080d13
+
- it: Should add extra pod annotations if `worker.annotations` is specified
+ template: worker/statefulset.yaml
set:
worker:
annotations:
@@ -48,6 +71,7 @@ tests:
value: value2
- it: Should use the specified image if `image.registry`, `image.repository`
and `image.tag` is set
+ template: worker/statefulset.yaml
set:
image:
registry: test-registry
@@ -59,6 +83,7 @@ tests:
value: test-registry/test-repository:test-tag
- it: Should use the specified image pull policy if `image.pullPolicy` is set
+ template: worker/statefulset.yaml
set:
image:
pullPolicy: Always
@@ -68,6 +93,7 @@ tests:
value: Always
- it: Should use the specified replicas if `worker.replicas` is set
+ template: worker/statefulset.yaml
set:
worker.replicas: 10
asserts:
@@ -76,6 +102,7 @@ tests:
value: 10
- it: Should use the specified terminationGracePeriodSeconds if
`worker.terminationGracePeriodSeconds` is set
+ template: worker/statefulset.yaml
set:
worker.terminationGracePeriodSeconds: 70
asserts:
@@ -84,6 +111,7 @@ tests:
value: 70
- it: Should add environment variables if `worker.env` is set
+ template: worker/statefulset.yaml
set:
worker:
env:
@@ -127,6 +155,7 @@ tests:
optional: false
- it: Should add environment variable sources if `worker.envFrom` is set
+ template: worker/statefulset.yaml
set:
worker:
envFrom:
@@ -151,6 +180,7 @@ tests:
optional: false
- it: Should add volume mounts if `worker.volumeMounts` is set
+ template: worker/statefulset.yaml
set:
worker:
volumeMounts:
@@ -171,6 +201,7 @@ tests:
mountPath: /mnt/disk1
- it: Should use the specified resources if `worker.resources` is set
+ template: worker/statefulset.yaml
set:
worker:
resources:
@@ -201,6 +232,7 @@ tests:
memory: 128Mi
- it: Should add container securityContext if `worker.securityContext` is set
+ template: worker/statefulset.yaml
set:
worker:
securityContext:
@@ -230,6 +262,7 @@ tests:
privileged: false
- it: Should add secrets if `imagePullSecrets` is set
+ template: worker/statefulset.yaml
set:
imagePullSecrets:
- name: test-secret1
@@ -243,6 +276,7 @@ tests:
value: test-secret2
- it: Should add volumes if `worker.volumes` is set
+ template: worker/statefulset.yaml
set:
worker:
volumes:
@@ -269,6 +303,7 @@ tests:
path: /mnt/disk2
- it: Should add node selector if `worker.nodeSelector` is set
+ template: worker/statefulset.yaml
set:
worker:
nodeSelector:
@@ -283,6 +318,7 @@ tests:
value: value2
- it: Should add tolerations if `worker.tolerations` is set
+ template: worker/statefulset.yaml
set:
worker:
tolerations:
@@ -306,6 +342,7 @@ tests:
effect: NoSchedule
- it: Should use the specified affinity if `worker.affinity` is specified
+ template: worker/statefulset.yaml
set:
worker:
affinity:
@@ -348,6 +385,7 @@ tests:
- another-node-label-value
- it: Should use the specified priority class name if
`worker.priorityClass.name` is set
+ template: worker/statefulset.yaml
set:
worker:
priorityClass:
@@ -358,6 +396,7 @@ tests:
value: test-priority-class
- it: Should use the specified dns policy if `worker.dnsPolicy` is set
+ template: worker/statefulset.yaml
set:
worker:
dnsPolicy: ClusterFirstWithHostNet
@@ -367,6 +406,7 @@ tests:
value: ClusterFirstWithHostNet
- it: Should enable host network if `worker.hostNetwork` is set to true
+ template: worker/statefulset.yaml
set:
worker:
hostNetwork: true
@@ -376,6 +416,7 @@ tests:
value: true
- it: Should use the specified security context if
`worker.podSecurityContext` is set
+ template: worker/statefulset.yaml
set:
worker:
podSecurityContext:
@@ -394,6 +435,7 @@ tests:
value: 3000
- it: Should add volume claim templates if `worker.volumeClaimTemplates` is
set
+ template: worker/statefulset.yaml
set:
worker:
volumeClaimTemplates:
