This is an automated email from the ASF dual-hosted git repository.
zaynt4606 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/celeborn.git
The following commit(s) were added to refs/heads/main by this push:
new ec098ee8db [CELEBORN-2446] Support topologySpreadConstraints for
master and worker pods in Helm chart
ec098ee8db is described below
commit ec098ee8db8843721ef1e6e7c09aee2cbccd2e42
Author: strelok89 <[email protected]>
AuthorDate: Mon Oct 12 19:36:28 2026 +0800
[CELEBORN-2446] Support topologySpreadConstraints for master and worker
pods in Helm chart
### What changes were proposed in this pull request?
Add support for `topologySpreadConstraints` on Celeborn master and worker
pods in the Helm chart:
- `charts/celeborn/templates/master/statefulset.yaml` and
`charts/celeborn/templates/worker/statefulset.yaml` — render
`topologySpreadConstraints` from the corresponding value, following the
existing `{{- with }}` idiom already used for `nodeSelector`, `affinity` and
`tolerations`.
- `charts/celeborn/values.yaml` — declare `topologySpreadConstraints: []`
for both roles.
- `charts/celeborn/tests/{master,worker}/statefulset_test.yaml` — unit
tests covering both the unset (field absent) and set (field rendered) cases.
Example usage:
```yaml
worker:
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app.kubernetes.io/name: celeborn
app.kubernetes.io/role: worker
```
### Why are the changes needed?
The chart already exposes `nodeSelector`, `affinity`, `tolerations` and
`priorityClass` for both master and worker pods, but not
`topologySpreadConstraints`. Setting the value today is silently ignored — Helm
accepts `worker.topologySpreadConstraints` without error and nothing is
rendered on the StatefulSet, so the constraint simply does not exist.
`podAntiAffinity` — which the chart defaults to, on
`kubernetes.io/hostname` — cannot substitute for a multi-AZ deployment. Pod
anti-affinity is one-or-none per topology domain: it can express "at most one
pod per zone", but not "N pods per zone" or "spread evenly across zones within
a bounded skew". That is what `topologySpreadConstraints` is for.
This also lines up with item 8, "Enhanced K8S support", on the improvements
schedule in `CONTRIBUTING.md`.
### Does this PR resolve a correctness bug?
- [ ] Yes
### Does this PR introduce _any_ user-facing change?
- [x] Yes
Two new chart values, `master.topologySpreadConstraints` and
`worker.topologySpreadConstraints`, both defaulting to `[]`. When they are
unset the rendered StatefulSet is unchanged, so this is purely additive.
### How was this patch tested?
- `helm unittest charts/celeborn --file "tests/**/*_test.yaml" --strict` —
all chart unit tests pass, including the four new cases. Reverting only the
template hunks makes exactly those new assertions fail, confirming they
exercise the change.
- `helm lint charts/celeborn` — passes.
- `helm template` with a zone-spread constraint set renders
`topologySpreadConstraints` on the worker StatefulSet; `helm template` with
default values renders no `topologySpreadConstraints` key anywhere.
Closes #3831 from strelok89/helm-topology-spread-constraints.
Authored-by: strelok89 <[email protected]>
Signed-off-by: zhengtao <[email protected]>
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Co-Authored-By: Codex <[email protected]>
AI-Model: gpt-6
AI-Contributed/Feature: 0/14
AI-Contributed/UT: 0/60
---
charts/celeborn/templates/master/statefulset.yaml | 4 +++
charts/celeborn/templates/worker/statefulset.yaml | 4 +++
charts/celeborn/tests/master/statefulset_test.yaml | 30 ++++++++++++++++++++++
charts/celeborn/tests/worker/statefulset_test.yaml | 30 ++++++++++++++++++++++
charts/celeborn/values.yaml | 6 +++++
5 files changed, 74 insertions(+)
diff --git a/charts/celeborn/templates/master/statefulset.yaml
b/charts/celeborn/templates/master/statefulset.yaml
index 987d0aa286..d2ea5175e4 100644
--- a/charts/celeborn/templates/master/statefulset.yaml
+++ b/charts/celeborn/templates/master/statefulset.yaml
@@ -165,6 +165,10 @@ spec:
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
+ {{- with .Values.master.topologySpreadConstraints }}
+ topologySpreadConstraints:
+ {{- toYaml . | nindent 8 }}
+ {{- end }}
{{- with .Values.master.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
diff --git a/charts/celeborn/templates/worker/statefulset.yaml
b/charts/celeborn/templates/worker/statefulset.yaml
index a3f73c96a4..6b7e340619 100644
--- a/charts/celeborn/templates/worker/statefulset.yaml
+++ b/charts/celeborn/templates/worker/statefulset.yaml
@@ -164,6 +164,10 @@ spec:
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
+ {{- with .Values.worker.topologySpreadConstraints }}
+ topologySpreadConstraints:
+ {{- toYaml . | nindent 8 }}
+ {{- end }}
{{- with .Values.worker.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
diff --git a/charts/celeborn/tests/master/statefulset_test.yaml
b/charts/celeborn/tests/master/statefulset_test.yaml
index 88f6c51fc2..01a9f40234 100644
--- a/charts/celeborn/tests/master/statefulset_test.yaml
+++ b/charts/celeborn/tests/master/statefulset_test.yaml
@@ -309,6 +309,36 @@ tests:
path: spec.template.spec.nodeSelector.key2
value: value2
+ - it: Should not add topology spread constraints if
`master.topologySpreadConstraints` is not set
+ template: master/statefulset.yaml
+ asserts:
+ - notExists:
+ path: spec.template.spec.topologySpreadConstraints
+
+ - it: Should add topology spread constraints if
`master.topologySpreadConstraints` is set
+ template: master/statefulset.yaml
+ set:
+ master:
+ topologySpreadConstraints:
+ - maxSkew: 1
+ topologyKey: topology.kubernetes.io/zone
+ whenUnsatisfiable: DoNotSchedule
+ labelSelector:
+ matchLabels:
+ app.kubernetes.io/name: celeborn
+ app.kubernetes.io/role: master
+ asserts:
+ - equal:
+ path: spec.template.spec.topologySpreadConstraints
+ value:
+ - maxSkew: 1
+ topologyKey: topology.kubernetes.io/zone
+ whenUnsatisfiable: DoNotSchedule
+ labelSelector:
+ matchLabels:
+ app.kubernetes.io/name: celeborn
+ app.kubernetes.io/role: master
+
- it: Should add tolerations if `master.tolerations` is set
template: master/statefulset.yaml
set:
diff --git a/charts/celeborn/tests/worker/statefulset_test.yaml
b/charts/celeborn/tests/worker/statefulset_test.yaml
index f586f13557..ba2b23b904 100644
--- a/charts/celeborn/tests/worker/statefulset_test.yaml
+++ b/charts/celeborn/tests/worker/statefulset_test.yaml
@@ -317,6 +317,36 @@ tests:
path: spec.template.spec.nodeSelector.key2
value: value2
+ - it: Should not add topology spread constraints if
`worker.topologySpreadConstraints` is not set
+ template: worker/statefulset.yaml
+ asserts:
+ - notExists:
+ path: spec.template.spec.topologySpreadConstraints
+
+ - it: Should add topology spread constraints if
`worker.topologySpreadConstraints` is set
+ template: worker/statefulset.yaml
+ set:
+ worker:
+ topologySpreadConstraints:
+ - maxSkew: 1
+ topologyKey: topology.kubernetes.io/zone
+ whenUnsatisfiable: DoNotSchedule
+ labelSelector:
+ matchLabels:
+ app.kubernetes.io/name: celeborn
+ app.kubernetes.io/role: worker
+ asserts:
+ - equal:
+ path: spec.template.spec.topologySpreadConstraints
+ value:
+ - maxSkew: 1
+ topologyKey: topology.kubernetes.io/zone
+ whenUnsatisfiable: DoNotSchedule
+ labelSelector:
+ matchLabels:
+ app.kubernetes.io/name: celeborn
+ app.kubernetes.io/role: worker
+
- it: Should add tolerations if `worker.tolerations` is set
template: worker/statefulset.yaml
set:
diff --git a/charts/celeborn/values.yaml b/charts/celeborn/values.yaml
index 4265101e45..4bdabfde66 100644
--- a/charts/celeborn/values.yaml
+++ b/charts/celeborn/values.yaml
@@ -171,6 +171,9 @@ master:
- master
topologyKey: kubernetes.io/hostname
+ # -- Topology spread constraints for Celeborn master pods.
+ topologySpreadConstraints: []
+
# -- Tolerations for Celeborn master pods.
tolerations:
# - key: key1
@@ -323,6 +326,9 @@ worker:
- worker
topologyKey: kubernetes.io/hostname
+ # -- Topology spread constraints for Celeborn worker pods.
+ topologySpreadConstraints: []
+
# -- Tolerations for Celeborn worker pods.
tolerations:
# - key: key1
