This is an automated email from the ASF dual-hosted git repository.
shahar1 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 6c8902b1ac5 Document per-set volumes for Celery workers (#74913)
6c8902b1ac5 is described below
commit 6c8902b1ac549f5cfbec645a5416f54973e9ff2c
Author: Himanshu Chawla <[email protected]>
AuthorDate: Thu Oct 22 10:10:29 2026 +0530
Document per-set volumes for Celery workers (#74913)
Clarify multi-node PVC requirements for the inherited replica and
autoscaling settings.
Generated-by: OpenAI Codex
---
chart/docs/customizing-workers.rst | 46 ++++++++++++++++++++++++++++++++++++++
1 file changed, 46 insertions(+)
diff --git a/chart/docs/customizing-workers.rst
b/chart/docs/customizing-workers.rst
index 9a201eb5111..f0238cd42cb 100644
--- a/chart/docs/customizing-workers.rst
+++ b/chart/docs/customizing-workers.rst
@@ -32,6 +32,52 @@ For example, to set resources on workers for CeleryExecutor:
limits:
cpu: 1
+Separate Celery worker sets
+---------------------------
+
+Use ``workers.celery.sets`` to configure workers for different queues. Each set
+inherits ``workers.celery`` settings and can override them, including
+``extraVolumes`` and ``extraVolumeMounts``. For example, mount different
existing
+PersistentVolumeClaims for two worker sets:
+
+.. code-block:: yaml
+ :caption: values.yaml
+
+ executor: CeleryExecutor
+ workers:
+ celery:
+ sets:
+ - name: analytics
+ queue: analytics
+ extraVolumes:
+ - name: input-data
+ persistentVolumeClaim:
+ claimName: analytics-input
+ extraVolumeMounts:
+ - name: input-data
+ mountPath: /opt/airflow/input
+ readOnly: true
+ - name: research
+ queue: research
+ extraVolumes:
+ - name: input-data
+ persistentVolumeClaim:
+ claimName: research-input
+ extraVolumeMounts:
+ - name: input-data
+ mountPath: /opt/airflow/input
+ readOnly: true
+
+Create both PersistentVolumeClaims before deploying the chart, and route tasks
+to the corresponding queue. The default worker set remains enabled to consume
+tasks sent to the ``default`` queue. Each set inherits replica and autoscaling
+settings. If its pods can run on multiple nodes, use a claim that supports
+read-only mounts from multiple nodes, such as ``ReadOnlyMany``, with a
compatible
+storage driver.
+Non-empty list overrides such as ``extraVolumes`` and ``extraVolumeMounts``
replace
+the inherited list, so include all additional volumes and mounts needed by each
+set. An empty list retains the inherited value.
+
Custom ``pod_template_file``
----------------------------
