HsiuChuanHsu commented on code in PR #56589:
URL: https://github.com/apache/airflow/pull/56589#discussion_r2448734533
##########
chart/templates/workers/worker-service.yaml:
##########
@@ -21,28 +21,56 @@
## Airflow Worker Service
#################################
{{- if or (contains "CeleryExecutor" .Values.executor) (contains
"CeleryKubernetesExecutor" .Values.executor) }}
+
+{{- /* Build worker groups list: use celeryQueueGroups if defined, otherwise
create default worker */ -}}
+{{- $workerGroups := list }}
+{{- if .Values.workers.celery.celeryQueueGroups }}
+ {{- $workerGroups = .Values.workers.celery.celeryQueueGroups }}
+{{- else }}
+ {{- $defaultWorker := dict "name" "" }}
+ {{- $workerGroups = list $defaultWorker }}
+{{- end }}
+
+{{- /* Loop through all worker groups (either celeryQueueGroups or default) */
-}}
+{{- range $groupIndex, $workerGroup := $workerGroups }}
+{{- if $groupIndex }}
+---
+{{- end }}
+
+{{- /* Set variables based on whether this is default worker or custom group
*/ -}}
+{{- $isDefaultWorker := eq $workerGroup.name "" }}
Review Comment:
The current naming is a bit of confusing. The intended logic is: when the
`workers.celery.queueGroups` setting is undefined, `isDefaultWorker` would to
be `true`.
This means we are relying solely on the original worker and are not
deploying any alternative worker configurations. The naming should be changed:
**Naming Fix**
- **`values.yaml`**
example config: `name: default-workers` -> `name: base-workers`
- **`worker-deployment.yaml` & `worker-service.yaml`**
- `$isDefaultWorker` -> `$isBaseWorker`
- `$defaultWorker` -> `$baseWorker`
##########
chart/values.yaml:
##########
@@ -1010,6 +1010,34 @@ workers:
# Annotations to add to worker kubernetes service account.
annotations: {}
+ # Celery queue groups for different queues and node assignments
+ # When defined, ONLY these worker groups will be created (no default
worker deployment)
+ # Each worker group creates a separate Deployment assigned to specific
nodes and queues
+ celeryQueueGroups: []
Review Comment:
Agree, fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
