bcanvural opened a new pull request, #74587: URL: https://github.com/apache/airflow/pull/74587
With `executor: "KubernetesExecutor,CeleryExecutor"`, KubernetesExecutor task pods carry the same `tier`/`component`/`release` labels as the default worker set's Celery pods, so the headless `<release>-worker` Service selects every task pod. Its endpoints change on every task start and finish; on one of our installs the EndpointSlice saw ~3.35M updates in ~6 months, and in an Istio mesh each update is a full config push to every sidecar. The Service's DNS name also resolves to pods that don't serve worker logs. Default-set Celery pods now always carry `worker-set: <name>`, as named sets already do, and the Service selects on it. The Deployment/StatefulSet `spec.selector` is untouched because it is immutable, so existing releases upgrade in place; until the rollout finishes, the Service only lists pods from the new revision. closes: #74584 Tested locally: - New `TestWorkerService::test_selector_excludes_kubernetes_executor_pods` fails on `main` (and with either template change reverted on its own) and passes with this change. - Chart test files that render the worker templates (`test_worker.py`, `test_worker_sets.py`, `test_basic_helm_chart.py` and 8 others): 669 passed. 9 `test_invalid_pull_policy` / `test_invalid_dags_access_mode` cases fail the same way on `main` with my local helm 3.13.2, whose schema error wording differs from the pinned 3.19.0. - `prek run --stage pre-commit` on the changed files passes, except `lint-helm-chart` and `kubeconform`, which need breeze (not installed here). `helm lint chart` passes. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Claude Code (Opus 5.5) Generated-by: Claude Code (Opus 5.5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
