aaron-y-chen opened a new pull request, #72555:
URL: https://github.com/apache/airflow/pull/72555
closes: #35154
## Summary
- `worker-kerberos` becomes a Kubernetes native sidecar
- moved from `containers` to `initContainers` with a fixed `restartPolicy:
Always` in both `worker-deployment.yaml` (Celery) and
`pod-template-file.kubernetes-helm-yaml` (KubernetesExecutor).
- As an ordinary container it never exits, and a task Pod (`restartPolicy:
Never`) only completes once **all** app containers terminate.
KubernetesExecutor Pods therefore never reach `Succeeded`: the task hangs and
the Pod leaks.
- The `klist -s` probe from #71221 could not fix this alone - an ordinary
sidecar's probe gates only its own readiness, not the main containers.
## E2E Tests
| | before (ordinary sidecar) | after (native sidecar) |
| --- | --- | --- |
| kerberos container started | `04:58:23Z` | `07:21:06Z` |
| `base` container started | `04:58:22Z` | `07:22:05Z` |
| startup relationship | parallel - `base` started 1s *earlier* | sequential
- `base` waited 59s for the ticket |
| `base` terminated | `Completed`, exit 0 | `Completed`, exit 0 |
| sidecar after `base` exited | still running | terminated by kubelet within
5s |
| Pod outcome | `Running`, unchanged for 24m37s | completed and removed
within 10s |
| leftover | `1/2 NotReady` | none |
Before: the Dag run stayed `running` and the Pod had to be deleted by hand.
After: two independent Dag runs both reached `success` with every task Pod
cleaned up, and `TestKubernetesExecutor::test_integration_run_dag` passes.
The ticket wait differed between the two runs (59s and 20s), so the gap is
real ticket acquisition rather than a fixed delay.
Celery worker on the same cluster: `restartPolicy=Always`, startup probe
preserved as `["klist","-s"]`, absent from `spec.containers`, `started=true`,
and `klist -s` exits 0 inside the sidecar.
This scenario is not covered by CI - `k8s-tests.yml` never enables
`kerberosSidecar`, and the kerberos overlay smoke test only checks the KDC and
keytab. Helm render tests for both templates are included and run in CI.
## Compatibility
`significant` newsfragment included. `SidecarContainers` is enabled by
default since Kubernetes 1.29 and GA in 1.33; the chart already requires 1.30+.
Anything querying `.spec.containers[name=worker-kerberos]` must move to
`.spec.initContainers` and `.status.initContainerStatuses`.
`kerberosInitContainer` is deliberately left untouched - it is a one-shot
`airflow kerberos -o` mode with different semantics, and deprecating it is a
separate public-values decision.
---
##### Was generative AI tooling used to co-author this PR?
<!--
If generative AI tooling has been used in the process of authoring this PR,
please
change below checkbox to `[X]` followed by the name of the tool, uncomment
the "Generated-by".
-->
- [x] Yes (please specify the tool below)
Generated-by: [Claude Opus 5] following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
