shahar1 opened a new issue, #69023: URL: https://github.com/apache/airflow/issues/69023
### Context `apache-airflow-providers-cncf-kubernetes` 10.18.0 widened the kubernetes client bound to allow 36.x (#68041). Client 36.x introduced regressions that break task execution, so the bound is being capped back to `<36.0.0` as a workaround. Two distinct 36.x failures, same root-cause family (the 36.x `Configuration` defaulting/copy change): 1. **401 Unauthorized authenticating to the cluster** — reported against the google (GKE) provider; tasks fail after the auth token rotates. A copied `Configuration` does not share the refreshed token, so requests go out with a stale/empty token (kubernetes-client/python#1946). Downgrading the provider to 10.17.1 (client 35.x) works around it. **Not yet fixed.** 2. **`PicklingError` on `pod_override` in-cluster** — tracked in #68827, already addressed for the executor-queue deserialization path by #68848. The auth 401 above is a separate code path that #68848 does **not** cover. ### Workaround applied `kubernetes` and `kubernetes_asyncio` capped back to `<36.0.0` in cncf.kubernetes to restore the known-good 35.x client until 36.x is fully supported. The cap site links back to this issue. ### Follow-up work required to lift the cap - [ ] **cncf hook** — stop relying on the process-global default `Configuration`. Pass `client_configuration=self.client_configuration` in the `in_cluster` branch (`hooks/kubernetes.py:329`) like the other branches, and default `self.client_configuration` to a fresh `client.Configuration()` instead of `None`, so the loaded token lives on one concrete object threaded into the `ApiClient`. (Same lesson #68848 applied to the deserialization path.) - [ ] **google GKE hook** — make the bearer-token path robust to 36.x copying: ensure the single token-carrying `Configuration` is used for every request, or set the `Authorization` header directly on the `ApiClient` instead of via `api_key` / `refresh_api_key_hook` (`hooks/kubernetes_engine.py:77-89`). - [ ] **Regression test** covering token rotation/expiry — the 401 only appears after the token expires (~15 min), which is why it passed RC smoke testing. ### Acceptance criteria - cncf.kubernetes and google providers validated end-to-end on client 36.x against real GKE and in-cluster `KubernetesExecutor` with an expiring token. - Cap lifted to allow 36.x again. ### References - kubernetes-client/python#1946 (stale-token-on-copy 401) - #68041 (the widening), #68827 + #68848 (pickling half), 10.17.1 (last known-good) --- Drafted-by: Claude Code (Opus 4.8) (no human review before posting) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
