shahar1 opened a new pull request, #74921: URL: https://github.com/apache/airflow/pull/74921
closes: #68396 replaces: #68557 <details> <summary>AI Summary</summary> `[kubernetes_executor] enable_tcp_keepalive` defaults to `True`, but it has had no effect since urllib3 2.x. The provider changed `HTTPConnection.default_socket_options` after import, but urllib3 2.x binds that list as a default argument of `HTTPConnection.__init__` when the module is imported, so new connections never see the change. When a cloud load balancer or firewall silently drops idle connections to the Kubernetes API server, they stay in the pool in `CLOSE_WAIT` (see #68396). This sets the options on the Kubernetes client's `Configuration.socket_options` instead. The Kubernetes client passes them to urllib3's pool manager (including the proxy path). Every sync client gets them: `get_kube_client()` (KubernetesExecutor and the pod watcher), every branch of `KubernetesHook.get_conn()`, and the GKE hook. - The keepalive options are merged into any `socket_options` the user already set, or into urllib3's defaults, so `TCP_NODELAY` is kept. Calling it twice adds no duplicates. - `enable_tcp_keepalive(configuration)` is now public because the google provider imports it. `_enable_tcp_keepalive()` stays as a deprecated wrapper. The google provider's dependency on cncf.kubernetes is marked `# use next version`. - kubernetes 35.0.0, which the current floor still allows, has no `Configuration.socket_options`. There a warning is logged and keepalive stays off, as it effectively already was. - The async (`kubernetes_asyncio`) clients are not changed. I checked this on a local kubeadm cluster by reading `getsockopt` on the pooled sockets after a real API call. On `main`, every path above has `SO_KEEPALIVE=0` with kernel defaults (`TCP_KEEPIDLE=7200`). With this change, `SO_KEEPALIVE=1` and the configured `tcp_keep_idle`/`tcp_keep_intvl`/`tcp_keep_cnt` are used. `disable_tcp_keepalive` / `enable_tcp_keepalive=False` still leave them off. This takes over #68557, which became inactive. The original commit and its authorship are kept, with review fixes in a follow-up commit. </details> --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Claude Code (Opus 5.5) Generated-by: Claude Code (Opus 5.5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
