GitHub user CieriS added a comment to the discussion: Airflow worker pod not able to fetch connections information during its execution
I think this is the cluster-wide proxy, and it also explains why your curl test passes while the task fails. **Why curl works but httpx doesn't** On OpenShift with a cluster-wide proxy configured, pods get `HTTP_PROXY` / `HTTPS_PROXY` / `NO_PROXY` injected in **uppercase**. curl deliberately **ignores uppercase `HTTP_PROXY`** for `http://` URLs (it only reads lowercase `http_proxy`, for historical security reasons), so your curl goes straight to the Service. httpx, which the Task SDK uses for the Execution API client, **does** honor `HTTP_PROXY`, so the request is sent to the corporate proxy instead. The proxy can't resolve `airflow-api-service` (a cluster-internal name) and answers with a bare `503`. That's consistent with what you saw: the API server logs show no matching request, because the request never reaches it. **How to confirm (from inside a worker/task pod)** ```bash env | grep -i proxy # what curl did in your test (direct): curl -sv http://airflow-api-service:8080/execution/ -o /dev/null # force curl to behave like httpx (through the proxy), you should get the same 503: curl -sv -x "$HTTP_PROXY" http://airflow-api-service:8080/execution/ -o /dev/null # same comparison in Python: python -c "import httpx; u='http://airflow-api-service:8080/execution/'; print(httpx.get(u, trust_env=True).status_code, httpx.get(u, trust_env=False).status_code)" ``` If `trust_env=True` gives 503 and `trust_env=False` gives a non-503 (e.g. 401/404/405), the proxy is the culprit. **Fix** Make sure the internal Service names are in `NO_PROXY` for the worker/task pods (and the API server, scheduler, etc.): ```yaml # values.yaml (Helm chart) env: - name: NO_PROXY value: "localhost,127.0.0.1,airflow-api-service,.svc,.svc.cluster.local,<your-existing-entries>" - name: no_proxy value: "localhost,127.0.0.1,airflow-api-service,.svc,.svc.cluster.local,<your-existing-entries>" ``` Notes: - The **short** name `airflow-api-service` must be listed explicitly: `.svc` / `.cluster.local` only match FQDNs. Alternatively, point `AIRFLOW__CORE__EXECUTION_API_SERVER_URL` to the FQDN (`http://airflow-api-service.<namespace>.svc.cluster.local:8080/execution/`) and keep `.svc.cluster.local` in `NO_PROXY`. - httpx doesn't support CIDR ranges in `NO_PROXY` (e.g. `10.0.0.0/8`), so existing CIDR-only entries won't help here. - With KubernetesExecutor, check that the env actually lands in the **task pods** (pod template), not only in the scheduler. If `env | grep -i proxy` shows nothing in the task pod, let me know and I'll look at other causes (Service `targetPort`, NetworkPolicy, Istio/service mesh sidecar). GitHub link: https://github.com/apache/airflow/discussions/69733#discussioncomment-18760462 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
