kaxil opened a new pull request, #74478: URL: https://github.com/apache/airflow/pull/74478
Follow-up to #49782, which kept `http://localhost:8080` as the execution API fallback when `[api] base_url` is unset or relative, for backward compatibility with the old literal default. The port came along with it, and `[api] port` was never consulted. On a single host that changes `[api] port` (say to 9091) and leaves `base_url` unset, the API server listens on 9091 but the executor sends every task to `http://localhost:8080/execution/`. Every task fails, the task log shows only "Pre Execute", and the scheduler log has an `httpx.ConnectError: [Errno 111] Connection refused` that never names the URL. #50739 reported this and was resolved by setting `base_url`. [`get_execution_api_server_url()`](https://github.com/apache/airflow/blob/bc96a14f77642cac7ea939e0837bf1bf66220f7a/airflow-core/src/airflow/executors/base_executor.py#L53-L69) now builds the fallback from `[api] port`. It still applies only when `base_url` is unset or relative. An absolute `base_url` or an explicit `[core] execution_api_server_url` wins as before. **An explicit `execution_api_server_url` is now returned before anything else is read.** The helper used to build the fallback first and look up the explicit URL last. With the port lookup added, that order would have failed tasks on a worker with an explicit URL and an unparseable `[api] port` it never needs. **Team executors use the team's `[api] port`.** The port comes from the same `conf_source` as `base_url`, and team config [does not fall back to global settings](https://github.com/apache/airflow/blob/bc96a14f77642cac7ea939e0837bf1bf66220f7a/airflow-core/docs/core-concepts/multi-team.rst#L537-L542). Reading the global port instead would build one URL from two config scopes. One behaviour change: a deployment that sets `[api] port` to X but starts the server with `airflow api-server --port 8080` worked by coincidence and now sends tasks to port X. Setting `execution_api_server_url` or `base_url` keeps it working. The fallback is still plain HTTP to `localhost`, so TLS, a non-loopback `[api] host`, and multi-host or split deployments still need `execution_api_server_url` or an absolute `base_url`. The `web-stack.rst` change says this for `--apps execution`, including the path prefix an execution-only server serves under when `base_url` has one. It also corrects an older line on that page that said a prefixed `base_url` always needs `execution_api_server_url` updated: the default follows `base_url`. The Celery executor's [copy of this fallback](https://github.com/apache/airflow/blob/bc96a14f77642cac7ea939e0837bf1bf66220f7a/providers/celery/src/airflow/providers/celery/executors/celery_executor_utils.py#L257-L276) runs only on Airflow below 3.3 and is unchanged. Making this failure visible is separate work: #70290 puts the attempted URL and the settings to check in the task log when the execution API is unreachable (see also #51163). `airflow standalone` in breeze with `AIRFLOW__API__PORT=9091` and a one-task Dag: | Code | `base_url` | Resolved execution API URL | Task | |---|---|---|---| | `main` (bc96a14) | unset | `http://localhost:8080/execution/` | failed, `Connection refused` | | this branch | unset | `http://localhost:9091/execution/` | success | | this branch | `http://localhost:9091` | `http://localhost:9091/execution/` | success | | this branch | `http://127.0.0.1:9091` | `http://127.0.0.1:9091/execution/` | success | related: #50739, #51163, #70290 --- * Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged. * For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed. * When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). * For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
