sushrutikhar opened a new pull request, #74983: URL: https://github.com/apache/airflow/pull/74983
FastAPI validates a sync route's response in the threadpool but serializes it on the event loop. `GET /dags` still hit the database during serialization: the route passed the un-iterated `ScalarResult` into `DAGCollectionResponse.dags: Iterable[DAGResponse]`, which pydantic validates lazily during serialization, and the query loads `DagModel.tags` with `selectinload`. So the `dag_tag` query ran inside `dump_json`, on the event loop and on the route's own connection, and while it waited on the database (a lock, a slow disk, a busy server) the API server answered no other request, health checks included. - `DAGCollectionResponse.dags` is now `list[DAGResponse]`, like `PoolCollectionResponse.pools` since #74247, and `get_dags` builds the items with `DAGResponse.model_validate`, so the tags are loaded while the route builds the response in the threadpool. - `patch_dags` returns the same model. It selects Dags without their tags, so the tags of each Dag were lazy-loaded on the event loop, one query per Dag. It now builds the items the same way, and those queries run in the threadpool. `Iterable` and `list` give the same array schema, so the OpenAPI spec, the UI client and the airflow-ctl datamodels do not change; I regenerated them to check. The UI's Dag listing uses the same query but already builds its items in the route, so it is not affected. The other `Iterable` response fields mentioned in the issue are not part of this PR. The new tests record SQL statements executed on the event loop thread while serving `GET /dags` and `PATCH /dags`. Without the change they catch 1 and 2 statements respectively. `PATCH /dags` also commits its audit log entry on the event loop, in the async `action_logging` dependency before the route runs; that is not part of serializing the response, so the test leaves it out. I ran the new tests before and after the change, and the whole of `test_dags.py` for the public and UI routes after it, on SQLite and PostgreSQL with Python 3.11, plus the prek hooks for the changed files. closes: #74948 --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: Claude Code (Claude Opus 5.5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) --- * Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged. * For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed. * When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). * For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
