hussein-awala opened a new pull request, #76024: URL: https://github.com/apache/airflow/pull/76024
Asset-event reads through the Execution API currently load related assets and created Dag runs lazily while building the response. This adds database round trips as the number of returned events grows. Load each event's asset with the event query and fetch created Dag runs in batches, reducing the work for both asset and alias lookups. In a local PostgreSQL 14 benchmark, responses containing 100–1,200 events were about 8–11× faster in the measured helper: | Request | SQL queries before → after | Median time before → after | | --- | ---: | ---: | | 100 asset events | 102 → 2 | 27.696 → 3.487 ms | | 150 alias events | 153 → 2 | 42.382 → 5.223 ms | | 20 alias events, descending | 23 → 2 | 7.080 → 2.041 ms | | 1,200 alias events, unbounded | 1,203 → 4 | 317.535 → 29.449 ms | | No matching asset | 1 → 1 | 0.352 → 0.473 ms | | One event | 3 → 2 | 1.019 → 1.034 ms | The benchmark ran the actual before/after helpers in Breeze with Python 3.11.17, fresh SQLAlchemy sessions, and the same isolated ORM-created schema. The fixture contained 1,200 events across two assets, an alias spanning both, and empty/single/multiple created-run relationships, including queued runs with null start dates. Each case used two warm-ups and 31 measured samples per variant, alternating before/after order. An unchanged control query varied by at most 7.8%. Timings include helper execution, response construction, JSON model serialization, and SQL-capture callbacks; they exclude session setup/teardown and HTTP/JWT handling. These are local helper measurements, not end-to-end request latency. Event timestamp ordering, filtering, limits, related-run membership, and returned fields are preserved. The order of multiple `created_dagruns` within an event is unspecified and can change with batched loading; the benchmark observed that difference. Empty responses add about 0.12 ms locally, while one-event responses are essentially unchanged. Select-in loading batches the returned event IDs, so an unbounded response is not guaranteed to use only two queries. No schema or index change is needed. Six query-count regression cases cover asset/alias lookup, ascending/descending order, finite/unbounded limits, and response relationships using fresh sessions. All six fail on the previous helper with the expected extra-query errors. Validation: - The endpoint test file passed all 64 tests on PostgreSQL, MySQL, and SQLite. PostgreSQL was checked again after the final rebase. - Selected API/Always suites: database tests 6,516 passed, 15 skipped; tests without database access 1,259 passed, 1 expected failure. Batched runs were checked against the complete original test-ID lists. - Selected common.compat/fab suites: database tests 415 passed, 5 skipped; tests without database access 622 passed, 12 skipped. Airflow, common.compat, and fab documentation builds passed. - Regular/manual prek checks and installed commit hooks passed. The final `breeze verify` selection is unchanged. The broader suites and documentation builds ran before the final rebase. The intervening upstream changes leave this helper, its event relationships, and its dependencies unchanged; the focused PostgreSQL suite and verification-plan check passed afterward. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Codex (GPT-6) Generated-by: Codex (GPT-6) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) --- Drafted-by: Codex (GPT-6) (no human review before posting) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
