1067147135 commented on issue #49135:
URL: https://github.com/apache/airflow/issues/49135#issuecomment-5392786891
Reproducing this deterministically on **Airflow 3.0.6**, 129 DAGs, and I've
pinned down the exact mechanism via the raw API response — this is the same
issue jsjasonseba diagnosed above (ordering by last-run recency causes a
mismatched/incomplete DAG set), just triggered in a fully deterministic way
by our schedule layout.
We have two DAGs per data source: a `_sync` DAG (runs at 08:30/20:30) and a
`_check` DAG for the same table (runs 2 hours later, at 10:30/22:30).
Between 10:30 and 20:30 each day, every `_check` DAG's last run is *more
recent* than every `_sync` DAG's last run.
Request: `GET /ui/dags/recent_dag_runs?dag_runs_limit=1&limit=50&offset=0`
The response contains exactly 50 `dags` entries, ordered by most-recent
last-run descending. Since all `_check` DAGs ran more recently (10:30) than
all regular `_sync` DAGs (08:30), the top-50 window fills up almost entirely
with `_check` DAGs — the corresponding `_sync` DAGs (e.g.
`cloud_db_sync_AIndexDescription_sync`, `cloud_db_sync_AIndexMembers_sync`,
`cloud_db_sync_aindexfinancialderivative_sync`, and many others) are
**completely absent from the JSON response** — not null, just not present
as entries at all.
This is confirmed further by paging: requesting `offset=50` (page 2) returns
almost exclusively `_sync` DAGs, with essentially zero `_check` DAGs —
because
the global sort-by-recency has already exhausted the `_check` category
(all more recent) within page 1, and page 2 picks up wherever that ordering
left off, which happens to line up with the `_sync` category entirely.
Since our schedule only has two fixed run times (08:30 and 10:30), the two
categories don't interleave at all — they cleanly separate into "page 1 =
check, page 2 = sync", which nicely demonstrates the ordering-by-recency
mechanism isn't paginating per-DAG-list-order, it's a global sort by
last-run time.
This is 100% reproducible and stable (same missing set on every refresh)
as long as the sync-vs-check recency relationship holds, which for us is
most of the business day.
Given #51115's automated backport to `v3-0-test` failed
("Backport failed to create: v3-0-test"), could someone confirm whether the
fix (or any per-DAG-limit approach) actually landed in any 3.0.x release?
This looks like exactly the failure mode the original diagnosis in this
thread described, still present with real production data.
Happy to share the full JSON response if useful for reproduction.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
