Abdel39 opened a new pull request, #74854: URL: https://github.com/apache/airflow/pull/74854
related: #73686 When a time-scheduled Dag is at `max_active_runs`, `DagModel.dags_needing_dagruns` filters it out (`exceeds_max_non_backfill == false`), so its next run is never created. Nothing is queued, `scheduler.dagruns.queued` (#70013) stays at zero, and nothing tells you the Dag is falling behind its schedule until someone notices missing intervals. This adds a `dag.schedule_overdue_seconds` gauge, tagged by `dag_id`: seconds since the Dag's `next_dagrun_create_after`, emitted only while the Dag is due and held back by `max_active_runs`. - Emitted on the existing `[scheduler] dagrun_metrics_interval` timer, as a separate read-only query. It does not touch `dags_needing_dagruns`, which is row-locked and `LIMIT`ed. - Only Dags in the blocked state are emitted, so the series count is bounded by the number of blocked Dags, not the number of Dags. A healthy deployment emits nothing. - Asset-gated Dags are excluded, since `next_dagrun_create_after` alone does not make them due. We've run the same metric as a scheduler patch on our Airflow staging deployments. It was the signal teams needed to decide when to raise `max_active_runs` for scheduled Dags, where queued-run counts showed nothing. --- ##### Was generative AI tooling used to co-author this PR? - [x] Yes -- Claude Opus 5.5 Generated-by: Claude Opus 5.5 following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
