1fanwang opened a new issue, #66791: URL: https://github.com/apache/airflow/issues/66791
### Description When a Dag is configured with `catchup=False` and a scheduler restart (or a DAG re-enable after a pause) leaves a gap between the previous data interval and the current time, the scheduler calls the timetable's `_skip_to_latest()` to advance past the missed intervals. The skip happens silently: no listener event, no callback, and no DagRun row is created for the gap. This is the right default — `catchup=False` users explicitly opt out of backfill. But operators of business-critical Dags still need to *know* an interval was skipped, distinct from "no run because the schedule didn't fire" and "no run because the task failed". Today the only way to detect a skip is to compare `last_automated_data_interval` against the scheduler's wall clock externally. Dag-level callbacks exist for terminal DagRun states (`on_success_callback`, `on_failure_callback`) and for SLA misses (`sla_miss_callback`). There is no symmetric hook for "an interval was skipped". ### Use case / motivation - SLA accounting on data-aware Dags: an analyst needs to know the 2024-01-15 partition will never be filled because the scheduler skipped past it on a Saturday restart. - Observability dashboards that count Dag runs per window need a counter for skipped intervals, not just `success` / `failed`. - Compliance / audit Dags need a paper trail for any interval the platform decided not to backfill. ### Proposal Add an `on_skipped_intervals_callback: Callable[[DAG, list[DataInterval]], None]` parameter to the `DAG` constructor. The scheduler invokes it once per call to `_skip_to_latest` with the list of (start, end) intervals it advanced past. The callback runs in the scheduler the same way `sla_miss_callback` does today; it does not create DagRun rows. A symmetric listener hook (e.g. `on_intervals_skipped`) could be added alongside the callback so AIP-61 listeners can observe the same event. ### Related issues _No response_ ### Are you willing to submit a PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's Code of Conduct -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
