Tegh25 commented on code in PR #68359:
URL: https://github.com/apache/airflow/pull/68359#discussion_r3907540815
##########
airflow-core/src/airflow/jobs/scheduler_job_runner.py:
##########
@@ -2520,8 +2545,58 @@ def _mark_backfills_complete(self, *, session: Session =
NEW_SESSION) -> None:
for b in backfills:
b.completed_at = now
- def _create_dag_runs(self, dag_models: Collection[DagModel], session:
Session) -> None:
- """Create a DAG run and update the dag_model to control if/when the
next DAGRun should be created."""
+ def _collect_skipped_intervals(
+ self,
+ serdag: SerializedDAG,
+ new_info: DagRunInfo,
+ session: Session,
+ *,
+ listener_has_impls: bool,
+ ) -> SkippedIntervalsSummary | None:
+ """
+ Summarize intervals skipped due to catchup=False.
+
+ Asks the timetable whether the previous automated DagRun's immediate
+ successor (with catchup enabled) is earlier than the new run. Returns
+ ``None`` when there is no schedulable gap or when no previous run
exists.
+ """
+ if serdag.catchup:
+ return None
+ if new_info.data_interval is None:
+ return None
+ if not serdag.has_on_skipped_intervals_callback and not
listener_has_impls:
+ return None
+
+ prev_run = session.scalar(
+ select(DagRun)
+ .where(
+ DagRun.dag_id == serdag.dag_id,
+ DagRun.run_type == DagRunType.SCHEDULED,
+ DagRun.data_interval_end.is_not(None),
+ DagRun.data_interval_end <= new_info.data_interval.start,
Review Comment:
Thanks for catching this. I added a scheduler test as you suggested and it
failed to trigger the skipped intervals callback for the multi-skipped, trigger
timetable scenario. Moving the skipped intervals lookup before
`create_dagrun()` resolved the issue in
[975d06e](https://github.com/apache/airflow/pull/68359/commits/975d06ef7298f76d5e186ffda2e9acac919e8646).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
