avanish-garg opened a new issue, #72427:
URL: https://github.com/apache/airflow/issues/72427
### Description
When `start_date` is set to a value that falls inside the currently
in-progress period (e.g.
deploying a DAG with `start_date` a few hours ago, catchup=False),
`CronDataIntervalTimetable`'s
`_align_to_next(earliest)` skips the in-progress period entirely — it only
considers periods that
start at or after `start_date` — which pushes the first run out to nearly
two full periods later,
not one. Concretely, for `@daily`, deploying at `start_date = now - 3h`
today doesn't produce a
first run for today's or tomorrow's data, it produces one for the
day-after-tomorrow.
Verified with a script run through `breeze testing core-tests`, calling
`CronDataIntervalTimetable("@daily", timezone="UTC").next_dagrun_info(...)`
directly with
`last_automated_data_interval=None`, `restriction.catchup=False`:
| `start_date` | first run's data interval | delay |
|---|---|---|
| `None` | yesterday -> today (already complete) | none -- fires immediately
|
| long in the past | yesterday -> today (already complete) | none -- fires
immediately |
| `now - 3h` (today) | day-after-tomorrow's full period | ~2 periods |
### Use case/motivation
I don't know if this is intentional (only ever run "clean" full periods that
start entirely after
`start_date`, never a period the DAG only existed for part of) or an
oversight -- I couldn't find
prior discussion either way. If it's intentional, this issue should probably
just be closed. If
not, the `CronTriggerTimetable`-style `_calc_first_run()` -- which chooses
between the past-tick
and next-tick based on `run_immediately` / proximity -- is a fix pattern
already proven for the
trigger timetables and could plausibly be adapted, though I haven't scoped
that out since it
depends on the answer above.
### Related issues
Was previously opened as #72380 (closed to re-open via this form so template
labels apply
correctly). That version originally claimed CronDataIntervalTimetable
"always waits for the next
boundary" by analogy with CronTriggerTimetable's run_immediately -- verified
against actual
runtime behavior and that claim was wrong for the common case (no
start_date, or an old one); this
version describes only the narrower, verified edge case.
### Are you willing to submit a PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
