waterWang opened a new pull request, #70629: URL: https://github.com/apache/airflow/pull/70629
## Description `NullableDatetimeRangeFilter` was added in #66696 to replace `COALESCE(column, now())` with index-friendly `OR` predicates. For lower bounds (`gte`/`gt`), the NULL branch passes unconditionally: ```python select = select.where(or_(self.attribute >= x, self.attribute.is_(None))) ``` This is correct for **non-terminal** rows (a not-yet-started task will eventually start). But it's wrong for **terminal** dag runs whose `start_date` is `NULL` — a failed/success run that never started will never have a `start_date`, yet it matches every `start_date_gte` filter forever. ### Root cause When querying `GET /dags/~/dagRuns?start_date_gte=<recent-timestamp>`, the response includes terminal dag runs from **years ago** (`state: failed`, `start_date: null`, populated `end_date`) as if they had just occurred. Roughly 99% of a `-logical_date`-ordered page are these stale null-`start_date` rows. ### Fix Add an `extra_null_condition` parameter to `NullableDatetimeRangeFilter`. For `start_date` lower bounds, the factory passes `model.end_date.is_(None)`, so the NULL branch becomes: ```python or_(start_date >= x, and_(start_date IS NULL, end_date IS NULL)) ``` A genuinely pending run has `start_date IS NULL AND end_date IS NULL`. A terminal run with `NULL start_date` has `end_date IS NOT NULL` and is correctly excluded. ### Changes - `parameters.py`: Add `extra_null_condition` to `NullableDatetimeRangeFilter`; update `datetime_range_filter_factory` to pass it for `start_date` - `test_parameters.py`: Update existing lower-bound test to verify `end_date IS NULL`; add test for terminal-run exclusion ### Related issues Fixes #70627 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
