ferruzzi opened a new issue, #71750:
URL: https://github.com/apache/airflow/issues/71750
### Under which category would you file this issue?
Airflow Core
### Apache Airflow version
3.3.0+
### What happened and how to reproduce it?
A Dag carrying a `DeadlineReference.DAGRUN_LOGICAL_DATE` deadline does not
get a `Deadline` row when the DagRun has a NULL `logical_date`. There is no
error and no exception, and the only output is a warning naming a DagRun that
demonstrably exists:
```
Could not find DagRun for dag_id=..., run_id=...
```
The cause is in `_fetch_from_db`
(`serialization/definitions/deadline.py:361`):
```python
result = session.execute(
select(column).where(DagRun.dag_id == dag_id, DagRun.run_id == run_id)
).scalar()
if result is None:
logger.warning("Could not find DagRun for dag_id=%s, run_id=%s", dag_id,
run_id)
return result
```
`.scalar()` returns `None` in two different situations: no row matched OR a
row matched and the selected column was NULL. The warning assumes the first.
For a NULL `logical_date` it is always the second, so the message is guaranteed
wrong in exactly the case it fires, and the caller receives `None` either way
and skips the deadline silently.
Two ways to produce a NULL `logical_date`:
1. CLI: `airflow dags trigger <dag_id>`
When you create a Dag this way, unless you set the `-l` flag, the run is not
assigned a `logical_date`
2. Asset-triggered: Dag A produces an asset, Dag B consumes it and carries a
`DAGRUN_LOGICAL_DATE` deadline.
As above, when Dag B gets created, it gets a `queued_at` timestamp but does
not get a `logical_date`.
As a control, run the same Dag using `airflow dags trigger <dag_id> -l
"$(date -Iseconds)"` which creates the run with the current time as the
`logical_date`.
### What you think should happen instead?
`_fetch_from_db` should distinguish "no such DagRun" from "the column is
NULL". The row lookup and the column read are separate questions and only the
former justifies that warning. A NULL column should produce a message naming
the real cause: this reference requires `logical_date` and this run has none.
### Operating System
_No response_
### Deployment
None
### Apache Airflow Provider(s)
_No response_
### Versions of Apache Airflow Providers
_No response_
### Official Helm Chart version
Not Applicable
### Kubernetes Version
_No response_
### Helm Chart configuration
_No response_
### Docker Image customizations
_No response_
### Anything else?
This is not specific to `logical_date`. The same helper serves
`DagRunQueuedAtDeadline` (`serialization/definitions/deadline.py:188`), so any
run created directly in RUNNING rather than QUEUED has a NULL `queued_at` and
takes the identical silent path with the identical wrong warning.
`cli/commands/task_command.py:144` and the `get_or_create_dagrun` at
`models/dagrun.py:2568` used by `DAG.test` both do this. Any nullable column
reached through this helper has the same failure.
There are two copies of `_fetch_from_db` on `main`, at
`models/deadline.py:513` and `serialization/definitions/deadline.py:361`, each
with callers for both `logical_date` and `queued_at`. The serialization copy
is the one on the DagRun-creation path.
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
