AdityaBhattacharya1 opened a new pull request, #70092:
URL: https://github.com/apache/airflow/pull/70092
<!-- SPDX-License-Identifier: Apache-2.0
https://www.apache.org/licenses/LICENSE-2.0 -->
A task instance can end up in the `up_for_retry` state with `end_date` left
as `None`,
for example if the process running it is killed abruptly before the end date
gets
recorded, or via any code path that transitions task state without going
through
`TaskInstance.set_state()`. When that happens, `next_retry_datetime()`
unconditionally
computes `self.end_date + delay`, raising:
```
TypeError: unsupported operand type(s) for +: 'NoneType' and
'datetime.timedelta'
```
This is called from `NotInRetryPeriodDep._get_dep_statuses()`, which the
scheduler
invokes on every scheduling pass for a retry-eligible task. The exception is
unhandled
at that call site, so depending on version this either crashes the scheduler
process
outright, or (after the per-DagRun failure isolation added in #62893) gets
caught and
logged but causes that DagRun's scheduling pass to fail identically every
time it's
re-evaluated — the affected task instance never recovers on its own without
directly
patching its `end_date` in the metadata database.
Personally I faced this error for a DAG run where a DAG had been
transitioned to `up_for_retry` status and then paused. Later revision of the
DAG changed the DAG structurally. When I unpaused the DAG and cleared state for
the previous DAG run, the end_date was never repopulated for that task
instance: DagRun.verify_integrity() / expand_mapped_task() had to reconcile the
stale map_index=-1 row against a task definition that had since become mapped,
and that reconciliation left the row up_for_retry with end_date still NULL.
Also added a unit test on `next_retry_datetime()` covering the
`end_date=None` case
directly, and one on `NotInRetryPeriodDep` reproducing the actual crash path
named
in the linked issues (`state=up_for_retry`, `end_date=None`) to confirm the
dependency
check no longer raises.
closes: #51640
related: #19615
---
##### Was generative AI tooling used to co-author this PR?
- [ ] Yes
---
* Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information. Note: commit author/co-author name and email in commits
become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment:
`{pr_number}.significant.rst`, in
[airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
You can add this file in a follow-up commit after the PR is created so you
know the PR number.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
