bmanan7 opened a new pull request, #76834:
URL: https://github.com/apache/airflow/pull/76834
A task instance that is cleared while deferred ends up with a NULL state but
keeps its `trigger_id`.
`Trigger.clean_unused` only releases task instances where `state !=
'deferred'`, and in SQL a NULL
state never satisfies that comparison, so the row is skipped. The trigger
then stays referenced
forever: the triggerer keeps running it, every completion is dropped because
no task is deferred on
it any more, and the trigger is picked up again on the next sync.
`clear_task_instances` on 3.3.x and earlier only resets the state and
`next_method`, so this is
reachable through a normal "Clear" of a deferred task there. On `main` and
3.4 a clear archives the
old row and clears its `trigger_id`, so that path no longer leaves the
trigger behind. The cleanup
predicate is still wrong for any task instance with no state that holds a
trigger, so this makes it
match NULL explicitly. Please backport to `v3-3-test`.
How to reproduce (2.11 and 3.3.x):
1. Run a Dag with a deferrable sensor, for example
`TimeDeltaSensorAsync(task_id="wait", delta=timedelta(minutes=4))`.
2. While the task is deferred, clear it from the UI or the REST API.
3. The task instance row has `state = NULL` and a non-NULL `trigger_id`, and
the triggerer logs show the same
trigger id starting and completing over and over (about 30 times a minute
in my run). It only stops once the
task is marked failed or success. With a remote logging handler that
uploads on close, every pass also
uploads the task log again.
Tested on `v3-3-test`: clearing a deferred task and running
`Trigger.clean_unused` left the trigger in place
before the change and deleted it after.
Checks run: `uv run --project airflow-core pytest
airflow-core/tests/unit/models/test_trigger.py -k clean_unused`
(12 passed, 1 skipped because it needs `BACKEND=sqlite`), the new test fails
without the source change,
`ruff format`, `ruff check` and the `prek` pre-commit hooks on the changed
files. Not run: `breeze verify`,
mypy for `airflow-core`, Postgres and MySQL.
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes — Claude Code (Sonnet 5.5)
Generated-by: Claude Code (Sonnet 5.5) following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
