seanghaeli opened a new issue, #77054:
URL: https://github.com/apache/airflow/issues/77054
`Trigger.clean_unused` (`airflow-core/src/airflow/models/trigger.py`), which
the triggerer runs every loop, and the scheduler's
`_remove_unreferenced_triggers` delete a trigger as soon as no task instance,
asset watcher or callback points at it. A watcher stops pointing at its trigger
when it's removed from the Dag, when its kwargs change (that makes a new
trigger), or when every Dag scheduled on its asset is paused
(`add_asset_trigger_references` in
`airflow-core/src/airflow/dag_processing/collection.py`).
So the usual reactions to a failing watcher, pausing the Dag or fixing the
watcher, remove the record of what went wrong: `GET
/api/v2/triggers/{trigger_id}` returns 404, and the failure count, first and
last failure and last error recorded since #76101 are gone. With #76560, the
trigger's logs can't be read through the API either, although its files are
still in remote storage under its trigger id.
Suggestion: keep a trigger that recorded failures for a configurable time
after nothing uses it, marked finished and released from its triggerer so it's
never run again, and have the API list it with its own status. Triggers that
never failed can still be deleted right away, so the table doesn't grow with
every deferred task. Both cleanups would then delete a failed trigger only once
it has been finished for longer than that.
Done when: a trigger that recorded failures stays readable through the API
(and its logs, with #76560) for the configured time after nothing uses it, is
never run again, and is then deleted, with tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
