pedro-cf opened a new issue, #74727:
URL: https://github.com/apache/airflow/issues/74727
### Description
`@teardown(on_failure_fail_dagrun=True)` makes the teardown the only task
that counts for the Dag run state. If the teardown succeeds, the Dag run is
marked successful even when another task failed.
The 3.3.2 docs say the run's success depends on the teardown when that flag
is set, so this looks intended. The flag name only covers the failure case.
There is no argument to keep a failed task in the result.
Add `on_success_succeed_dagrun` on the teardown. Default `True`, so current
teardowns stay the same.
`False` together with `on_failure_fail_dagrun=True`: a failed teardown still
fails the run, and a failed task still fails the run when the teardown succeeds.
```python
@teardown(on_failure_fail_dagrun=True, on_success_succeed_dagrun=False)
def cleanup(value):
print("cleanup ok")
```
Repro of the current behavior, Airflow 3.3.1 (`airflow dags test`). Same
`_tis_for_dagrun_state` logic is still on main.
```python
from airflow.sdk import dag, setup, task, teardown
@dag(schedule=None)
def teardown_hides_failure():
@setup
@task
def create():
return 1
@task
def work(value):
raise RuntimeError("work failed")
@teardown(on_failure_fail_dagrun=True)
def cleanup(value):
print("cleanup ok")
cleanup(work(create()))
teardown_hides_failure()
```
`work` fails, `cleanup` succeeds, and the Dag run is marked successful.
### Use case/motivation
A cleanup teardown should run after the work, including when the work fails.
Both outcomes should fail the Dag run:
- work fails, teardown succeeds: the Dag run is marked failed
- work succeeds, teardown fails: the Dag run is marked failed
- both succeed: the Dag run is marked successful
`on_failure_fail_dagrun=True` only gives the second line. The first line is
marked successful. `on_failure_fail_dagrun=False` gives the first line, and a
failed teardown no longer fails the run.
The workaround is to leave the flag off, so the failed work task still
counts, and add a normal task after the teardown. An `EmptyOperator` with
`trigger_rule=ALL_SUCCESS` is enough: a failed teardown marks it
`upstream_failed`, and that fails the Dag run. A successful teardown does not
hide the failed work task.
```python
work() >> cleanup() >> EmptyOperator(task_id="end",
trigger_rule=TriggerRule.ALL_SUCCESS)
```
`cleanup` is a `@teardown` with `on_failure_fail_dagrun` left at `False`.
That extra task does nothing. It only exists because a teardown cannot both
fail the run and leave the work task in the result.
### Related issues
No open issue for this. The flag itself came from:
- https://github.com/apache/airflow/issues/29125
- https://github.com/apache/airflow/issues/31656
Docs:
https://airflow.apache.org/docs/apache-airflow/3.3.2/howto/setup-and-teardown.html#controlling-dag-run-state
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
