sdaberdaku opened a new issue, #75796:
URL: https://github.com/apache/airflow/issues/75796
### Description
Let a retry policy end a task as `skipped`. AIP-105 retry policies can
already choose `RETRY` or `FAIL` for an exception, and #74719 added
`RESCHEDULE`. A `SKIP` action (with a `RetryDecision.skip(reason=...)` helper)
would mark the task `SKIPPED` instead of failing or retrying it: no retry
consumed, no failure callbacks or failure metrics, `on_skipped_callback` and
the skipped listener hook run, and downstream tasks see a skipped upstream as
they do for `AirflowSkipException`.
```python
from airflow.sdk import ExceptionRetryPolicy, RetryAction, RetryRule
retry_policy = ExceptionRetryPolicy(
rules=[
RetryRule(exception=NoNewDataError, action=RetryAction.SKIP,
reason="nothing to ingest"),
RetryRule(exception=ConnectionError, action=RetryAction.RETRY,
retry_delay=timedelta(seconds=30)),
]
)
```
### Use case/motivation
A task can end in three ways other than a plain failure by raising one of
Airflow's exceptions: `AirflowFailException`, `AirflowRescheduleException` and
`AirflowSkipException`. A retry policy can already produce the first two
outcomes from a domain exception. Skipping is the remaining one.
Today a domain exception that means "nothing to do" (no input for this
partition, the upstream source has not changed, a feature flag is off) has to
be caught in every task body and turned into `AirflowSkipException`, which ties
business code to Airflow. With `SKIP`, the whole mapping from exceptions to
outcomes lives in one reusable policy object attached to the task.
`SKIP` needs no bounds such as `RESCHEDULE`'s `max_reschedules`, because it
ends the task and cannot loop. Its reason can be stored as `retry_reason`,
which `TaskState` already carries.
### Related issues
Implementation: #74740 (open since 14 October, waiting for a maintainer to
approve its CI workflows and review it). It follows AIP-105 (#65474),
`ChainRetryPolicy` (#73508) and `RetryAction.RESCHEDULE` (#74719).
### Are you willing to submit a PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
