karenbraganz opened a new issue, #44893: URL: https://github.com/apache/airflow/issues/44893
### Apache Airflow version 2.10.3 ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? Using `soft_fail=True` with sensors does not allow retries. If a sensor is going to fail and `soft_fail=True`, the task will skip instead of failing. However, if retries are set for the task, those are not executed since an [AirflowSkipExecution is raised in the code](https://github.com/apache/airflow/blob/35087d7d10714130cc3e9e9730e34b07fc56938d/airflow/sensors/base.py#L293), and the task's status is marked as `skipped`, not `up_for_retry`. ### What you think should happen instead? If the task has retries left, its status should be set to `up_for_retry`. The `AirflowSkipException` should only be raised if no retries are left. ### How to reproduce I have seen a few reports of this and was also able to reproduce it with the below DAG code. ``` import datetime from airflow.decorators import dag from airflow.decorators import task from airflow.sensors.external_task import ExternalTaskSensor @dag(start_date=datetime.datetime(2024, 10, 1), schedule="* * * * *", catchup=False) def dag_2(): ets = ExternalTaskSensor( task_id="ets", external_dag_id="non_existent_dag", soft_fail=True, retries=3, poke_interval=10, timeout=5, ) ets dag_2() ``` ### Operating System Debian GNU/Linux 12 ### Versions of Apache Airflow Providers _No response_ ### Deployment Astronomer ### Deployment details _No response_ ### Anything else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
