yuqian90 opened a new pull request #12058:
URL: https://github.com/apache/airflow/pull/12058
## Expected behaviour
For a sensor like this, the intention of the DAG author is usually to fail
the sensor if it's still not done after ten minutes. However, if the sensor
fails prematurely due to other unexpected reasons (such as network outage),
retry at most twice.
```python
sensor = PythonSensor(
task_id='sensor',
python_callable=python_callable,
timeout=60 * 10,
retries=2,
mode="reschedule",
)
```
## Actual behaviour
The actual current behaviour of Airflow is to retry when the sensor times
out. So the effective timeout of the sensor becomes 60 * 10 * (retries + 1) =
30min. This often causes confusion. It also makes it impossible to achieve the
expected behaviour no matter how the author configures the sensor.
## Fix
This PR fixes this issue. `AirflowSensorTimeout` is now treated as immediate
failure. This achieves the expected behaviour. The sensor will fail if timeout
is reached. If someone really wants the previous behaviour, he can always
increase the timeout. I.e instead of failing and retrying every ten minutes
three times, just set the timeout to 30min.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
