hussein-awala commented on code in PR #27597:
URL: https://github.com/apache/airflow/pull/27597#discussion_r1019513643
##########
airflow/decorators/__init__.pyi:
##########
@@ -444,6 +445,9 @@ class TaskDecoratorCollection:
prevent too much load on the scheduler.
:param exponential_backoff: allow progressive longer waits between
pokes by using exponential backoff algorithm
+ :param max_wait: maximum wait interval between pokes, can be set as
+ ``timedelta`` or ``float`` seconds, which will be converted into
Review Comment:
When `exponential_backoff` is False, the sensor pokes every `poke_interval`
seconds, but when it's True, we
[calculate](https://github.com/apache/airflow/pull/27597/files#diff-7486f32e385d7ad0376cccda08d80e54939aa901a24616d11fb1f5cba6af7f83L226)
a new exponential interval based `poke_interval` and the try number. After
some tries, if we have a big timeout, the new interval will be very big, so
this new argument is the max wait interval for the sensor waits.
For example, if we have a sensor with `poke_interval=600` and
`timeout=24*60*60`(24h)
- Without max_wait, the senor will poke at the each retry:
1. Try 1: 546s
2. Try 2: 773s
3. Try 3: 1875s
4. Try 4: 3986s
5. Try 5: 5912s
6. Try 6: 18896s
- With max_wait=1 hour, it will poke at the each retry:
1. Try 1: 546s
2. Try 2: 773s
3. Try 3: 1875s
4. Try 4: 3600s
5. Try 5: 3600s
6. Try 6: 3600s
It is the same logic used for the [TaskInstance
retries](https://github.com/apache/airflow/blob/main/airflow/models/taskinstance.py#L1123-L1124).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
