vladyslavnsa opened a new issue #22381:
URL: https://github.com/apache/airflow/issues/22381
### Apache Airflow version
2.2.4 (latest released)
### What happened
After a recent upgrade from 1.10.9 to 2.2.4, an odd behavior where the
aforementioned Attributes, are wrongfully Coupled, is observed.
An example to showcase the issue:
```
AthenaOperator(
...
retries=3,
max_tries=30,
...)
```
Related Documentation states:
* retries: Number of retries that should be performed before failing the task
* max_tries: Number of times to poll for query state before function exits
Regardless of the above specification `max_tries=30`, inspection of related
_Task Instance Details_ shows that the Value of both Attributes is **3**
### What you think should happen instead
_No response_
### How to reproduce
_No response_
### Operating System
Ubuntu 20.04.3 LTS
### Versions of Apache Airflow Providers
apache-airflow-providers-amazon==3.0.0
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
_No response_
### Anything else
Imagine a Query, executed on an hourly basis, with a varying scope, causing
it to 'organically' execute for anywhere between 5 - 10 minutes. This Query
Task should Fail after 3 execution attempts.
In such cases, we would like to poll the state of the Query frequently
(every 15 seconds), in order to avoid redundant idle time for downstream Tasks.
A configuration matching the above description:
```
AthenaOperator(
...
retry_delay=15,
retries=3,
max_tries=40, # 40 polls * 15 seconds delay between polls = 10 minutes
...)
```
When deployed, `retries == max_tries == 3`, thus causing the Task to
terminate after 45 seconds
In order to quickly avert this situation where our ETL breaks, we are using
the following configuration:
```
AthenaOperator(
...
retry_delay=15,
retries=40,
max_tries=40,
...)
```
With the last configuration, our Task does not terminate preemptively but
will retry **40 times** before failing - which causes an issue with downstream
Tasks SLA, at the very least (that is before weighing in the waste of time and
operational costs)
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
