0x26res commented on code in PR #39608:
URL: https://github.com/apache/airflow/pull/39608#discussion_r1601137265
##########
airflow/providers/amazon/aws/operators/batch.py:
##########
@@ -206,8 +206,8 @@ def __init__(
self.scheduling_priority_override = scheduling_priority_override
self.array_properties = array_properties
self.parameters = parameters or {}
- self.retry_strategy = retry_strategy or {}
- if not self.retry_strategy.get("attempts", None):
+ self.retry_strategy = retry_strategy
+ if self.retry_strategy is not None and not
self.retry_strategy.get("attempts", None):
Review Comment:
@o-nikolas I don't disagree with you, but specifying the default number of
attempts explicitly is not necessary.
Prior to https://github.com/apache/airflow/pull/35789 and
https://github.com/apache/airflow/pull/35808 the aws `BatchOperator` didn't
specify any retryStrategy when submitting the job with boto. That meant it
would **use the retry strategy set in the job definition**. The retry strategy
specified in the job definition is either specified by the user, or if the user
doesn't specify one, it will have one attempt. See "By default, each job is
given one attempt to move to either the SUCCEEDED or FAILED job state"
[here](https://docs.aws.amazon.com/batch/latest/userguide/job_retries.html)
The new behaviour after these 2 MRs is, if the user doesn't specify the
retry policy, to **ignore the job definition retry strategy**. This constitute
a breaking change / change in the API behaviour.
Now given when the job definition doesn't have a custom strategy, and the
retry strategy is not specified in boto3, it will fallback to the default of 1
attempt, there is no need to specify a explicitly a `{"attempts":1}` in the
retry strategy. So it's best to leave it to None. And then we will preserve the
previous behaviour.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
