o-nikolas commented on code in PR #30864:
URL: https://github.com/apache/airflow/pull/30864#discussion_r1177085989
##########
airflow/providers/amazon/aws/operators/redshift_cluster.py:
##########
@@ -436,64 +438,55 @@ def __init__(
aws_conn_id: str = "aws_default",
deferrable: bool = False,
poll_interval: int = 10,
+ max_attempts: int = 10,
Review Comment:
Can you update the doc string to describe this param. (and on the other hand
you added `deferrable` to the docs but it's not in the signature here?
##########
airflow/providers/amazon/aws/operators/redshift_cluster.py:
##########
@@ -436,64 +438,55 @@ def __init__(
aws_conn_id: str = "aws_default",
deferrable: bool = False,
poll_interval: int = 10,
+ max_attempts: int = 10,
**kwargs,
):
super().__init__(**kwargs)
self.cluster_identifier = cluster_identifier
self.aws_conn_id = aws_conn_id
self.deferrable = deferrable
+ self.max_attempts = max_attempts
self.poll_interval = poll_interval
- # These parameters are added to address an issue with the boto3 API
where the API
+ # These parameters are used to address an issue with the boto3 API
where the API
# prematurely reports the cluster as available to receive requests.
This causes the cluster
# to reject initial attempts to resume the cluster despite reporting
the correct state.
self._attempts = 10
self._attempt_interval = 15
def execute(self, context: Context):
redshift_hook = RedshiftHook(aws_conn_id=self.aws_conn_id)
+ self.log.info("Starting resume cluster")
+ while self._attempts >= 1:
+ try:
+
redshift_hook.get_conn().resume_cluster(ClusterIdentifier=self.cluster_identifier)
+ break
+ except
redshift_hook.get_conn().exceptions.InvalidClusterStateFault as error:
+ self._attempts = self._attempts - 1
+ if self._attempts > 0:
+ self.log.error("Unable to resume cluster. %d attempts
remaining.", self._attempts)
+ time.sleep(self._attempt_interval)
+ else:
+ raise error
+ self.log.info("Starting resume cluster")
Review Comment:
You've already logged this?
##########
airflow/providers/amazon/aws/operators/redshift_cluster.py:
##########
@@ -423,6 +424,7 @@ class RedshiftResumeClusterOperator(BaseOperator):
The default connection id is ``aws_default``
:param deferrable: Run operator in deferrable mode
:param poll_interval: Time (in seconds) to wait between two consecutive
calls to check cluster state
+ :param deferrable: If True, the operator will run as a deferrable operator.
Review Comment:
There already is a param for deferrable above?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
