o-nikolas commented on a change in pull request #19665:
URL: https://github.com/apache/airflow/pull/19665#discussion_r753513595
##########
File path: airflow/providers/amazon/aws/hooks/redshift.py
##########
@@ -138,6 +138,26 @@ def create_cluster_snapshot(self, snapshot_identifier:
str, cluster_identifier:
)
return response['Snapshot'] if response['Snapshot'] else None
+ def wait_for_state(self, cluster_identifier: str, target_state: str,
check_interval: float) -> None:
Review comment:
Thanks for the contribution!
Instead of writing this wait method in the hook and injecting it inside many
of the operators you can instead create a sensor for waiting for states. That
sensor can then be added to dags as needed. It's a bit more graceful of a
solution especially when combined with something like Smart Sensors.
You can see an example in the recently developed EKS code. You can see the
sensor here:
https://github.com/apache/airflow/blob/752575cb84a28c41723b5151fd6451a72f4da5fb/airflow/providers/amazon/aws/sensors/eks.py#L54
And you can see it's usage in the EKS example dags, for example:
https://github.com/apache/airflow/blob/752575cb84a28c41723b5151fd6451a72f4da5fb/airflow/providers/amazon/aws/example_dags/example_eks_with_nodegroup_in_one_step.py#L93
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
