phanikumv commented on code in PR #32029:
URL: https://github.com/apache/airflow/pull/32029#discussion_r1241712428
##########
docs/apache-airflow-providers-amazon/operators/emr/emr.rst:
##########
@@ -188,6 +188,10 @@ Wait on an Amazon EMR job flow state
To monitor the state of an EMR job flow you can use
:class:`~airflow.providers.amazon.aws.sensors.emr.EmrJobFlowSensor`.
+This sensor can be run in deferrable mode by passing ``deferrable=True`` as a
parameter.
Review Comment:
```suggestion
This sensor can be run in deferrable mode by passing ``deferrable=True`` as
a parameter.
```
##########
airflow/providers/amazon/aws/sensors/emr.py:
##########
@@ -488,6 +499,26 @@ def failure_message_from_response(response: dict[str,
Any]) -> str | None:
)
return None
+ def execute(self, context: Context) -> None:
+ if not self.deferrable:
+ super().execute(context=context)
+ else:
+ self.defer(
Review Comment:
same comment as above.what if job is already complete before deferring? You
dont need to defer then, so lets introduce a check whether job is in a terminal
state before deferring
##########
airflow/providers/amazon/aws/operators/emr.py:
##########
@@ -556,6 +560,22 @@ def execute(self, context: Context) -> str | None:
self.client_request_token,
self.tags,
)
+ if self.deferrable:
Review Comment:
what if job is already complete before deferring? You dont need to defer
then, so lets introduce a check whether job is in a terminal state before
deferring
##########
docs/apache-airflow-providers-amazon/operators/emr/emr.rst:
##########
@@ -202,6 +206,10 @@ Wait on an Amazon EMR step state
To monitor the state of an EMR job step you can use
:class:`~airflow.providers.amazon.aws.sensors.emr.EmrStepSensor`.
+This sensor can be run in deferrable mode by passing ``deferrable=True`` as a
parameter.
Review Comment:
```suggestion
This sensor can be run in deferrable mode by passing ``deferrable=True`` as
a parameter.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
