eladkal commented on code in PR #44055:
URL: https://github.com/apache/airflow/pull/44055#discussion_r1846119572
##########
providers/src/airflow/providers/amazon/aws/operators/emr.py:
##########
@@ -638,6 +642,10 @@ class EmrCreateJobFlowOperator(BaseOperator):
:param region_name: Region named passed to EmrHook
:param wait_for_completion: Whether to finish task immediately after
creation (False) or wait for jobflow
completion (True)
Review Comment:
Given the new functionality we will need to deprecate `wait_for_completion`
and add backward compatibility to the new functionality. You can see similar
example in https://github.com/apache/airflow/pull/30718
##########
providers/src/airflow/providers/amazon/aws/utils/waiter.py:
##########
@@ -83,3 +84,16 @@ def get_state(response, keys) -> str:
if value is not None:
value = value.get(key, None)
return value
+
+
+class WaitPolicy(str, Enum):
+ # Wait for the cluster to be up.
+ DEFAULT = "default"
Review Comment:
Lets not say default but be explicit about what it is.
I think we have
`wait_for_completion` - equivalent to what we have now
`wait_for_completion_with_machine_terminated` - the new functionality you
wanted
You can suggest better names
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
