vincbeck commented on code in PR #44055: URL: https://github.com/apache/airflow/pull/44055#discussion_r1851999154
########## newsfragments/44055.feature.rst: ########## @@ -0,0 +1 @@ +New argument ``wait_policy`` to control waiting behaviour when using ``EmrCreateJobFlowOperator``. Review Comment: You do not need this file, it is not a breaking change and more importantly, it is a provider change, not an Airflow change. Thus, you can just remove this file :) ########## docs/apache-airflow-providers-amazon/operators/emr/emr.rst: ########## @@ -47,6 +47,15 @@ Create an EMR job flow You can use :class:`~airflow.providers.amazon.aws.operators.emr.EmrCreateJobFlowOperator` to create a new EMR job flow. The cluster will be terminated automatically after finishing the steps. + +The default behaviour is to mark the DAG Task node as success as soon as the cluster is launched +(``wait_policy=None``). +It is possible to modify this behaviour by using a different ``wait_policy``. Available options are: + +- ``WaitPolicy.WAIT_FOR_COMPLETION`` - DAG Task node waits for the cluster to be Running +- ``WaitPolicy.WAIT_FOR_STEPS_COMPLETION`` - DAG Task node waits for the cluster to Terminate Review Comment: nits ```suggestion - ``WaitPolicy.WAIT_FOR_COMPLETION`` - DAG Task node waits for the cluster to be running - ``WaitPolicy.WAIT_FOR_STEPS_COMPLETION`` - DAG Task node waits for the cluster to terminate ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
