ashb commented on a change in pull request #6655: [AIRFLOW-6056] Allow
EmrAddStepsOperator to accept job_flow_name as alternative to job_flow_id
URL: https://github.com/apache/airflow/pull/6655#discussion_r352484275
##########
File path: airflow/contrib/hooks/emr_hook.py
##########
@@ -36,6 +36,26 @@ def get_conn(self):
self.conn = self.get_client_type('emr', self.region_name)
return self.conn
+ def get_cluster_id_by_name(self, emr_cluster_name):
+ conn = self.get_conn()
+
+ response = conn.list_clusters(
+ ClusterStates=[
+ 'RUNNING', 'WAITING'
+ ]
+ )
+
+ matching_clusters = list(
+ filter(lambda cluster: cluster['Name'] == emr_cluster_name,
response['Clusters'])
+ )
+
+ if len(matching_clusters) > 0:
Review comment:
If there is more than a single cluster of the given name that should
probably be an error.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
