[
https://issues.apache.org/jira/browse/AIRFLOW-247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15998681#comment-15998681
]
Al Johri edited comment on AIRFLOW-247 at 5/5/17 6:01 PM:
----------------------------------------------------------
I'm searching for documentation related to how Airflow works with EMR. I'm
struggling to find anything here:
https://airflow.incubator.apache.org/integration.html#aws
My main question is, can Airflow create an EMR cluster and bring it back down
like AWS Data Pipeline?
Thanks!
EDIT: Found some information here:
https://docs.google.com/presentation/d/1NG1P86HRlX43qTVucCTOsFqIbCvYdOhq_np90VlbVRc/edit#slide=id.gd40eeee67_1_0
was (Author: al.johri):
I'm searching for documentation related to how Airflow works with EMR. I'm
struggling to find anything here:
https://airflow.incubator.apache.org/integration.html#aws
My main question is, can Airflow create an EMR cluster and bring it back down
like AWS Data Pipeline?
Thanks!
> EMR Hook, Operators, Sensor
> ---------------------------
>
> Key: AIRFLOW-247
> URL: https://issues.apache.org/jira/browse/AIRFLOW-247
> Project: Apache Airflow
> Issue Type: New Feature
> Reporter: Rob Froetscher
> Assignee: Rob Froetscher
> Priority: Minor
>
> Substory of https://issues.apache.org/jira/browse/AIRFLOW-115. It would be
> nice to have an EMR hook and operators.
> Hook to generally interact with EMR.
> Operators to:
> * setup and start a job flow
> * add steps to an existing jobflow
> A sensor to:
> * monitor completion and status of EMR jobs
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
