[ https://issues.apache.org/jira/browse/AIRFLOW-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297881#comment-16297881 ]
Vikram Fugro commented on AIRFLOW-1936: --------------------------------------- ok, thanks! > EmrCreateJobFlowOperator is unable to launch the aws emr cluster > ---------------------------------------------------------------- > > Key: AIRFLOW-1936 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1936 > Project: Apache Airflow > Issue Type: Bug > Components: aws, boto3, DAG, hooks, operators > Affects Versions: Airflow 1.8 > Environment: ubuntu 16.04 , python 2.7, boto3 > Reporter: Vikram Fugro > Labels: newbie > Attachments: dag_logs.txt, emr_hook_with_hardcoded_values.py, > my_emr_boto_script(works).py, my_emr_dag(doesNotWork).py > > > The EmrCreateJobFlowOperator operator is unable to create the emr cluster, > although it returns success. The most strange thing that I see in the logs > is that it also returns the jobflow-Id but I see no cluster starting up in > the aws emr console. If I try to launch the emr with my boto3 script , it > works. I even hardcoded the arguments (taken from my boto3 script as is) in > self.get_conn().run_job_flow(), in contrib/hooks/emr_hook.py , but no luck. > I am running airflow 1.8, localexecutor with metadb postgres. I have tried > both ways ; setting up a schedule and also by command 'airflow run -f > emr_job_flow_manual_steps_dag2 create_job_flow 2017-12-18' > I am attaching the logs of my dag, my dag itself , my boto3 script and > emr_hook.py(changed only for testing purpose). -- This message was sent by Atlassian JIRA (v6.4.14#64029)