[
https://issues.apache.org/jira/browse/AIRFLOW-3215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ash Berlin-Taylor closed AIRFLOW-3215.
--------------------------------------
Resolution: Not A Problem
> Creating EMR using python from airflow
> --------------------------------------
>
> Key: AIRFLOW-3215
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3215
> Project: Apache Airflow
> Issue Type: Task
> Components: aws, DAG
> Affects Versions: 1.7.0
> Environment: Airflow with boto3 - connecting AWS -configure with
> access and security
> Reporter: Pandu
> Priority: Major
>
> I have problem with imports while creating EMR.
> import boto3
> connection = boto3.client(
> 'emr'
> )
> cluster_id = connection.run_job_flow(
> Name='emr123',
> LogUri='s3://emr-spark-application/log.txt',
> ReleaseLabel='emr-4.1.0',
> Instances={
> 'InstanceGroups': [
> {
> 'Name': "Master nodes",
> 'Market': 'ON_DEMAND',
> 'InstanceRole': 'MASTER',
> 'InstanceType': 'm1.large',
> 'InstanceCount': 1
> },
> {
> 'Name': "Slave nodes",
> 'Market': 'ON_DEMAND',
> 'InstanceRole': 'CORE',
> 'InstanceType': 'm1.large',
> 'InstanceCount': 1
> }
> ],
> 'KeepJobFlowAliveWhenNoSteps': True,
> 'TerminationProtected': False
> },
> Applications=[{
> 'Name': 'Hadoop'
> }, {
> 'Name': 'Spark'
> }],
> JobFlowRole='EMR_EC2_DefaultRole',
> ServiceRole='EMR_DefaultRole'
> )
> print (cluster_id['JobFlowId'])
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
