GitHub user niroam created a discussion: Glue Operator - Update Job and Tags issue
https://github.com/apache/airflow/blob/3c2163729d353b2eb723907ed8e286d94e601c81/providers/amazon/src/airflow/providers/amazon/aws/hooks/glue.py#L463 Hi Team, When creating and running a new Glue Job via this operator e.g GlueJobOperator_1765238188320 = GlueJobOperator( task_id='job-task', job_name='mygluejob, script_location='test.py', s3_bucket='mys3bucket', create_job_kwargs={ 'GlueVersion': GLUE_JOB_CONFIG['GlueVersion'], 'NumberOfWorkers': GLUE_JOB_CONFIG['NumberOfWorkers'], 'WorkerType': GLUE_JOB_CONFIG['WorkerType'], 'Timeout': GLUE_JOB_CONFIG['Timeout'], 'MaxRetries': GLUE_JOB_CONFIG['MaxRetries'], 'ExecutionClass': GLUE_JOB_CONFIG['ExecutionClass'], 'Command': GLUE_JOB_CONFIG['Command'], 'DefaultArguments': GLUE_JOB_CONFIG['DefaultArguments'], 'Connections': GLUE_JOB_CONFIG['Connections'], 'Tags': GLUE_JOB_CONFIG['Tags'] }, iam_role_arn=project.iam_role, update_config=True ) the boto3 create_job supports Tags, however update_job does not should we be removing these from the args in our update job method in the airflow operator ? do we need a seperate check on tags changing here ? GitHub link: https://github.com/apache/airflow/discussions/59238 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
