phanikumv commented on issue #25996:
URL: https://github.com/apache/airflow/issues/25996#issuecomment-1240712467
@kumbarsg I tried with below DAG and the task ran fine.
```
with DAG(
dag_id="example_aws_glue_dag",
schedule_interval=None,
start_date=datetime(2022, 1, 1),
catchup=False,
) as dag:
glue_task = GlueJobOperator(
task_id="glue_task",
job_name = 'glue_job_from_airflow',
job_desc = 'AWS Glue Job with Airflow',
script_location =
"s3://aws-glue-assets-*********-us-east-2/scripts/providers-job-2.py",
create_job_kwargs={
'GlueVersion': '2.0', 'NumberOfWorkers': 2, 'WorkerType': 'G.1X',
},
s3_bucket="aws-glue-assets-********-us-east-2",
iam_role_name="glue_role"
)
```
```
22-09-08, 13:04:29 UTC] {glue.py:142} INFO - Initializing AWS Glue Job:
glue_job_from_airflow. Wait for completion: True
[2022-09-08, 13:04:29 UTC] {base.py:69} INFO - Using connection ID
'aws_default' for task execution.
[2022-09-08, 13:04:29 UTC] {connection_wrapper.py:292} INFO - AWS Connection
(conn_id='aws_default', conn_type='aws') credentials retrieved from login and
password.
[2022-09-08, 13:04:30 UTC] {glue.py:246} INFO - Job doesn't exist. Now
creating and running AWS Glue Job
[2022-09-08, 13:04:31 UTC] {glue.py:107} INFO - Iam Role Name: glue_role
[2022-09-08, 13:04:32 UTC] {glue.py:222} INFO - Polling for AWS Glue Job
glue_job_from_airflow current run state with status RUNNING
[2022-09-08, 13:04:38 UTC] {glue.py:222} INFO - Polling for AWS Glue Job
glue_job_from_airflow current run state with status RUNNING
[2022-09-08, 13:04:44 UTC] {glue.py:222} INFO - Polling for AWS Glue Job
glue_job_from_airflow current run state with status RUNNING
[2022-09-08, 13:04:51 UTC] {glue.py:222} INFO - Polling for AWS Glue Job
glue_job_from_airflow current run state with status RUNNING
[2022-09-08, 13:04:57 UTC] {glue.py:222} INFO - Polling for AWS Glue Job
glue_job_from_airflow current run state with status RUNNING
[2022-09-08, 13:05:03 UTC] {glue.py:222} INFO - Polling for AWS Glue Job
glue_job_from_airflow current run state with status RUNNING
[2022-09-08, 13:05:10 UTC] {glue.py:222} INFO - Polling for AWS Glue Job
glue_job_from_airflow current run state with status RUNNING
[2022-09-08, 13:05:16 UTC] {glue.py:222} INFO - Polling for AWS Glue Job
glue_job_from_airflow current run state with status RUNNING
[2022-09-08, 13:05:22 UTC] {glue.py:222} INFO - Polling for AWS Glue Job
glue_job_from_airflow current run state with status RUNNING
[2022-09-08, 13:05:29 UTC] {glue.py:211} INFO - Exiting Job
jr_a3274a900afd04e35d2e8112af725fd403da0579a090de1bccec457c4995631d Run State:
SUCCEEDED
[2022-09-08, 13:05:29 UTC] {glue.py:151} INFO - AWS Glue Job:
glue_job_from_airflow status: SUCCEEDED. Run Id:
jr_a3274a900afd04e35d2e8112af725fd403da0579a090de1bccec457c4995631d
[2022-09-08, 13:05:29 UTC] {taskinstance.py:1411} INFO - Marking task as
SUCCESS. dag_id=example_aws_glue_dag, task_id=glue_task,
execution_date=20220908T130426, start_date=20220908T130428,
end_date=20220908T130529
[2022-09-08, 13:05:29 UTC] {local_task_job.py:163} INFO - Task exited with
return code 0
[2022-09-08, 13:05:29 UTC] {local_task_job.py:272} INFO - 0 downstream tasks
scheduled from follow-on schedule check
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
