Dafyddjb opened a new issue, #29960:
URL: https://github.com/apache/airflow/issues/29960
### Apache Airflow Provider(s)
amazon
### Versions of Apache Airflow Providers
apache-airflow-providers-amazon = "7.3.0"
### Apache Airflow version
2.5.1
### Operating System
Debian GNU/Linux
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
_No response_
### What happened
After updating the provider version to 7.3.0 from 6.0.0, our glue jobs
started failing. We currently use the GlueJobOperator to run existing Glue jobs
that we manage in Terraform. The full traceback is below:
```
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.9/site-packages/airflow/providers/amazon/aws/operators/glue.py",
line 150, in execute
glue_job_run = glue_job.initialize_job(self.script_args,
self.run_job_kwargs)
File
"/home/airflow/.local/lib/python3.9/site-packages/airflow/providers/amazon/aws/hooks/glue.py",
line 165, in initialize_job
job_name = self.create_or_update_glue_job()
File
"/home/airflow/.local/lib/python3.9/site-packages/airflow/providers/amazon/aws/hooks/glue.py",
line 325, in create_or_update_glue_job
config = self.create_glue_job_config()
File
"/home/airflow/.local/lib/python3.9/site-packages/airflow/providers/amazon/aws/hooks/glue.py",
line 108, in create_glue_job_config
execution_role = self.get_iam_execution_role()
File
"/home/airflow/.local/lib/python3.9/site-packages/airflow/providers/amazon/aws/hooks/glue.py",
line 143, in get_iam_execution_role
glue_execution_role = iam_client.get_role(RoleName=self.role_name)
File
"/home/airflow/.local/lib/python3.9/site-packages/botocore/client.py", line
530, in _api_call
return self._make_api_call(operation_name, kwargs)
File
"/home/airflow/.local/lib/python3.9/site-packages/botocore/client.py", line
919, in _make_api_call
request_dict = self._convert_to_request_dict(
File
"/home/airflow/.local/lib/python3.9/site-packages/botocore/client.py", line
990, in _convert_to_request_dict
request_dict = self._serializer.serialize_to_request(
File
"/home/airflow/.local/lib/python3.9/site-packages/botocore/validate.py", line
381, in serialize_to_request
raise ParamValidationError(report=report.generate_report())
botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid type for parameter RoleName, value: None, type: <class 'NoneType'>,
valid types: <class 'str'>
```
### What you think should happen instead
The operator creates a new job run for a glue job without additional
configuration.
### How to reproduce
Create a DAG with a GlueJobOperator without using `script_location`,
`s3_bucket` and `iam_role_name`. Example:
```python
task = GlueJobOperator(task_id="glue-task", job_name=<glue-job-name>)
```
### Anything else
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
