romibuzi opened a new issue, #31208:
URL: https://github.com/apache/airflow/issues/31208
### Apache Airflow version
Other Airflow 2 version (please specify below)
### What happened
Running `apache-airflow==2.4.2`.
I recently upgraded `apache-airflow-providers-amazon` to version 8.0.0 and
when I use AWS hook with an assume_role authentication I end up with an import
error on module **aiobotocore** even if my code doesnt make usage of async:
```python
[2023-05-11 11:09:07,603] {base.py:71} INFO - Using connection ID
'aws_default' for task execution.
Traceback (most recent call last):
File "/Users/romain/airflow/scripts/s3_pipeline.py", line 108, in <module>
s3_hook.load_file(filename=temp_file.name, key="tests/test.json",
bucket_name="xxx", replace=True)
File
"/Users/romain/airflow/venv/lib/python3.7/site-packages/airflow/providers/amazon/aws/hooks/s3.py",
line 111, in wrapper
return func(*bound_args.args, **bound_args.kwargs)
File
"/Users/romain/airflow/venv/lib/python3.7/site-packages/airflow/providers/amazon/aws/hooks/s3.py",
line 82, in wrapper
return func(*bound_args.args, **bound_args.kwargs)
File
"/Users/romain/airflow/venv/lib/python3.7/site-packages/airflow/providers/amazon/aws/hooks/s3.py",
line 691, in load_file
client = self.get_conn()
File
"/Users/romain/airflow/venv/lib/python3.7/site-packages/airflow/providers/amazon/aws/hooks/base_aws.py",
line 689, in get_conn
return self.conn
File
"/Users/romain/airflow/venv/lib/python3.7/site-packages/cached_property.py",
line 36, in __get__
value = obj.__dict__[self.func.__name__] = self.func(obj)
File
"/Users/romain/airflow/venv/lib/python3.7/site-packages/airflow/providers/amazon/aws/hooks/base_aws.py",
line 649, in conn
return self.get_client_type(region_name=self.region_name)
File
"/Users/romain/airflow/venv/lib/python3.7/site-packages/airflow/providers/amazon/aws/hooks/base_aws.py",
line 605, in get_client_type
session = self.get_session(region_name=region_name,
deferrable=deferrable)
File
"/Users/romain/airflow/venv/lib/python3.7/site-packages/airflow/providers/amazon/aws/hooks/base_aws.py",
line 578, in get_session
).create_session(deferrable=deferrable)
File
"/Users/romain/airflow/venv/lib/python3.7/site-packages/airflow/providers/amazon/aws/hooks/base_aws.py",
line 188, in create_session
session_kwargs=assume_session_kwargs, deferrable=deferrable
File
"/Users/romain/airflow/venv/lib/python3.7/site-packages/airflow/providers/amazon/aws/hooks/base_aws.py",
line 197, in _create_session_with_assume_role
from aiobotocore.session import get_session as async_get_session
ModuleNotFoundError: No module named 'aiobotocore'
```
### What you think should happen instead
I think aiobotocore should be used only when async code is involved
### How to reproduce
```python
import os
import json
from tempfile import NamedTemporaryFile
from airflow.models import Connection
from airflow.providers.amazon.aws.hooks.s3 import S3Hook
if __name__ == "__main__":
aws_default = Connection(
conn_id="aws_default",
conn_type="aws",
extra=json.dumps(
{
"role_arn": "arn:aws:iam::XXXX:role/ROLE_NAME",
"region_name": "eu-central-1",
}
),
)
os.environ["AIRFLOW_CONN_AWS_DEFAULT"] = aws_default.get_uri()
s3_hook = S3Hook("aws_default")
with NamedTemporaryFile("w") as temp_file:
s3_hook.load_file(filename=temp_file.name, key="test.txt",
bucket_name="<YOUR_BUCKET>", replace=True)
```
### Operating System
MacOS
### Versions of Apache Airflow Providers
apache-airflow-providers-amazon==8.0.0
### Deployment
Virtualenv installation
### Deployment details
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
