potiuk commented on code in PR #30032: URL: https://github.com/apache/airflow/pull/30032#discussion_r1136167424
########## airflow/providers/amazon/aws/hooks/base_aws.py: ########## @@ -42,6 +42,7 @@ import botocore.session import requests import tenacity +from aiobotocore.session import AioSession, get_session as async_get_session Review Comment: > But what about boto3 version pin? My view: Assuming that aiobotocore will be released semi-frequently and catch-up with few months old version of boto3/botocore the worst that can happen is some new features/fixes will only be available with few months delay (when new version of aiobotocore is released and we release next version of the provider). Which is not that far from expectations of the users IMHO. Also if - for example - AWS team will see the need of acceleration of an upgrade, there is always possibility of not only opening an issue to aiobotocore team, but also making PR to update to the latest API/features (though it would likely require more involvement and learning). The users will always have an option to install newer boto3 version in Virtualenv/External or use DockerOperator - following https://airflow.apache.org/docs/apache-airflow/stable/best-practices.html#handling-conflicting-complex-python-dependencies (and we could add docs explaining this). Also airflow itself and even provider will not limit the boto3. The constraints are "golden" set of dependencies and if someone would like to use newer boto3 and not use aiobotocore extra - it will be possible for them to upgrade boto (and we can also document). There will be no "check" that the newer version will work - as we will not run tests for it - but the user will be able to do their own tests and use newer boto. We will not prevent that. I don't think we can solve all issues here - it's impossible. We have to sacrifice something. This approach is not perfect, but at least we might have steady progress with new deferrable operators for AWS being added. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
