syedahsn commented on code in PR #30032: URL: https://github.com/apache/airflow/pull/30032#discussion_r1136030496
########## airflow/providers/amazon/aws/hooks/base_aws.py: ########## @@ -42,6 +42,7 @@ import botocore.session import requests import tenacity +from aiobotocore.session import AioSession, get_session as async_get_session Review Comment: Thanks for looking at the PR! You raised some good points: The versioning of aiobotocore is a valid issue. However, as @o-nikolas pointed out, the 7 month gap seems to be an outlier. The aiobotocore library has a fairly active community that regularly releases newer versions. Also, while botocore gets updated regularly, generally, there aren't major changes between each update, so while it seems like aiobotocore may pin the botocore version, it shouldn't have major impact on user functionality. >There was plan to add more consistency (not early than 30 Apr) in hooks and add type hinting by separate existed hooks by https://github.com/apache/airflow/discussions/28560 by separate base hooks. And aiobotocore.session.AioSession is not a replacement of boto3.session.Session. This is a really cool idea, and I'm not certain where aiobotocore would fit in with this paradigm. But while you are correct that `aiobotocore.session.AioSession` is not a replacement to a `boto3.session.Session`, I think that because they both share the botocore library, there are enough similarities between the 2 libraries that the `AioSession` can fit into that model. >all DB queries are use io blocking implementation (the regular one) The purpose of this PR was to allow workers to be freed up when involved with time consuming network tasks such as polling services for a certain state etc. DB queries are io blocking operations, and we can apply those optimizations in Amazon Provider once we have a precedence for applying it across Airflow. I believe Network operation optimizations are themselves a big step forward. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
