dimon222 opened a new pull request #19725: URL: https://github.com/apache/airflow/pull/19725
What is this? This PR is to help with consequences of https://github.com/apache/airflow/commit/867e9305f08bf9580f25430d8b6e84071c59f9e6 Currently out of the box base_aws implementation is going to show warnings whenever connection cache is invoked (that is potentially always when you use any AWS-based resource in Airflow like S3 as storage for logging or sensors) Why? Newly added warning for deprecation is shown every time anyone is passing `not None` value to `client_type` to `get_client_type` method. Same is happening with `not None` value `resource_type` being passed to `get_resource_type`. While this new deprecation warning might have been created with good intentions, the out of the box calls to mentioned methods weren't modified accordingly (missed during review?), so right now Airflow is barking on its own code for deprecation on any AWS-hook based component. Condition 1: https://github.com/apache/airflow/blob/aa2cb5545f09d694b9143b323efcd4f6b6c66e60/airflow/providers/amazon/aws/hooks/base_aws.py#L444-L449 Calls: https://github.com/apache/airflow/blob/aa2cb5545f09d694b9143b323efcd4f6b6c66e60/airflow/providers/amazon/aws/hooks/base_aws.py#L493-L494 https://github.com/apache/airflow/blob/aa2cb5545f09d694b9143b323efcd4f6b6c66e60/airflow/providers/amazon/aws/hooks/base_aws.py#L541-L542 (for `iam` call please read the notes on the very end of this PR message) Condition 2: https://github.com/apache/airflow/blob/aa2cb5545f09d694b9143b323efcd4f6b6c66e60/airflow/providers/amazon/aws/hooks/base_aws.py#L469-L474 Calls: https://github.com/apache/airflow/blob/aa2cb5545f09d694b9143b323efcd4f6b6c66e60/airflow/providers/amazon/aws/hooks/base_aws.py#L495-L496 --- Now, to mitigate this I took next approach: I pass temporarily None in mentioned arguments until the code is refactored to get rid of entire argument in method (I assume somewhen in the future?). Few notes: 1. One piece of code that I randomly noticed is that IAM request is not standard (see call 2 in above mentioned cuts, and is using special hardcoded value "iam"). As I'm not certain about all the specifics of Airflow internals I have added "iam" to list of exceptions for deprecation for time being. Feel free to modify to proper approach. 2. There might be other invocations of mentioned methods with "not None" being passed by Airflow codebase, but unfortunately, I haven't done deep inspection of entire codebase to verify. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
