Taragolis commented on code in PR #34381: URL: https://github.com/apache/airflow/pull/34381#discussion_r1336461473
########## airflow/providers/amazon/aws/config_templates/config.yml: ########## @@ -0,0 +1,131 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +--- + +aws_ecs_executor: + description: | + This section only applies if you are using the AwsEcsExecutor in + Airflow's ``[core]`` configuration. + For more information on any of these execution parameters, see the link below: + https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ecs/client/run_task.html + For boto3 credential management, see + https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html + options: + conn_id: + description: | + The Airflow connection (i.e. credentials) used by the ECS executor to make API calls to AWS ECS. + version_added: "2.8" + type: string + example: "aws_default" + default: "aws_default" Review Comment: Yes it have a fallback, I think this is one of the dirtiest fallback. I guess only Amazon Provider has this behavior with connections: when Airflow can’t lookup connection from Metabase and secrets backend it just fallback to boto3 credentials strategy. And there is cost for this fallback: 1. need to lookup every time for non-existed connection 2. It could be some misconfiguration with secrets backend, and fallback might hide this 3. Annoying warning message In case of None it use default boto3 strategy without even touch Connection. ########## airflow/providers/amazon/aws/config_templates/config.yml: ########## @@ -0,0 +1,131 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +--- + +aws_ecs_executor: + description: | + This section only applies if you are using the AwsEcsExecutor in + Airflow's ``[core]`` configuration. + For more information on any of these execution parameters, see the link below: + https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ecs/client/run_task.html + For boto3 credential management, see + https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html + options: + conn_id: + description: | + The Airflow connection (i.e. credentials) used by the ECS executor to make API calls to AWS ECS. + version_added: "2.8" + type: string + example: "aws_default" + default: "aws_default" Review Comment: Yes it have a fallback, I think this is one of the dirtiest fallback. I guess only Amazon Provider has this behavior with connections: when Airflow can’t lookup connection from Metabase and secrets backend it just fallback to boto3 credentials strategy. And there is cost for this fallback: 1. need to lookup every time for non-existed connection 2. It could be some misconfiguration with secrets backend, and fallback might hide this 3. Annoying warning message In case of None it use default boto3 strategy without even touch Connection. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
