o-nikolas commented on code in PR #34381: URL: https://github.com/apache/airflow/pull/34381#discussion_r1367562842
########## airflow/providers/amazon/aws/config_templates/config.yml: ########## @@ -0,0 +1,131 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +--- + +aws_ecs_executor: + description: | + This section only applies if you are using the AwsEcsExecutor in + Airflow's ``[core]`` configuration. + For more information on any of these execution parameters, see the link below: + https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ecs/client/run_task.html + For boto3 credential management, see + https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html + options: + conn_id: + description: | + The Airflow connection (i.e. credentials) used by the ECS executor to make API calls to AWS ECS. + version_added: "2.8" + type: string + example: "aws_default" + default: "aws_default" + region: + description: | + The name of the AWS Region where Amazon ECS is configured. Required. + version_added: "2.8" + type: string + example: "us-east-1" + default: ~ + assign_public_ip: + description: | + Whether to assign a public IP address to the containers launched by the ECS executor. + For more info see url to Boto3 docs above. + version_added: "2.8" + type: boolean + example: "True" + default: "False" + cluster: + description: | + Name of the Amazon ECS Cluster. Required. + version_added: "2.8" + type: string + example: "ecs_executor_cluster" + default: ~ + container_name: + description: | + Name of the container that will be used to execute Airflow tasks via the ECS executor. + The container should be specified in the ECS Task Definition and will receive an airflow + CLI command as an additional parameter to its entrypoint. For more info see url to Boto3 + docs above. Required. + version_added: "2.8" + type: string + example: "ecs_executor_container" + default: ~ + launch_type: + description: | + Launch type can either be 'FARGATE' OR 'EC2'. For more info see url to + Boto3 docs above. + + If the launch type is EC2, the executor will attempt to place tasks on + empty EC2 instances. If there are no EC2 instances available, no task + is placed and this function will be called again in the next heart-beat. + + If the launch type is FARGATE, this will run the tasks on new AWS Fargate + instances. + version_added: "2.8" + type: string + example: "FARGATE" + default: "FARGATE" + platform_version: + description: | + The platform version the task uses. A platform version is only specified + for tasks hosted on Fargate. If one isn't specified, the LATEST platform + version is used. + version_added: "2.8" + type: string + example: "1.4.0" + default: "LATEST" + security_groups: + description: | + The comma-seperated IDs of the security groups associated with the task. If you + don't specify a security group, the default security group for the VPC is used. + There's a limit of 5 security groups. For more info see url to Boto3 docs above. + version_added: "2.8" + type: string + example: "sg-XXXX,sg-YYYY" + default: ~ + subnets: + description: | + The comma-separated IDs of the subnets associated with the task or service. + There's a limit of 16 subnets. For more info see url to Boto3 docs above. + version_added: "2.8" + type: string + example: "subnet-XXXXXXXX,subnet-YYYYYYYY" + default: ~ + task_definition: + description: | + The family and revision (family:revision) or full ARN of the task definition + to run. If a revision isn't specified, the latest ACTIVE revision is used. + For more info see url to Boto3 docs above. + version_added: "2.8" + type: string + example: executor_task_definition:LATEST + default: ~ + max_run_task_attempts: + description: | + The maximum number of times the Ecs Executor should attempt to run a task. + version_added: "2.8" + type: int + example: "3" + default: "3" Review Comment: We'll see how it behaves once it's merged and we have some alpha users. We've also updated the description to clarify which retries exactly this configures. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
