andreas-thomik opened a new issue, #51429:
URL: https://github.com/apache/airflow/issues/51429
### Apache Airflow Provider(s)
amazon
### Versions of Apache Airflow Providers
```
apache-airflow-providers-amazon==9.7.0
```
### Apache Airflow version
3.0.1
### Operating System
Debian GNU/Linux 12 (bookworm)
### Deployment
Docker-Compose
### Deployment details
Pretty much default docker-compose deployment, but using a persistent DB
instead of the postgres container.
### What happened
The `EcsRunTaskOperator` fails when the `'containers'` list is empty. This
is because it tries to infer the `container_name` variable from the response,
without checking whether it exists or not.
We've recently updated Airflow, which highlighted the problem. Our response
(see below) has been the same for a long time. As you'll notice, the
`'containers'` list is empty. This _might_ be because the status is still
provisioning, though the AWS documentation is quite sparse on the matter.
```python
{
"tasks": [
{
"attachments": [],
"capacityProviderName":
"Infra-ECS-Cluster-***-EC2CapacityProvider-***",
"clusterArn": "arn:aws:ecs:***:***:cluster/***",
"containers": [],
"cpu": "1024",
"createdAt": datetime.datetime(
2025, 6, 5, 8, 55, 51, 381000, tzinfo=tzlocal()
),
"desiredStatus": "RUNNING",
"enableExecuteCommand": False,
"group": "family:***",
"lastStatus": "PROVISIONING",
"launchType": "EC2",
"memory": "4096",
"overrides": {
"containerOverrides": [{ ... }],
"inferenceAcceleratorOverrides": [],
},
"startedBy": "airflow",
"tags": [],
"taskArn": "arn:aws:ecs:***:***:task/***/***",
"taskDefinitionArn": "arn:aws:ecs:***:***:task-definition/***",
"version": 1,
}
],
"failures": [],
"ResponseMetadata": {
"RequestId": "***",
"HTTPStatusCode": 200,
"HTTPHeaders": {
"x-amzn-requestid": "***",
"content-type": "application/x-amz-json-1.1",
"content-length": "1360",
"date": "Thu, 05 Jun 2025 08:55:51 GMT",
},
"RetryAttempts": 0,
},
}
```
The code currently picks the first container without checking for its
existence:
```python
self.arn = response["tasks"][0]["taskArn"]
if not self.container_name:
self.container_name =
response["tasks"][0]["containers"][0]["name"]
```
### What you think should happen instead
Either handle the case of missing container names gracefully, or poll the
task until a container name is available.
### How to reproduce
Run a task on ECS which returns a response with an empty array in the
`'containers'` field.
### Anything else
Not all tasks seem to have an empty `containers` field, but those which do,
do so consistently.
The AWS boto3 documentation does not list the field as required, it's
therefore not clear whether or not this is an upstream issue or expected
behaviour, but either way it would be useful to handle it more gracefully here.
The `container_name` field seems to be a relatively recent addition
(426dba01c089c9a662301d091133cdcde5f8d729), I'm not quite clear on its usage
and whether we can simply not set it without causing downstream problems. Any
insights would be appreciated!
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
