ferruzzi commented on code in PR #37786:
URL: https://github.com/apache/airflow/pull/37786#discussion_r1508383523
##########
airflow/providers/amazon/aws/executors/ecs/ecs_executor.py:
##########
@@ -493,3 +496,39 @@ def get_container(self, container_list):
'container "name" must be provided in "containerOverrides"
configuration'
)
raise KeyError(f"No such container found by container name:
{self.container_name}")
+
+ def try_adopt_task_instances(self, tis: Sequence[TaskInstance]) ->
Sequence[TaskInstance]:
+ """
+ Adopt task instances which have an external_executor_id (the ECS task
ARN).
+
+ Anything that is not adopted will be cleared by the scheduler and
becomes eligible for re-scheduling.
+ """
+ with Stats.timer("ecs_executor.adopt_task_instances.duration"):
Review Comment:
Here's a quick demo. The timestamps don't line up perfectly because of the
delay in metric collection, but roughly:
| Time | # Dags | Seconds to Adopt |
|----------|--------|------------------|
| 00:55:42 | 0 | 0.0000346 |
| 00:56:08 | 2 | 0.988 |
| 00:58:24 | 8 | 3.79 |
| 01:00:53 | 3 | 1.41 |
| 01:04:11 | 2 | 0.952 |
| 01:14:00 | 0 | 0.0000144 |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
