This is an automated email from the ASF dual-hosted git repository.
henry3260 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new f23c096635f Stop airflowctl integration tests timing out right after a
Dag run starts (#72523)
f23c096635f is described below
commit f23c096635f09b3f610bd9a72fa7874d1a173bc7
Author: Andrew Chang <[email protected]>
AuthorDate: Sat Sep 5 02:18:44 2026 +0800
Stop airflowctl integration tests timing out right after a Dag run starts
(#72523)
The compose stack used by the tests runs one api-server process that also
serves the Execution API. With the default Celery concurrency of 16, one
trigger of example_bash_operator forks more than ten task processes at once
on the 4 vCPU runner, and the api-server gets no CPU for 20 to 35 seconds.
airflowctl commands running in that window hit the 5 second read timeout
three times and exit 1, which is the flaky failure seen in
test_airflowctl_commands[tasks failed-deps0] and neighbouring commands.
Limiting the worker to 2 concurrent tasks in the test environment removes
the burst. Under the same CPU pressure that produced a 19 second stall
locally, the health endpoint never took more than 2 seconds afterwards.
---
airflow-ctl-tests/tests/airflowctl_tests/conftest.py | 3 +++
1 file changed, 3 insertions(+)
diff --git a/airflow-ctl-tests/tests/airflowctl_tests/conftest.py
b/airflow-ctl-tests/tests/airflowctl_tests/conftest.py
index 227f9d7d431..878c2495401 100644
--- a/airflow-ctl-tests/tests/airflowctl_tests/conftest.py
+++ b/airflow-ctl-tests/tests/airflowctl_tests/conftest.py
@@ -370,6 +370,9 @@ def docker_compose_up(tmp_path_factory):
f"AIRFLOW_UID={os.getuid()}\n"
# To enable config operations to work
"AIRFLOW__API__EXPOSE_CONFIG=true\n"
+ # The single api-server process also serves the Execution API; a burst
of task processes
+ # starting at once starves it of CPU on small CI runners and
airflowctl calls time out.
+ "AIRFLOW__CELERY__WORKER_CONCURRENCY=2\n"
)
# Set environment variables for the test
