GitHub user prayaagmahajan created a discussion: Facing Inter-Task Delay while
executing nested pipeline tasks
I am experiencing an unexpected inter-task delay while executing a pipeline in
**Airflow 3.1.7.**
airflow==3.1.7
apache-airflow-providers-celery==3.13.0
**Pipeline Structure**
- Master Pipeline (DAG)
- Contains a Nested Pipeline
- The nested pipeline contains multiple pipeline tasks
- Each pipeline task executes three SQL tasks sequentially
**Issue Observed**
Within one of the pipeline tasks:
- The first two SQL tasks execute successfully and complete without delay.
- The third SQL task, however, starts approximately 40–45 minutes after the
second SQL task completes.
Due to this delay, the total execution time of the pipeline task becomes
approximately 43 minutes, even though the actual SQL execution time is minimal.
Below is the SQL entries for the tasks.
+--------------------------------------+----------------------------+----------------------------+
| demo_task | start_date
| end_date |
+--------------------------------------+----------------------------+----------------------------+
| START1 | 2026-02-18 11:03:00.572031
| 2026-02-18 11:03:05.461282 |
| SQL_TASK_1 | 2026-02-18 **11:05:03**.191372
| 2026-02-18 11:05:05.756707 |
| SQL_TASK_2 | 2026-02-18 **11:39:30**.286825 |
2026-02-18 11:39:32.576455 |
| SQL_TASK_3 | 2026-02-18 11:39:30.300086
| 2026-02-18 11:39:47.026262 |
GitHub link: https://github.com/apache/airflow/discussions/62120
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
