[airflow] branch main updated: Add implementation note on iterative approach (#31997)

[jedcunningham]

This is an automated email from the ASF dual-hosted git repository.

jedcunningham pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


The following commit(s) were added to refs/heads/main by this push:
     new 3dda1557a1 Add implementation note on iterative approach (#31997)
3dda1557a1 is described below

commit 3dda1557a19c2ae3923d6ce753be8b5a93831cbb
Author: Tzu-ping Chung <uranu...@gmail.com>
AuthorDate: Thu Jun 22 22:48:11 2023 +0800

    Add implementation note on iterative approach (#31997)
---
 airflow/models/abstractoperator.py | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/airflow/models/abstractoperator.py 
b/airflow/models/abstractoperator.py
index a119c5e2c1..bf06b39da9 100644
--- a/airflow/models/abstractoperator.py
+++ b/airflow/models/abstractoperator.py
@@ -156,8 +156,7 @@ class AbstractOperator(Templater, DAGNode):
         return self.downstream_task_ids
 
     def get_flat_relative_ids(self, *, upstream: bool = False) -> set[str]:
-        """
-        Get a flat set of relative IDs, upstream or downstream.
+        """Get a flat set of relative IDs, upstream or downstream.
 
         Will recurse each relative found in the direction specified.
 
@@ -169,6 +168,10 @@ class AbstractOperator(Templater, DAGNode):
 
         relatives: set[str] = set()
 
+        # This is intentionally implemented as a loop, instead of calling
+        # get_direct_relative_ids() recursively, since Python has significant
+        # limitation on stack level, and a recursive implementation can blow up
+        # if a DAG contains very long routes.
         task_ids_to_trace = self.get_direct_relative_ids(upstream)
         while task_ids_to_trace:
             task_ids_to_trace_next: set[str] = set()