steveahnahn commented on code in PR #71769:
URL: https://github.com/apache/airflow/pull/71769#discussion_r3997625498
##########
airflow-core/src/airflow/models/taskinstance.py:
##########
@@ -403,6 +403,13 @@ def clear_task_instances(
from airflow.models.dagbag import DBDagBag
scheduler_dagbag = DBDagBag(load_op_links=False)
+ # Cache per-dag lookups of the latest serialized DAG and DagVersion:
+ # get_latest_version_of_dag deserializes the whole DAG on every call, so
+ # calling it per task instance makes clearing O(n) full-DAG loads. One
+ # lookup per dag_id also keeps the clear consistent if a new version is
+ # serialized while it runs.
+ latest_dags: dict[str, Any] = {}
Review Comment:
Nit: `get_latest_version_of_dag` returns `SerializedDAG | None` and the name
is already imported under `TYPE_CHECKING`, so the cache can be typed precisely.
```suggestion
latest_dags: dict[str, SerializedDAG | None] = {}
```
##########
airflow-core/src/airflow/models/taskinstance.py:
##########
@@ -403,6 +403,13 @@ def clear_task_instances(
from airflow.models.dagbag import DBDagBag
scheduler_dagbag = DBDagBag(load_op_links=False)
+ # Cache per-dag lookups of the latest serialized DAG and DagVersion:
+ # get_latest_version_of_dag deserializes the whole DAG on every call, so
+ # calling it per task instance makes clearing O(n) full-DAG loads. One
Review Comment:
nit: "Dag" in prose, per the naming rule in the repo's AGENTS instructions.
```suggestion
# Cache per-dag lookups of the latest serialized Dag and DagVersion:
# get_latest_version_of_dag deserializes the whole Dag on every call, so
# calling it per task instance makes clearing O(n) full-Dag loads. One
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
