kaxil commented on code in PR #64929:
URL: https://github.com/apache/airflow/pull/64929#discussion_r3067467823
##########
airflow-core/src/airflow/dag_processing/collection.py:
##########
@@ -473,6 +481,11 @@ def update_dag_parsing_results_in_db(
SerializedDAG.bulk_write_to_db(
bundle_name, bundle_version, dags, parse_duration,
session=session
)
+ # Bulk prefetch metadata for all DAGs to avoid per-DAG queries
in write_dag.
Review Comment:
The comment says "3 SELECTs per DAG with 2 queries total," but DAGs with
deadlines still hit a per-DAG `select(cls).where(cls.dag_id ==
dag.dag_id).order_by(cls.created_at.desc()).limit(1)` inside `write_dag` for
deadline UUID reuse. Worth either noting that in the comment, or extending the
prefetch to also return the latest `existing_serialized_dag` when it exists.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
