pierrejeambrun commented on code in PR #69833:
URL: https://github.com/apache/airflow/pull/69833#discussion_r3614405109
##########
airflow-core/src/airflow/api_fastapi/core_api/services/ui/dependencies.py:
##########
@@ -163,6 +333,38 @@ def get_scheduling_dependencies(readable_dag_ids: set[str]
| None = None) -> dic
}
+def _get_dag_entry_point(dag_id: str, session: Session) -> tuple[str, dict |
None] | None:
+ """Get the id of a Dag's topologically-first task/group and its
asset_expression, if any."""
+ from sqlalchemy import select
+ from sqlalchemy.orm import joinedload
+
+ from airflow.api_fastapi.core_api.services.ui.task_group import
task_group_to_dict
+ from airflow.models.dag_version import DagVersion
+ from airflow.models.serialized_dag import SerializedDagModel
+
+ dag_version = DagVersion.get_latest_version(dag_id, session=session)
+ if dag_version is None:
+ return None
+
+ serialized_dag = session.scalar(
+ select(SerializedDagModel)
+ .join(DagVersion)
+ .where(
+ SerializedDagModel.dag_id == dag_id,
+ DagVersion.version_number == dag_version.version_number,
+ )
+ .options(joinedload(SerializedDagModel.dag_model))
+ )
+ if serialized_dag is None:
+ return None
+
+ entry_points = serialized_dag.dag.task_group.topological_sort()
Review Comment:
This deserializes the full DAG for every scheduled Dag in the BFS just to
read its first task — N full-DAG loads for an asset feeding many Dags. Is it
possible to do batching here for the serialized-dag fetch or deriving the entry
task without full deserialization.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
