nuclearpinguin commented on a change in pull request #7674: [AIRFLOW-7022]
Simplify DagFileProcessor.process_file method
URL: https://github.com/apache/airflow/pull/7674#discussion_r390247701
##########
File path: airflow/jobs/scheduler_job.py
##########
@@ -905,6 +898,28 @@ def process_file(
return simple_dags, len(dagbag.import_errors)
+ @provide_session
+ def _prepare_simple_dags(self, dags: List[DAG], pickle_dags: bool,
session) -> List[SimpleDag]:
+ """
+ Convert DAGS to SimpleDags. If necessary, it also Pickle the DAGs
+
+ :param dags: List of DAGs
+ :param pickle_dags: whether serialize the DAGs found in the file and
+ save them to the db
+ :type pickle_dags: bool
+ :return: List of SimpleDag
+ :rtype: List[airflow.utils.dag_processing.SimpleDag]
+ """
+
+ simple_dags = []
+ # Pickle the DAGs (if necessary) and put them into a SimpleDag
+ for dag in dags:
+ pickle_id = None
+ if pickle_dags:
+ pickle_id = dag.pickle(session).id
Review comment:
There's even a chance to use list comprehension but I'm not sure if this
will be more readable. If yes, maybe generator expression?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
