jason810496 commented on code in PR #50371:
URL: https://github.com/apache/airflow/pull/50371#discussion_r2082772825
##########
airflow-core/src/airflow/dag_processing/processor.py:
##########
@@ -94,6 +94,18 @@ def _parse_file_entrypoint():
def _parse_file(msg: DagFileParseRequest, log: FilteringBoundLogger) ->
DagFileParsingResult | None:
# TODO: Set known_pool names on DagBag!
+ # Pre-import modules
+ if conf.getboolean("scheduler", "parsing_pre_import_modules",
fallback=True):
+ import importlib
+
+ from airflow.utils.file import iter_airflow_imports
+
+ for module in iter_airflow_imports(msg.file):
Review Comment:
I just double checked the `for module in ..` part with
https://github.com/apache/airflow/pull/30495/files#diff-3101df8e7c54fe1b9305774befb54b31edfe84cb81425a300e70142bc07637d4R192-R208
Yes, we should add one more indentation.
##########
airflow-core/src/airflow/dag_processing/processor.py:
##########
@@ -94,6 +94,18 @@ def _parse_file_entrypoint():
def _parse_file(msg: DagFileParseRequest, log: FilteringBoundLogger) ->
DagFileParsingResult | None:
# TODO: Set known_pool names on DagBag!
+ # Pre-import modules
+ if conf.getboolean("scheduler", "parsing_pre_import_modules",
fallback=True):
+ import importlib
+
+ from airflow.utils.file import iter_airflow_imports
Review Comment:
nit: I think we can move the import to top level?
Also, the comments in #30495 are useful, we can port the comments back as
well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
