pierrejeambrun commented on code in PR #74043:
URL: https://github.com/apache/airflow/pull/74043#discussion_r4182967452
##########
airflow-core/src/airflow/dag_processing/lang_sdk_processor.py:
##########
@@ -226,6 +229,53 @@ def start( # type: ignore[override]
)
return proc
+ @classmethod
+ def run(
+ cls,
+ *,
+ path: str | os.PathLike[str],
+ bundle_path: Path,
+ bundle_name: str,
+ dag_file_rel_path: str,
+ logger: FilteringBoundLogger,
+ ) -> DagFileParsingResult:
+ """
+ Parse *path* outside the Dag processor and wait for the result.
+
+ There is no API client, so each request of the runtime that needs one
gets an error. The file's import
+ timeout bounds the parse, and ``[dag_processor]
dag_file_processor_timeout`` until the parse child
+ reports it.
+ """
+ processor_timeout = conf.getfloat("dag_processor",
"dag_file_processor_timeout")
+ with selectors.DefaultSelector() as selector:
+ proc = cls.start(
+ id=uuid7(),
+ path=path,
+ bundle_path=bundle_path,
+ bundle_name=bundle_name,
+ dag_file_rel_path=dag_file_rel_path,
+ selector=selector,
+ logger=logger,
+ )
+ try:
+ while not proc.is_ready:
Review Comment:
Same problem with `is_ready` this can crash the whole thing I believe.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
