uranusjr commented on code in PR #73118:
URL: https://github.com/apache/airflow/pull/73118#discussion_r4013531306
##########
task-sdk/src/airflow/sdk/importers/python_importer.py:
##########
@@ -78,41 +135,43 @@ def can_handle(self, definition: DagDefinition | str |
Path) -> bool:
def list_dag_definitions(
self,
bundle: BaseDagBundle,
- *,
- safe_mode: bool = True,
- ) -> Iterator[DagDefinition]:
- """List Python DAG definitions in a bundle matching supported
extensions."""
- yield from find_file_dag_definitions(bundle.path,
self.supported_extensions, safe_mode=safe_mode)
+ ) -> Iterator[FileDagDefinition | ZipFileDagDefinition]:
+ """
+ List Python DAG files in a bundle matching supported extensions.
+
+ This does not look for members inside zip archives. Definition
discovery
+ of zip archives is done inside :class:`..zip_importer.ZipImporter`
+ instead.
+ """
+ if bundle.path.is_dir():
+ yield from find_file_dag_definitions(bundle.path,
self.supported_extensions)
Review Comment:
I reworked the DagDefinition types to like this:
- One base FileDagDefinition referenced by all dag importers that work on
file-like objects.
- FilesystemDagDefinition to represent a “real” file.
- ZipMemberDagDefinition for a member in a zip archive.
PythonDagImporter only interfaces with FileDagDefinition; other importers
that will be used as ZipDagImporter’s internal importers will too. Individual
importers can provide zip-specific optimization if it wants to, but it’s not
required for each importer to know it’s working on zip.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
