dstandish commented on code in PR #26509:
URL: https://github.com/apache/airflow/pull/26509#discussion_r978437183
##########
airflow/models/dag.py:
##########
@@ -1189,11 +1195,25 @@ def relative_fileloc(self) -> pathlib.Path:
"""File location of the importable dag 'file' relative to the
configured DAGs folder."""
path = pathlib.Path(self.fileloc)
try:
- return path.relative_to(settings.DAGS_FOLDER)
+ rel_path = path.relative_to(self.dag_processor_dags_folder or
settings.DAGS_FOLDER)
+ if rel_path == pathlib.Path('.'):
+ return path
+ else:
+ return rel_path
except ValueError:
# Not relative to DAGS_FOLDER.
return path
+ @property
+ def dag_processor_dags_folder(self):
Review Comment:
> I see that name may be a little confusing, but --subdir just points to the
dag folder - the value can be absolute path and two different processors may
point ot complete different locations.
what do you mean "just points to the dag folder"? i have looked at the code
and it does not seem that there's any logic e.g. to make the subdir agree with
configured dags folder. e.g. setting subdir on dag processor command does not
force dags_folder to agree. so fundamentally, they are two different things
right? the dag processor can run without them agreeing, no?
if they are in completely different locations, i.e. not a "subdir" of the
dags folder, then the concept of "relative location" of a dag file becomes kind
of meaningless, and if you can't use relative loc, then the worker can't really
rely on ser dag to know where a dag is located.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
