I dont see how the scheduler would know. DAGs can come from any location, any module, and import is dependent on the python interpreter.
If you know a way the suits every kind of structure please provide a pr. Cheers Bolke Verstuurd vanaf mijn iPad > Op 5 dec. 2017 om 19:05 heeft Alek Storm <[email protected]> het volgende > geschreven: > > We solved this (hackily) by setting the fileloc field on the generated DAG > object to inspect.getsourcefile(inspect.stack()[1][0]), as the DAG > constructor itself does. I agree a more general solution is needed; > presumably the scheduler knows which Python file in the dags folder it was > processing when it found the DAG object. > > Alek > > >> On Tue, Dec 5, 2017 at 12:00 PM, Michael Erdely <[email protected]> wrote: >> >> Hi, >> >> In order to support multiple environments with different DAG settings per >> environment, we created a DAG factory to create the DAG operator where each >> DAG has different params (eg schedule, catchup, etc). >> >> Unfortunately, we noticed that the webserver code view shows the factory >> code versus the actual Python code in the dagbag file. I also assume that >> the factory file modified date will be used versus the DAG file to >> determine if the scheduler should reload. >> >> I see this issue has already been reported here >> https://issues.apache.org/jira/browse/AIRFLOW-1108. >> >> Any ideas if this will be patched soon or have better suggestions on >> handling this? >> >> -Michael >>
