Is there any way to import custom python modules into dag file without mixing dag environs and sys.path? Can't use something like
environ["PROJECT_HOME"] = "/path/to/some/project/files"# import certain project files sys.path.append(environ["PROJECT_HOME"])import mymodule because it the sys.path is shared among all dags and this causes problems <https://stackoverflow.com/q/58423137/8236733> (eg. sharing of values between dag definitions) if want to import modules from different places that have the same name for different dag definitions (and if there are many dags, this is hard to keep track of). The docs for using packaged dags <https://airflow.apache.org/concepts.html?highlight=zip#packaged-dags> (which seemed like a solution) do not seem to avoid the problem the zip file will be inserted at the beginning of module search list (sys.path) and as such it will be available to any other code that resides within the same interpreter. Anyone with more airflow knowledge know how to handle this kind of situation? -- This electronic message is intended only for the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.
