Hi Everyone, we are facing issues, when trying to smoothly work collaboratively on the same DAGs in the same (development) environment on Airflow.
As of now, we have a common code-base for our project, where multiple users in various teams are making changes at the same time. This leads to problems and conflicts when regularly pushing changes to airflow. So we would like to be able to work in parallel and in isolation. I thought I was smart and tried to enable it by: • Enabling relative imports • Create a user-specific ansible deployment that creates a new DAG folder per user and adds a dag-id prefix per user While the ansible stuff works nicely, I didn’t manage to convince Airflow to use relative imports. Also failed with https://airflow.apache.org/concepts.html#packaged-dags approach, so far… but it might as well be due my limited understanding of how Airflow imports DAGs. The most promising folder structure so far looked like this: {airflow-dag-root}/my_project/ + __init__.py + my_dag.py + utils/ + __init__.py + util.py When trying to import like this in my_dag.py: from .utils.util import my_helper ... I get this error: "attempted relative import with no known parent package" Do you have any suggestions on how to achieve relative imports or another nice way of working in isolation in airflow? Thanks! Best, Andreas
