Why can't you just add the root to your configuration? That seems like we are trying to implement an Airflow solution to handle the repository's non-Python standard layout. Otherwise, why don't you have pyproject.toml at the root and use the python-standard way?
In pyproject.toml in the "repository" folder in your repo would be both standard and support for it could be implemented in Airflow. I would very much prefer the solution where what is submitted to Airflow auto-detects the roots - and you just configure your repo to use "standards". And I thin it's a very strange argument with .airflowignore. Yes it needs to talk to other teams, and i understand it might be a bit "red-tap-y" in some organisations, but you repo is **ALREADY** airflow specific - it has a "dag" folder. J. On Mon, Sep 21, 2026 at 7:09 PM Sameer Raj <[email protected]> wrote: > Hi all, > Thank you for the feedback on this proposal. > > Regarding some of the feedback: > > > You should checkout Git at the root and refer to your Dags as from > project.dags... and utilities as from project.lib.... > > This should not introduce several roots. > > Being free to choose roots can create confusion, especially with implicit > namespaces. > > I have revised the PR: https://github.com/apache/airflow/pull/73388. > Now, I am only adding the specified import root in the sys.path > > bundle.path = <repo>/team/dags > bundle.import_root = <repo> > -> sys.path += bundle.path only > > We scan bundle.discovery_path only for DAGs > This way, we only enforce one root as per the feedback.(Infact, this is the > same pattern being followed in monorepo.) > > > Generally, the "dag" folder where dags are should be the root of the > Python imports, and they should come as a single commit hash. > > This is ensured as part of PR now too. In GitDagBundle, both the discovery > path and import root are resolved within the same checkout for the selected > commit. > > > Standard Python projects using pyproject.toml and uv or Hatch workspaces > would provide a standard package structure. > I agree that standard Python packaging is the preferred model when a > repository can adopt it. The proposal is not intended to replace that model > or detect multiple Python projects. It provides an explicit option for > existing monorepos that currently use repository-rooted imports. > > Regarding Amogh's feedback on using .airflowignore > > I agree that it technically provides both efficient discovery and > repository-rooted imports. However, adding .airflowingore at repository > root is not a feasible solution in monorepos (or shared repositories) since > It would introduce Airflow specific construct for many teams. Also, > regarding adding explicit .airflowignore in `lib`, I understand the > structure of the repository. We have 90+ directories from where DAG authors > can import dependencies, and thus adding .airflowignore isn't a feasible > option as that is owned by different teams. > > The aim of this functionality proposed is to decouple bundle_path with > module import, and user wont be needing to maintain airflowignore. It makes > the discovery path and single Python import root explicit without requiring > changes to repository. > > Thanks, > Sameer Raj > > > On Mon, Sep 21, 2026 at 4:27 PM Amogh Desai <[email protected]> wrote: > > > Hi Sameer, > > > > I have a correction to the .airflowignore approach: it does not just > filter > > the results but it prunes the ignored > > directories out of the *os.walk* itself. > > > > So setting the bundle path to the repository root and ignoring > project/lib/ > > gives you both halves: > > 1. `from project.lib import some_function` resolves > > 2. Airflow never walks the ignored trees. > > > > Ash's variant works too: `*` in project/lib/.airflowignore, which keeps > the > > ignore file next to the code it covers > > instead of at the monorepo root. > > > > Thanks & Regards, > > Amogh > > > > On Sun, Sep 20, 2026 at 7:48 PM Jarek Potiuk <[email protected]> wrote: > > > > > I am not sure this is good idea. Generally "dag" folder where dags are > > > **should** be root of the Python imports - and they **should** come as > a > > > single commit hash exported for example when you use Git Bundle. > > > > > > So I would rather say that If you need to get Dag + It's utility > > libraries > > > for example you should checkout Git at the root and refer to your Dags > as > > > "from project.dags.example_dag import Dag" and "from > > > project.lib.example_library import ExampleUtil" - rather than have > > several > > > roots. > > > > > > It creates a lot of confusion otherwise when you are "free" to choose > > where > > > the "roots" for your Python packages should be - especially when you > > > include implicit namespaces. > > > > > > Unless of course we decide to use standard Python project settings > (minus > > > workspaces) with pyproject.toml and (optionally with uv/hatch > > workspaces). > > > For example I would be quite receptive to this: > > > > > > ``` > > > repository/ > > > ├── project/ > > > │ ├───── lib1/ > > > │ │ ├── pyproject.toml > > > | | └── src/ > > > | | └─ toplevelutilpackage1/ > > > | | └─ lib1_util.py > > > │ ├───── lib2/ > > > │ │ ├── pyproject.toml > > > | | ├── src/ > > > | | | └─ toplevelutilpackage2/ > > > | | | └─ > secondlevelutilpackage/ > > > | | | > └─ > > > lib2_util.py > > > | | └── tests/ > > > | | └─ toplevelutilpackage2/ > > > | | └─ > secondlevelutilpackage/ > > > | | > > └─ > > > tests_lib2_util.py > > > │ └── dags/ > > > | ├── pyproject.toml > > > | ├── src/ > > > | | └─ topleveldagpackage > > > | | └─ secondleveldagpackage > > > | | └─ > > > example_dag1.py > > > | └── tests/ > > > | └─ topleveldagpackage > > > | └─ secondleveldagpackage > > > | └─ > > > test_example_dag1.py > > > ``` > > > > > > Then, the set of ROOTS for Python could be detected and used in a fully > > > standard way: > > > > > > * from toplevelutilpackage1.lib1_util import Util1 > > > * from toplevelutilpackage2.secondlevelutilpackage.lib2_util import > Util2 > > > * from topleveldagpackage.secondleveldagpackage.example_dag1 import > Dag1 > > > > > > This would allow all tooling/IDEs to work appropriately, adhere to > modern > > > Python standards, and follow the standard setup for tests etc. > > > > > > J. > > > > > > > > > On Sat, Sep 19, 2026 at 10:48 PM Sameer Raj <[email protected]> > wrote: > > > > > > > Hi Airflow community, > > > > > > > > I would like feedback on separating Dag discovery paths from Python > > > import > > > > roots. > > > > I had previously raised a Github discussion: > > > > https://github.com/apache/airflow/discussions/70313 > > > > and received comments regarding starting a dev-list thread > > > > > > > > A DAG bundle can contain DAG files together with shared Python > modules, > > > > configuration files, and other resources. > > > > Airflow currently uses BaseDagBundle.path for two purposes: > > > > 1. Scan for Dag files. > > > > 2. Add a directory to sys.path. > > > > > > > > These paths can differ in a monorepo. For our use case too with > > > repository > > > > like: > > > > ``` > > > > repository/ > > > > ├── project/ > > > > │ ├── lib/ > > > > │ └── dags/ > > > > │ └── example_dag.py > > > > ``` > > > > The DAG may import shared code using: > > > > >> from project.lib import some_function > > > > > > > > Airflow automatically adds BaseDagBundle.path to sys.path. In this > > > example, > > > > it adds only `repository/project/dags`. However, an import such as > > `from > > > > project.lib import some_function` requires `repository` to be on > > > sys.path. > > > > > > > > Airflow may need to scan only a Dag subdirectory, while Python > imports > > > > start at the repository root. Using the repository root for discovery > > can > > > > make Airflow scan many unrelated files. Using only the Dag directory > > can > > > > make valid imports fail. > > > > > > > > Thus, the proposal is to decouple import root with bundle path. > > > > > > > > For the same, I have raised the PR: > > > > https://github.com/apache/airflow/pull/73388 > > > > > > > > The proposed change adds BaseDagBundle.import_root: > > > > > > > > - import_root defaults to path. > > > > - Existing bundle providers keep their current behavior. > > > > - GitDagBundle uses the repository root as import_root when path > points > > > to > > > > a subdirectory. > > > > - Airflow uses the same import root during Dag parsing, task > execution, > > > and > > > > callback execution. > > > > - The import root must belong to the same initialized bundle > checkout. > > > > > > > > The change is backward compatible for LocalDagBundle, S3DagBundle, > > > > GCSDagBundle, and custom providers that do not override import_root. > > > > Comments from the discussion thread have also been addressed in the > PR. > > > > > > > > I would appreciate feedback on these questions: > > > > 1. Is Airflow community aligned with this change. > > > > 2. If yes, is import_root the correct public API for this use case. > > > > > > > > and any other feedbacks is appreciated! > > > > > > > > Several other users have also requested this: > > > > https://github.com/apache/airflow/discussions/61901 > > > > So this can be a value add for Airflow community! > > > > > > > > Thanks, > > > > Sameer > > > > > > > > > >
