Hi Sameer, I have a correction to the .airflowignore approach: it does not just filter the results but it prunes the ignored directories out of the *os.walk* itself.
So setting the bundle path to the repository root and ignoring project/lib/ gives you both halves: 1. `from project.lib import some_function` resolves 2. Airflow never walks the ignored trees. Ash's variant works too: `*` in project/lib/.airflowignore, which keeps the ignore file next to the code it covers instead of at the monorepo root. Thanks & Regards, Amogh On Sun, Sep 20, 2026 at 7:48 PM Jarek Potiuk <[email protected]> wrote: > I am not sure this is good idea. Generally "dag" folder where dags are > **should** be root of the Python imports - and they **should** come as a > single commit hash exported for example when you use Git Bundle. > > So I would rather say that If you need to get Dag + It's utility libraries > for example you should checkout Git at the root and refer to your Dags as > "from project.dags.example_dag import Dag" and "from > project.lib.example_library import ExampleUtil" - rather than have several > roots. > > It creates a lot of confusion otherwise when you are "free" to choose where > the "roots" for your Python packages should be - especially when you > include implicit namespaces. > > Unless of course we decide to use standard Python project settings (minus > workspaces) with pyproject.toml and (optionally with uv/hatch workspaces). > For example I would be quite receptive to this: > > ``` > repository/ > ├── project/ > │ ├───── lib1/ > │ │ ├── pyproject.toml > | | └── src/ > | | └─ toplevelutilpackage1/ > | | └─ lib1_util.py > │ ├───── lib2/ > │ │ ├── pyproject.toml > | | ├── src/ > | | | └─ toplevelutilpackage2/ > | | | └─ secondlevelutilpackage/ > | | | └─ > lib2_util.py > | | └── tests/ > | | └─ toplevelutilpackage2/ > | | └─ secondlevelutilpackage/ > | | └─ > tests_lib2_util.py > │ └── dags/ > | ├── pyproject.toml > | ├── src/ > | | └─ topleveldagpackage > | | └─ secondleveldagpackage > | | └─ > example_dag1.py > | └── tests/ > | └─ topleveldagpackage > | └─ secondleveldagpackage > | └─ > test_example_dag1.py > ``` > > Then, the set of ROOTS for Python could be detected and used in a fully > standard way: > > * from toplevelutilpackage1.lib1_util import Util1 > * from toplevelutilpackage2.secondlevelutilpackage.lib2_util import Util2 > * from topleveldagpackage.secondleveldagpackage.example_dag1 import Dag1 > > This would allow all tooling/IDEs to work appropriately, adhere to modern > Python standards, and follow the standard setup for tests etc. > > J. > > > On Sat, Sep 19, 2026 at 10:48 PM Sameer Raj <[email protected]> wrote: > > > Hi Airflow community, > > > > I would like feedback on separating Dag discovery paths from Python > import > > roots. > > I had previously raised a Github discussion: > > https://github.com/apache/airflow/discussions/70313 > > and received comments regarding starting a dev-list thread > > > > A DAG bundle can contain DAG files together with shared Python modules, > > configuration files, and other resources. > > Airflow currently uses BaseDagBundle.path for two purposes: > > 1. Scan for Dag files. > > 2. Add a directory to sys.path. > > > > These paths can differ in a monorepo. For our use case too with > repository > > like: > > ``` > > repository/ > > ├── project/ > > │ ├── lib/ > > │ └── dags/ > > │ └── example_dag.py > > ``` > > The DAG may import shared code using: > > >> from project.lib import some_function > > > > Airflow automatically adds BaseDagBundle.path to sys.path. In this > example, > > it adds only `repository/project/dags`. However, an import such as `from > > project.lib import some_function` requires `repository` to be on > sys.path. > > > > Airflow may need to scan only a Dag subdirectory, while Python imports > > start at the repository root. Using the repository root for discovery can > > make Airflow scan many unrelated files. Using only the Dag directory can > > make valid imports fail. > > > > Thus, the proposal is to decouple import root with bundle path. > > > > For the same, I have raised the PR: > > https://github.com/apache/airflow/pull/73388 > > > > The proposed change adds BaseDagBundle.import_root: > > > > - import_root defaults to path. > > - Existing bundle providers keep their current behavior. > > - GitDagBundle uses the repository root as import_root when path points > to > > a subdirectory. > > - Airflow uses the same import root during Dag parsing, task execution, > and > > callback execution. > > - The import root must belong to the same initialized bundle checkout. > > > > The change is backward compatible for LocalDagBundle, S3DagBundle, > > GCSDagBundle, and custom providers that do not override import_root. > > Comments from the discussion thread have also been addressed in the PR. > > > > I would appreciate feedback on these questions: > > 1. Is Airflow community aligned with this change. > > 2. If yes, is import_root the correct public API for this use case. > > > > and any other feedbacks is appreciated! > > > > Several other users have also requested this: > > https://github.com/apache/airflow/discussions/61901 > > So this can be a value add for Airflow community! > > > > Thanks, > > Sameer > > >
