Strong +1 on this feature, as almost all of the companies using Airflow are probably using one or more such DAG generator frameworks, and a problem in one of the configs causes all the DAGs to go into a stale state.
I would like to write a Google Doc proposal or an AIP (whichever the > community prefers) and work on it. Will this change require changes in all the generator frameworks, or will it be backward compatible? If it can be implemented without forcing existing DAG generator frameworks, then I think we should not require an AIP. On Mon, Aug 10, 2026 at 6:32 PM Prajwal Agarwal <[email protected]> wrote: > Hi Airflow Community, > *TL;DR:* There is no way to surface import errors alongside healthy dags > from a driver (like dag- <https://github.com/astronomer/dag-factory > >factory > <https://github.com/astronomer/dag-factory>), if some configs passed into > the loader file are wrong. It is either “all” or “none”. If the driver file > raises an error, it can deactivate all healthy DAGs belonging to that > driver. Driver owners handle this by gracefully managing the exceptions in > their code. > I would like to discuss potential solutions (or learn if one already exists > or is planned). > > *Details* > When a single Python file generates many DAGs — as dag-factory and similar > "driver file" patterns do — Airflow's import-error handling is file-scoped, > not DAG-scoped. One bad config in a factory that builds hundreds of DAGs > currently forces an all-or-nothing outcome. > If the factory lets the exception propagate, the whole file fails to import > and every DAG it would have produced disappears (has_import_error=True, > is_stale=True in dag-processor). > If the factory swallows the bad config to keep the others alive, the > failure is silent—no import error is surfaced to the end user, on the > Airflow UI. > There is no way to say, "These 3 configurations are broken, here is why, > but the other 497 DAGs are healthy and should keep running." > I'd like to discuss a contract that makes that possible. > > *Current Behavior:* > Import errors are keyed by file, one string per file. > Staleness is applied to every DAG sharing the file's fileloc. > > *Proposed Solution (Details are intentionally missing)* > A contract between dag-processor and driver file to return healthy dags as > well as import errors (per file or per dag, given it is not necessary that > dag_id is available) > > *Motivation* > We use Airflow to power 100s of dags from a single driver (dag-factory in > our case) where our customers write YAML files. We do have some build time > validations, but serialization errors can happen due to dag_policies > execution and failure as well. We would like to provide better > observability over these failures to our end users. > > *Ask* > I would like to know the community's opinion on supporting such a feature. > If there is a use case, I would like to write a Google Doc proposal or an > AIP (whichever the community prefers) and work on it. > > Related Discussion thread on Github: > https://github.com/apache/airflow/discussions/70119 > > Looking forward to get some thoughts on this. > > Thank you & Regards > Prajwal >
