Agreed on the definitional split, and I should have been clearer about what I meant.
I wasn't suggesting import errors become DAG-scoped. I agree that an import error means Airflow couldn't make sense of the input and the whole file should fail. My point was about what counts as "the file". Today a driver file (Example: example_dag_factory.py for Dag factory as shown in tutorial: https://astronomer.github.io/dag-factory/latest/getting-started/quick-start-airflow-standalone/#step-6-generate-the-dag-from-yaml ) is the parse unit, so the file scope is wrong by construction: the thing that failed is one YAML config, but the unit Airflow keys the error on is the Python bridge that loaded 500 of them. That is the entire reason this looks like it needs DAG scoping. Once a YAML file is itself a DAG file, which is what AIP-85 is for, or atleast it provides hook points for it, the existing file-scoped semantics land in the right place on their own. 3 files get import errors, 497 files parse fine, and no new contract between the processor and a driver is needed. AIP-85 doesn't address error scoping, so you're right that it isn't covered there. But it names this exact workaround as motivation: because the processor only recognizes .py, teams either accept one bridge file as a bottleneck or generate 1-to-1 Python files per config in CI to shard it. Prajwal's setup is the second one. Removing that layer removes most of the reported problem as a side effect, which is why I asked. On DagWarning, I agree it is the right home for "this imported, but you should fix it". It can't carry Prajwal's case though. It is keyed on (dag_id, warning_type) with a foreign key to dag.dag_id, so it needs a DAG that was successfully produced, and a config broken enough to produce no DAG has nothing to attach to. warning_type is also validated against a closed enum today. Worth noting that AIP-85 already proposes relaxing that to a namespaced string plus a context dict for frontend translation, so the two threads do overlap there. The part I would separate out as a real bug, independent of both: _update_import_errors sets is_stale=True on every DagModel sharing the relative_fileloc. The healthy DAGs from that file are written and serialized first, then staled because a sibling failed. Even with one YAML per file that will cause us problems whenever a file legitimately defines several DAGs, or when the failure comes from serialization or a dag_policy rather than the parse itself, which is what Prajwal actually reported. Regards, Kaxil On Wed, 12 Aug 2026 at 12:26, Tzu-ping Chung via dev <[email protected]> wrote: > I don’t think it’s covered by AIP-85. > > However, doesn’t we already have DagWarning for this? Import errors are > source-level errors that implies the dags cannot be parsed at all, and by > definition should fail the entire thing without dag scopes because Airflow > simply can’t make sense of the input. > > A DagWarning, on the other hand, means the dags can be imported, but there > are still things you probably should fix. This already exists for a long > time. > > TP > > > > On Aug 12, 2026, at 15:13, Kaxil Naik <[email protected]> wrote: > > > > Is this part of AIP-85 already? > > > > On Tue, 11 Aug 2026 at 15:00, Sumit Maheshwari <[email protected]> > > wrote: > > > >> Strong +1 on this feature, as almost all of the companies using Airflow > are > >> probably using one or more such DAG generator frameworks, and a problem > in > >> one of the configs causes all the DAGs to go into a stale state. > >> > >> I would like to write a Google Doc proposal or an AIP (whichever the > >>> community prefers) and work on it. > >> > >> > >> Will this change require changes in all the generator frameworks, or > will > >> it be backward compatible? If it can be implemented without forcing > >> existing DAG generator frameworks, then I think we should not require an > >> AIP. > >> > >> On Mon, Aug 10, 2026 at 6:32 PM Prajwal Agarwal < > [email protected]> > >> wrote: > >> > >>> Hi Airflow Community, > >>> *TL;DR:* There is no way to surface import errors alongside healthy > dags > >>> from a driver (like dag- <https://github.com/astronomer/dag-factory > >>>> factory > >>> <https://github.com/astronomer/dag-factory>), if some configs passed > >> into > >>> the loader file are wrong. It is either “all” or “none”. If the driver > >> file > >>> raises an error, it can deactivate all healthy DAGs belonging to that > >>> driver. Driver owners handle this by gracefully managing the exceptions > >> in > >>> their code. > >>> I would like to discuss potential solutions (or learn if one already > >> exists > >>> or is planned). > >>> > >>> *Details* > >>> When a single Python file generates many DAGs — as dag-factory and > >> similar > >>> "driver file" patterns do — Airflow's import-error handling is > >> file-scoped, > >>> not DAG-scoped. One bad config in a factory that builds hundreds of > DAGs > >>> currently forces an all-or-nothing outcome. > >>> If the factory lets the exception propagate, the whole file fails to > >> import > >>> and every DAG it would have produced disappears (has_import_error=True, > >>> is_stale=True in dag-processor). > >>> If the factory swallows the bad config to keep the others alive, the > >>> failure is silent—no import error is surfaced to the end user, on the > >>> Airflow UI. > >>> There is no way to say, "These 3 configurations are broken, here is > why, > >>> but the other 497 DAGs are healthy and should keep running." > >>> I'd like to discuss a contract that makes that possible. > >>> > >>> *Current Behavior:* > >>> Import errors are keyed by file, one string per file. > >>> Staleness is applied to every DAG sharing the file's fileloc. > >>> > >>> *Proposed Solution (Details are intentionally missing)* > >>> A contract between dag-processor and driver file to return healthy dags > >> as > >>> well as import errors (per file or per dag, given it is not necessary > >> that > >>> dag_id is available) > >>> > >>> *Motivation* > >>> We use Airflow to power 100s of dags from a single driver (dag-factory > in > >>> our case) where our customers write YAML files. We do have some build > >> time > >>> validations, but serialization errors can happen due to dag_policies > >>> execution and failure as well. We would like to provide better > >>> observability over these failures to our end users. > >>> > >>> *Ask* > >>> I would like to know the community's opinion on supporting such a > >> feature. > >>> If there is a use case, I would like to write a Google Doc proposal or > an > >>> AIP (whichever the community prefers) and work on it. > >>> > >>> Related Discussion thread on Github: > >>> https://github.com/apache/airflow/discussions/70119 > >>> > >>> Looking forward to get some thoughts on this. > >>> > >>> Thank you & Regards > >>> Prajwal > >>> > >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
