Hi everyone,

I'd like to propose a schema change and ask for lazy consensus (as
suggested in the PR review -
https://github.com/apache/airflow/pull/75673#pullrequestreview-5724819572).

Background
When a Dag file dynamically builds several Dags and some factories fail,
successful Dags still load, but the failures are easy to miss (often only
in processor logs).
Issue https://github.com/apache/airflow/issues/29275 asks for a first-class
way to surface those partial parse failures in the UI/API.

Proposal
PR: https://github.com/apache/airflow/pull/75673

Added a Task SDK helper:

  emit_dag_warning(dag_id, message, *, stacktrace=None)

Call this during the Dag file parse. The Dag processor carries the records
on DagFileParsingResult and stores them in the table: dag_warning. They
show up on GET /api/v2/dagWarnings and a Dashboard parse-failure card.

Schema Changes Summary:

table: dag_warning

1. Dropped Foreign Key (dcw_dag_id_fkey to dag.dag_id)
   Parse-failure rows may use a dag_id that was never registered, so the
Foreign key cannot remain.

2. Added nullable bundle_name and filename so these warnings are
file-scoped.

3. Added a new index: idx_dag_warning_bundle_filename on (bundle_name,
filename) for the per-parse DELETE that replaces warnings for that file.

Please reply with objections or concerns, and if this change is ok to go
ahead with.

Thanks,
Sakshi Yadav

Reply via email to