GitHub user VectorPeak added a comment to the discussion: Event-driven scheduling: duplicate DAG runs when an `AssetWatcher` runs on multiple triggerers
This looks like #65792 more than a new `AssetWatcher` dedupe-key design gap. The short version: on 3.2.1, asset triggers were selected differently from task-instance triggers. The task-trigger path filtered to triggers that were unassigned or assigned to a dead triggerer, but the asset-trigger branch just selected asset triggers, so multiple live triggerers could pick up the same watcher. That matches your “N triggerers => N events/runs” symptom pretty closely. In 3.2.2, the asset-trigger query includes the same `triggerer_id is null or not in alive_triggerer_ids` filter. ([raw.githubusercontent.com](https://raw.githubusercontent.com/apache/airflow/3.2.1/airflow-core/src/airflow/models/trigger.py)) That behavior should not be considered the intended steady-state semantics. Airflow’s trigger HA model is meant to have a trigger row owned by one live triggerer, with reassignment on failover; duplicate execution can still happen around failures, but “every live triggerer runs the same asset watcher” is the bug #65792 fixed. The PR description calls out exactly that multiple triggerer replicas could pick up the same asset trigger, and it was marked for the Airflow 3.2.2 milestone/backport. ([github.com](https://github.com/apache/airflow/pull/65792)) The practical next step is to test this on 3.2.2 or newer. If the same Git HEAD watcher still produces one event per triggerer there, that would be worth opening as a focused bug with the watcher definition, triggerer count, DB backend, and a snippet of triggerer logs showing the same trigger ID running on multiple triggerers. A dedupe key on `AssetWatcher` still looks like a reasonable feature discussion, but it is a separate layer. It would help with “same external change observed more than once” semantics, especially for polling sources or broker redelivery, but it should not be required just to prevent all healthy triggerers from running the same persisted watcher. For today, consumer idempotency is still the right defensive pattern. Even with the ownership fix, event-driven systems generally need to tolerate retries, triggerer restarts, and external-source redelivery. The difference is that after #65792, idempotency should protect against occasional duplicate delivery, not deterministic duplication proportional to triggerer count. If upgrading is blocked, the safest mitigation is to run only one triggerer capable of handling event-driven asset watchers, accepting the HA tradeoff, or put a compare-and-set checkpoint in the watcher/first task keyed by the Git SHA. That is not as nice as a first-class dedupe key, but it avoids paying for the expensive downstream work twice. --- If my answer solved your problem, you can click **answered the question**. I'm really here to help, and along the way I'm also collecting Galaxy Brain badges haha 😆 GitHub link: https://github.com/apache/airflow/discussions/69319#discussioncomment-17539518 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
