Chris7 opened a new issue #21616: URL: https://github.com/apache/airflow/issues/21616
### Apache Airflow version 2.2.3 (latest released) ### What happened I'm currently using airflow triggers, but I've noticed that deferred tasks are being ran on every triggerer instance right away: ``` host 1 -> 2/10/2022 1:06:08 PM Job a702656f-01ce-4e7a-893a-5b42cdaa38e2 progressed from Unknown to RUNNABLE. host 2 -> 2/10/2022 1:06:06 PM Job a702656f-01ce-4e7a-893a-5b42cdaa38e2 progressed from Unknown to RUNNABLE. ``` within 2 seconds, the job was issued on both triggerer instances. ### What you expected to happen The deferred task is only scheduled on 1 triggerer instance. ### How to reproduce Create many jobs that have to be deferred, and start multiple triggerers. ### Operating System Ubuntu 20.04.3 LTS ### Versions of Apache Airflow Providers 2.2.3 ### Deployment Other Docker-based deployment ### Deployment details A docker setup w/ multiple triggerer instances running. ### Anything else I believe this issue is due to a race condition here: https://github.com/apache/airflow/blob/84a59a6d74510aff4059a3ca2da793d996e86fa1/airflow/models/trigger.py#L175. If multiple instances start at the same time, each instance will get the same tasks in their alive_job_ids. ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
