tnyz opened a new issue, #26847: URL: https://github.com/apache/airflow/issues/26847
### Apache Airflow version Other Airflow 2 version ### What happened We noticed that tasks using the dynamic task mapping api crashes the scheduler with the error ``` psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "task_instance_pkey" 80 DETAIL: Key (dag_id, task_id, run_id, map_index)=(ingestion-scheduler, check_for_success_flag, manual__2022-10-03T15:56:03.241451+00:00, 0) already exists. ``` We have 2 scheduler so my guess is that both try to schedule the dynamically mapped task ### What you think should happen instead it should be not crash the scheduler ### How to reproduce run a large dynamically mapped task in multi-scheduler environment ### Operating System linux ### Versions of Apache Airflow Providers _No response_ ### Deployment Other Docker-based deployment ### Deployment details _No response_ ### Anything else _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
