imbaczek opened a new issue #21947: URL: https://github.com/apache/airflow/issues/21947
### Apache Airflow version 2.0.2 ### What happened ``` 2022-02-26 15:42:51 UTC [104192]: user=prod_airflow,db=prod_airflow,app=[unknown],client=10.80.166.208 STATEMENT: UPDATE dag SET next_dagrun='2022-02-26T15:32:50.428672+00:00'::timestamptz, next_dagrun_create_after='2022-02-26T15:42:50.428672+00:00'::timestamptz WHERE dag.dag_id = 'redacted_1' 2022-02-26 15:43:07 UTC [130186]: user=prod_airflow,db=prod_airflow,app=[unknown],client=10.80.166.208 ERROR: deadlock detected 2022-02-26 15:43:07 UTC [130186]: user=prod_airflow,db=prod_airflow,app=[unknown],client=10.80.166.208 DETAIL: Process 130186 waits for ShareLock on transaction 1728784371; blocked by process 130573. Process 130573 waits for ShareLock on transaction 1728784356; blocked by process 130186. Process 130186: UPDATE dag SET next_dagrun='2022-02-26T15:42:47.604016+00:00'::timestamptz, next_dagrun_create_after='2022-02-26T15:52:47.604016+00:00'::timestamptz WHERE dag.dag_id = 'redacted_1' Process 130573: UPDATE dag SET last_parsed_time='2022-02-26T15:43:03.633626+00:00'::timestamptz, next_dagrun='2022-02-26T15:40:03.633649+00:00'::timestamptz WHERE dag.dag_id = 'redacted_2' 2022-02-26 15:43:07 UTC [130186]: user=prod_airflow,db=prod_airflow,app=[unknown],client=10.80.166.208 HINT: See server log for query details. 2022-02-26 15:43:07 UTC [130186]: user=prod_airflow,db=prod_airflow,app=[unknown],client=10.80.166.208 CONTEXT: while updating tuple (4,2) in relation "dag" 2022-02-26 15:43:07 UTC [130186]: user=prod_airflow,db=prod_airflow,app=[unknown],client=10.80.166.208 STATEMENT: UPDATE dag SET next_dagrun='2022-02-26T15:42:47.604016+00:00'::timestamptz, next_dagrun_create_after='2022-02-26T15:52:47.604016+00:00'::timestamptz WHERE dag.dag_id = 'redacted_1' ``` Likely places which conflict in the code: https://github.com/apache/airflow/blob/d7265791187fb2117dfd090cdb7cce3f8c20866c/airflow/models/dag.py#L2436 https://github.com/apache/airflow/blob/d7265791187fb2117dfd090cdb7cce3f8c20866c/airflow/models/dag.py#L2919 ### What you expected to happen _No response_ ### How to reproduce We have 830 dags and this number keeps growing due to business requirements. No special steps to reproduce are required, it just happens every now and then, as would be expected. Increase fillfactor, perhaps? ### Operating System VERSION="20.04.3 LTS (Focal Fossa)" ### Versions of Apache Airflow Providers apache-airflow-providers-celery==1.0.1 apache-airflow-providers-ftp==1.0.1 apache-airflow-providers-http==1.1.0 apache-airflow-providers-imap==1.0.1 apache-airflow-providers-sqlite==1.0.1 ### Deployment Other Docker-based deployment ### Deployment details ``` $ docker --version Docker version 19.03.5, build 633a0ea838 ``` Postgres: psql (11.5 (Ubuntu 11.5-1.pgdg16.04+1)) ### Anything else Multiple times a day. Took remediation steps to reduce amount of rows on page: ``` prod_airflow=# alter table dag set (fillfactor = 10); ALTER TABLE prod_airflow=# vacuum full dag; VACUUM ``` which reduced frequency to ~1/day. ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
