Vignesh created AIRFLOW-2660:
--------------------------------
Summary: Deadlock error from SQLAlchemy when the parallelism is
increased.
Key: AIRFLOW-2660
URL: https://issues.apache.org/jira/browse/AIRFLOW-2660
Project: Apache Airflow
Issue Type: Bug
Components: db
Affects Versions: Airflow 1.9.0
Environment: RHEL 7
Reporter: Vignesh
Fix For: 1.9.0
Faced the following errors when running a multi worker distributed setup of
airflow 1.9. These error occurred when the number of scheduler threads,
parallelism and dag_concurrency were increased in the configuration.
Original exception was: (_mysql_exceptions.OperationalError) (1213, 'Deadlock
found when trying to get lock; try restarting transaction') [SQL: u'UPDATE
task_instance SET state=%s WHERE task_instance.task_id = %s AND
task_instance.dag_id = %s AND task_instance.execution_date = %s'] [parameters:
(u'queued', 'join', 'parent_dag.sub_dag', datetime.datetime(2018, 6, 18, 7, 1,
44))] (Background on this error at: [http://sqlalche.me/e/e3q8])
sqlalchemy.exc.InvalidRequestError: This Session's transaction has been rolled
back due to a previous exception during flush. To begin a new transaction with
this Session, first issue Session.rollback(). Original exception was:
(_mysql_exceptions.OperationalError) (1213, 'Deadlock found when trying to get
lock; try restarting transaction') [SQL: u'UPDATE task_instance SET state=%s
WHERE task_instance.task_id = %s AND task_instance.dag_id = %s AND
task_instance.execution_date = %s'] [parameters: (u'queued', 'join_0',
'parent_dag.sub_dag', datetime.datetime(2018, 6, 18, 7, 1, 44))] (Background on
this error at: [http://sqlalche.me/e/e3q8])
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
