[
https://issues.apache.org/jira/browse/AIRFLOW-2660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16722777#comment-16722777
]
jack commented on AIRFLOW-2660:
-------------------------------
I suspect that this is a known MySQL issue:
[https://stackoverflow.com/questions/41015813/avoiding-mysql-deadlock-when-upgrading-shared-to-exclusive-lock]
> Deadlock error from SQLAlchemy when the parallelism is increased.
> -----------------------------------------------------------------
>
> Key: AIRFLOW-2660
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2660
> Project: Apache Airflow
> Issue Type: Bug
> Components: db
> Affects Versions: 1.9.0
> Environment: RHEL 7
> Reporter: Vignesh
> Priority: Minor
> Labels: patch
> Fix For: 1.9.0
>
>
> Faced the following errors when running a multi worker distributed setup of
> airflow 1.9. These error occurred when the number of scheduler threads,
> parallelism and dag_concurrency were increased in the configuration.
>
> Original exception was: (_mysql_exceptions.OperationalError) (1213, 'Deadlock
> found when trying to get lock; try restarting transaction') [SQL: u'UPDATE
> task_instance SET state=%s WHERE task_instance.task_id = %s AND
> task_instance.dag_id = %s AND task_instance.execution_date = %s']
> [parameters: (u'queued', 'join', 'parent_dag.sub_dag',
> datetime.datetime(2018, 6, 18, 7, 1, 44))] (Background on this error at:
> [http://sqlalche.me/e/e3q8])
>
> sqlalchemy.exc.InvalidRequestError: This Session's transaction has been
> rolled back due to a previous exception during flush. To begin a new
> transaction with this Session, first issue Session.rollback(). Original
> exception was: (_mysql_exceptions.OperationalError) (1213, 'Deadlock found
> when trying to get lock; try restarting transaction') [SQL: u'UPDATE
> task_instance SET state=%s WHERE task_instance.task_id = %s AND
> task_instance.dag_id = %s AND task_instance.execution_date = %s']
> [parameters: (u'queued', 'join_0', 'parent_dag.sub_dag',
> datetime.datetime(2018, 6, 18, 7, 1, 44))] (Background on this error at:
> [http://sqlalche.me/e/e3q8])
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
