sangreddy32 commented on PR #14031:
URL: https://github.com/apache/airflow/pull/14031#issuecomment-1179187360
We have a similar issue in the Airflow '2.3.1' and MySQL 'Ver 14.14 Distrib
5.7.37'. Our scheduler shuts down whenever we get Deadlocks as below. Any
solution or temporary fix greatly appreciated.
rowcount = cursor.executemany(statement, parameters)
File
"/efs/airflow-venv/cdw-prod/airflow/lib/python3.9/site-packages/MySQLdb/cursors.py",
line 239, in executemany
self.rowcount = sum(self.execute(query, arg) for arg in args)
File
"/efs/airflow-venv/cdw-prod/airflow/lib/python3.9/site-packages/MySQLdb/cursors.py",
line 239, in <genexpr>
self.rowcount = sum(self.execute(query, arg) for arg in args)
File
"/efs/airflow-venv/cdw-prod/airflow/lib/python3.9/site-packages/MySQLdb/cursors.py",
line 206, in execute
res = self._query(query)
File
"/efs/airflow-venv/cdw-prod/airflow/lib/python3.9/site-packages/MySQLdb/cursors.py",
line 319, in _query
db.query(q)
File
"/efs/airflow-venv/cdw-prod/airflow/lib/python3.9/site-packages/MySQLdb/connections.py",
line 254, in query
_mysql.connection.query(self, query)
sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError)
(1213, 'Deadlock found when trying to get lock; try restarting transaction')
[SQL: UPDATE dag_run SET last_scheduling_decision=%s WHERE dag_run.id = %s]
[parameters: ((datetime.datetime(2022, 7, 8, 11, 0, 23, 20492), 4611),
(datetime.datetime(2022, 7, 8, 11, 0, 23, 92501), 4612),
(datetime.datetime(2022, 7, 8, 11, 0, 23, 180288), 4613),
(datetime.datetime(2022, 7, 8, 11, 0, 23, 261947), 4616),
(datetime.datetime(2022, 7, 8, 11, 0, 23, 360666), 4619),
(datetime.datetime(2022, 7, 8, 11, 0, 23, 484052), 4620),
(datetime.datetime(2022, 7, 8, 11, 0, 23, 605315), 4621),
(datetime.datetime(2022, 7, 8, 11, 0, 23, 696649), 4622) ... displaying 10 of
17 total bound parameter sets ... (datetime.datetime(2022, 7, 8, 11, 0, 22,
901407), 4632), (datetime.datetime(2022, 7, 8, 11, 0, 22, 964056), 4633))]
(Background on this error at: http://sqlalche.me/e/14/e3q8)
[^[[0m] {^[[34mlocal_executor.py:^[[0m390} INFO^[[0m - Shutting down
LocalExecutor; waiting for running tasks to finish. Signal again if you don't
want to wait.^[[0m
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
