honarkhah edited a comment on issue #8521: URL: https://github.com/apache/airflow/issues/8521#issuecomment-674938047
In my case issue is about concurrent dags, the insertion issue is fixed in airflow `1.10.11` [fixed PR](https://github.com/apache/airflow/pull/8775) Steps to reproduce: - Trigger more than 1 instance of a dag - Have more than 1 worker running at the same time If X tasks of a dag ran at the same time, (X - 1) of them will face lock and just 1 of them return success! ``` [2020-08-17 10:42:30,177] {taskinstance.py:882} INFO - -------------------------------------------------------------------------------- [2020-08-17 10:42:30,187] {taskinstance.py:901} INFO - Executing <Task(BranchPythonOperator): branch_task> on 2020-08-17T10:34:24.115521+00:00 [2020-08-17 10:42:30,190] {standard_task_runner.py:54} INFO - Started process 553 to run task [2020-08-17 10:42:30,212] {standard_task_runner.py:77} INFO - Running: ['airflow', 'run', 'logo_detection_webhook', 'branch_task', '2020-08-17T10:34:24.115521+00:00', '--job_id', '169070', '--pool', 'default_pool', '--raw', '-sd', 'DAGS_FOLDER/model_training/logo_detection_webhook.py', '--cfg_path', '/tmp/tmprvq51le4'] [2020-08-17 10:42:30,213] {standard_task_runner.py:78} INFO - Job 169070: Subtask branch_task [2020-08-17 10:42:30,251] {logging_mixin.py:112} INFO - Running %s on host %s <TaskInstance: logo_detection_webhook.branch_task 2020-08-17T10:34:24.115521+00:00 [running]> 91a09a3997d3 [2020-08-17 10:43:21,389] {taskinstance.py:1150} ERROR - (_mysql_exceptions.OperationalError) (1205, 'Lock wait timeout exceeded; try restarting transaction') [SQL: DELETE FROM rendered_task_instance_fields WHERE rendered_task_instance_fields.dag_id = %s AND rendered_task_instance_fields.task_id = %s AND (rendered_task_instance_fields.dag_id, rendered_task_instance_fields.task_id, rendered_task_instance_fields.execution_date) NOT IN (SELECT subq1.dag_id, subq1.task_id, subq1.execution_date FROM (SELECT rendered_task_instance_fields.dag_id AS dag_id, rendered_task_instance_fields.task_id AS task_id, rendered_task_instance_fields.execution_date AS execution_date FROM rendered_task_instance_fields WHERE rendered_task_instance_fields.dag_id = %s AND rendered_task_instance_fields.task_id = %s ORDER BY rendered_task_instance_fields.execution_date DESC LIMIT %s) AS subq1)] [parameters: ('logo_detection_webhook', 'branch_task', 'logo_detection_webhook', 'branch_task', 30)] (Background on this error at: http://sqlalche.me/e/13/e3q8) Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1278, in _execute_context cursor, statement, parameters, context File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 593, in do_execute cursor.execute(statement, parameters) File "/usr/local/lib/python3.7/site-packages/MySQLdb/cursors.py", line 255, in execute self.errorhandler(self, exc, value) File "/usr/local/lib/python3.7/site-packages/MySQLdb/connections.py", line 50, in defaulterrorhandler raise errorvalue File "/usr/local/lib/python3.7/site-packages/MySQLdb/cursors.py", line 252, in execute res = self._query(query) File "/usr/local/lib/python3.7/site-packages/MySQLdb/cursors.py", line 378, in _query db.query(q) File "/usr/local/lib/python3.7/site-packages/MySQLdb/connections.py", line 280, in query _mysql.connection.query(self, query) _mysql_exceptions.OperationalError: (1205, 'Lock wait timeout exceeded; try restarting transaction') ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
