ephraimbuddy commented on issue #34012:
URL: https://github.com/apache/airflow/issues/34012#issuecomment-1719221982

   What I see is deadlock:
   ```log
   [2023-09-14T10:41:55.788+0000] {backfill_job_runner.py:686} WARNING - 
Deadlock discovered for ti_status.to_run=dict_values([<TaskInstance: 
stale_data_test.insert_new_daily_data.insert 
backfill__2023-07-30T06:00:00+00:00 map_index=0 [scheduled]>, <TaskInstance: 
stale_data_test.insert_new_daily_data.insert 
backfill__2023-07-22T06:00:00+00:00 map_index=0 [scheduled]>, <TaskInstance: 
stale_data_test.insert_new_daily_data.insert 
backfill__2023-07-19T06:00:00+00:00 map_index=0 [scheduled]>])
   [2023-09-14T10:41:55.874+0000] {backfill_job_runner.py:416} INFO - [backfill 
progress] | finished run 0 of 31 | tasks waiting: 0 | succeeded: 16 | running: 
0 | failed: 0 | skipped: 0 | deadlocked: 3 | not ready: 3
   [2023-09-14T10:41:55.889+0000] {local_executor.py:402} INFO - Shutting down 
LocalExecutor; waiting for running tasks to finish.  Signal again if you don't 
want to wait.
   Traceback (most recent call last):
     File "/usr/local/bin/airflow", line 33, in <module>
       sys.exit(load_entry_point('apache-airflow', 'console_scripts', 
'airflow')())
     File "/opt/airflow/airflow/__main__.py", line 59, in main
       args.func(args)
     File "/opt/airflow/airflow/cli/cli_config.py", line 49, in command
       return func(*args, **kwargs)
     File "/opt/airflow/airflow/utils/cli.py", line 114, in wrapper
       return f(*args, **kwargs)
     File "/opt/airflow/airflow/utils/providers_configuration_loader.py", line 
55, in wrapped_function
       return func(*args, **kwargs)
     File "/opt/airflow/airflow/cli/commands/dag_command.py", line 153, in 
dag_backfill
       _run_dag_backfill(dags, args)
     File "/opt/airflow/airflow/cli/commands/dag_command.py", line 105, in 
_run_dag_backfill
       dag.run(
     File "/opt/airflow/airflow/models/dag.py", line 2671, in run
       run_job(job=job, execute_callable=job_runner._execute)
     File "/opt/airflow/airflow/utils/session.py", line 79, in wrapper
       return func(*args, session=session, **kwargs)
     File "/opt/airflow/airflow/jobs/job.py", line 305, in run_job
       return execute_job(job, execute_callable=execute_callable)
     File "/opt/airflow/airflow/jobs/job.py", line 334, in execute_job
       ret = execute_callable()
     File "/opt/airflow/airflow/utils/session.py", line 79, in wrapper
       return func(*args, session=session, **kwargs)
     File "/opt/airflow/airflow/jobs/backfill_job_runner.py", line 949, in 
_execute
       raise BackfillUnfinished(err, ti_status)
   airflow.exceptions.BackfillUnfinished: BackfillJob is deadlocked.
   These tasks have succeeded:
   DAG ID           Task ID       Run ID                                 Try 
number
   ---------------  ------------  -----------------------------------  
------------
   stale_data_test  get_run_data  backfill__2023-07-16T06:00:00+00:00           
  1
   stale_data_test  get_run_data  backfill__2023-07-17T06:00:00+00:00           
  1
   stale_data_test  get_run_data  backfill__2023-07-18T06:00:00+00:00           
  1
   stale_data_test  get_run_data  backfill__2023-07-19T06:00:00+00:00           
  1
   stale_data_test  get_run_data  backfill__2023-07-20T06:00:00+00:00           
  1
   stale_data_test  get_run_data  backfill__2023-07-21T06:00:00+00:00           
  1
   stale_data_test  get_run_data  backfill__2023-07-22T06:00:00+00:00           
  1
   stale_data_test  get_run_data  backfill__2023-07-23T06:00:00+00:00           
  1
   stale_data_test  get_run_data  backfill__2023-07-24T06:00:00+00:00           
  1
   stale_data_test  get_run_data  backfill__2023-07-25T06:00:00+00:00           
  1
   stale_data_test  get_run_data  backfill__2023-07-26T06:00:00+00:00           
  1
   stale_data_test  get_run_data  backfill__2023-07-27T06:00:00+00:00           
  1
   stale_data_test  get_run_data  backfill__2023-07-28T06:00:00+00:00           
  1
   stale_data_test  get_run_data  backfill__2023-07-29T06:00:00+00:00           
  1
   stale_data_test  get_run_data  backfill__2023-07-30T06:00:00+00:00           
  1
   stale_data_test  get_run_data  backfill__2023-07-31T06:00:00+00:00           
  1
   
   These tasks are running:
   DAG ID    Task ID    Run ID    Try number
   --------  ---------  --------  ------------
   
   These tasks have failed:
   DAG ID    Task ID    Run ID    Try number
   --------  ---------  --------  ------------
   
   These tasks are skipped:
   DAG ID    Task ID    Run ID    Try number
   --------  ---------  --------  ------------
   
   These tasks are deadlocked:
   DAG ID           Task ID                       Run ID                        
         Map Index    Try number
   ---------------  ----------------------------  
-----------------------------------  -----------  ------------
   stale_data_test  insert_new_daily_data.insert  
backfill__2023-07-19T06:00:00+00:00            1             0
   stale_data_test  insert_new_daily_data.insert  
backfill__2023-07-22T06:00:00+00:00            1             0
   stale_data_test  insert_new_daily_data.insert  
backfill__2023-07-30T06:00:00+00:00            1             0
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to