Greetlist opened a new issue #21817:
URL: https://github.com/apache/airflow/issues/21817


   ### Apache Airflow version
   
   2.2.1
   
   ### What happened
   
   My Scheduler or Trigger Process throw DeadLock Exception randomly.
   Scheduler  log:
   
![image](https://user-images.githubusercontent.com/18439743/155668354-1437c8e1-3f26-47c7-a6b7-2c9cc1bc76e7.png)
   
   Trigger log:
   
![image](https://user-images.githubusercontent.com/18439743/155668430-f6ddde19-2b1e-4e76-940c-58979e966d53.png)
   
   Scheduler exception is thrown out by ``` 
SchedulerJob.check_trigger_timeouts``` function
   Trigger exception is thrown out by ``` Trigger.submit_event ``` function
   
   
   
   
   ### What you expected to happen
   
   Scheduler or Trigger Process never throw DeadLock Exception.
   
   ### How to reproduce
   
   _No response_
   
   ### Operating System
   
   Ubuntu 16.04
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Other
   
   ### Deployment details
   
   1. Three Scheduler
   2. One WebServer
   3. One Trigger
   
   ### Anything else
   
   I think these codes may have conflict:
   
![image](https://user-images.githubusercontent.com/18439743/155668882-98175ebe-ea6f-432b-a660-f3dde6602e87.png)
   
![image](https://user-images.githubusercontent.com/18439743/155668981-ebdf9fcd-dda0-4d63-8265-e7392d25584b.png)
   
   Due to ``` task_instance.trigger_timeout ``` column is not index, Scheduler 
query may **lock all records whose state == "deferred"**. If 
Trigger.submib_event try to update these records which locked by Scheduler, 
should get deadlock.
   
   Now, I fix this bug by adding ``` with_row_locked ``` with these two query:
   
![image](https://user-images.githubusercontent.com/18439743/155670551-346458a9-53d2-48ff-a67e-582f0b27c7c4.png)
   
   
![image](https://user-images.githubusercontent.com/18439743/155670805-d9a501c1-950a-4ff0-9a59-f45e20b52cda.png)
   
   Hope for reply.
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


Reply via email to