ykanagarajapandiyan commented on issue #42006: URL: https://github.com/apache/airflow/issues/42006#issuecomment-2417845465
Setting param 'use-row-level-locking' to True prevents these DB errors. [https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#use-row-level-locking](url). Even though the doc recommends to set this True only when multiple schedulers are used, I think this conflicts with param [schedule-after-task-execution](https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#schedule-after-task-execution) where a mini scheduler runs and tries to update the same rows that scheduler is working on. But changing this param(use-row-level-locking) to True affects the task start times. I have noticed that even when the task completes, scheduler sometimes takes about 5-10 mins to terminate the pod (we use Local Kubernetes executor) and start the next ones. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
