Jorricks opened a new pull request #16550: URL: https://github.com/apache/airflow/pull/16550
Related: https://github.com/apache/airflow/issues/13542 The issue discussed here was caused by multiple things. One of the issues is that when the scheduler picks up 'assumed to be' `orphaned` tasks, these tasks might have never made it to celery. When the tasks execution never happens, it is automatically cleaned up but only partially. Then once the scheduler retries to queue the task again, it won't be able to, because there is still a reference of the task in the set of the `running` variable. This PR should fix the described issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
