bartgenuit commented on issue #33402: URL: https://github.com/apache/airflow/issues/33402#issuecomment-1770265317
@arjunanan6 That is interesting, thanks for pointing that out. We also encountered the problem where Airflow just "hangs" indefinitely, I suspect as a result of the same bug. We are indeed running two scheduler pods, it is the default in our Helm chart. We rolled back the requirements.txt we install on top of the official Airflow docker image, and now everything seems to run fine again. So most probably the bug has to do with the schedulers not working well together (one locking resources but crashing and not restarting, for example) and a certain combination of certain packages plus random network issues that serve as a trigger. I do not have infinite time either, but I am going to take a shot at creating a minimal example to reproduce this bug. I have at least one "bad" and one "good" example to start from. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
