potiuk edited a comment on issue #5159: [AIRFLOW-XXX] Attempt to remove flakeyness of LocalExecutor URL: https://github.com/apache/airflow/pull/5159#issuecomment-486063506 Hey @BasPH @ashb . I woke up this morning and could not stop thinking about that one and I took a deeeeep dive. It was great idea @BasPH to make it reproducible locally, that has helped me a lot. I added a lot of logging and eventually it boiled down to cases where queue.empty() returned True where queue.qsize() returned 1. I thought I lost my mind for a while, but well ... It turned out that this this is INTENDED BEHAVIOUR of multiprocessing.Queue. See the issue https://bugs.python.org/issue23582 (resolved as "not a bug"). I have no words for it, but well, we needed to solve it. Unfortunately simple solution does not work on MacOS (qsize() does not work on MacOS and throws NotImplemented) ... So at the end I had to do what others did in the same case - have a custom SynchronizedQueue deriving from multiprocessing.Queue and add extra synchronisation/counter to put/get methods :( I described all details in https://issues.apache.org/jira/browse/AIRFLOW-4401 and created a PR to fix it here: https://github.com/apache/airflow/pull/5167 Speaking about productive morning :).
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
