[
https://issues.apache.org/jira/browse/AIRFLOW-61?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15274695#comment-15274695
]
Dan Davydov commented on AIRFLOW-61:
------------------------------------
We rolled back pretty quickly once we saw errors in production so we didn't
really have time to see issues like this manifest. Since the roll back I set up
a staging cluster with production DAGs but it still doesn't have workers (only
a webserver and a scheduler) so we still won't be able to catch issues like
this in an automated matter, though we have some ideas on how to test workers
too. Agree about letting the fix bake. I'll cherry pick this commit (thanks for
pointing it out).
> Corner case with deadlocked multiprocessing queues
> --------------------------------------------------
>
> Key: AIRFLOW-61
> URL: https://issues.apache.org/jira/browse/AIRFLOW-61
> Project: Apache Airflow
> Issue Type: Bug
> Components: scheduler
> Affects Versions: Airflow 1.7.0
> Reporter: Jeremiah Lowin
> Assignee: Jeremiah Lowin
> Fix For: Airflow 1.7.1
>
>
> I've never experienced this before, but it came up while testing this
> morning. Joining a process that puts items in a queue can apparently lead to
> deadlocks. This happens in the multiprocessing-enabled scheduler.
> See for example:
> https://docs.python.org/3/library/multiprocessing.html#all-start-methods
> ("Joining processes that use queues") and related issues:
> http://stackoverflow.com/questions/31665328/python-3-multiprocessing-queue-deadlock-when-calling-join-before-the-queue-is-em
> http://stackoverflow.com/questions/31708646/process-join-and-queue-dont-work-with-large-numbers
> http://stackoverflow.com/questions/19071529/python-multiprocessing-125-list-never-finishes
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
