[issue33945] concurrent.futures ProcessPoolExecutor submit() blocks on results being written

Daniel Barcay added the comment: Just got the drop of the python3.7 release. I can confirm that this is fixed in python3.7 in my workload. Nice job! Thanks for changing the mechanism of thread-sync. I'm grateful. -- resolution: -> fixed stage: -> resolved status: open -> closed

[issue33945] concurrent.futures ProcessPoolExecutor submit() blocks on results being written

Thomas Moreau added the comment: This behavior results from the fact that in 3.6, the result_queue is used to pass messages to the queue_manager_thread. This behavior has been changed in 3.7 as we rely on a _ThreadWakeup object. In 3.6, when the result_queue is filled with many large

[issue33945] concurrent.futures ProcessPoolExecutor submit() blocks on results being written

Antoine Pitrou added the comment: I'm not sure what happens exactly in your workload, but waiting 20 seconds when posting some data on an unbounded queue sounds enormous. -- nosy: +tomMoral ___ Python tracker

[issue33945] concurrent.futures ProcessPoolExecutor submit() blocks on results being written

Daniel Barcay added the comment: adding experts bquinlan and pitrou for concurrent.futures to nosy-list as per bug tracker directions. -- nosy: +bquinlan, pitrou ___ Python tracker

[issue33945] concurrent.futures ProcessPoolExecutor submit() blocks on results being written

Daniel Barcay added the comment: Line number was incorrect due to local edits. Correct line number is process.py:L464 "self._result_queue.put(None)" -- ___ Python tracker

[issue33945] concurrent.futures ProcessPoolExecutor submit() blocks on results being written

New submission from Daniel Barcay : I have tracked down the exact cause of a sizable performance issue in using concurrent.futures.ProcessPoolExecutors, especially visible in cases where large amounts of data are being copied across the result. The line-number causing the bad behavior, and