[issue31886] Multiprocessing.Pool hangs after re-spawning several worker process.

2017-10-29 Thread Antoine Pitrou
Change by Antoine Pitrou : -- nosy: +Olivier.Grisel, tomMoral ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue31886] Multiprocessing.Pool hangs after re-spawning several worker process.

2017-10-29 Thread Antoine Pitrou
Antoine Pitrou added the comment: Generally speaking, queues can remain in an inconsistent state after a process crash (because the process might have crashed just after acquiring a shared semaphore or sending part of a large message). It's not obvious to me how we could make them safer, at

[issue31886] Multiprocessing.Pool hangs after re-spawning several worker process.

2017-10-27 Thread olarn
New submission from olarn : Multiprocessing's pool apparently attempts to repopulate the pool in an event of sub-process worker crash. However the pool seems to hangs after about ~ 4*(number of worker) process re-spawns. I've tracked the issue down to queue.get() stalling at multiprocessing.po