[issue35267] reproducible deadlock with multiprocessing.Pool

Davin Potts added the comment: I second what @vstinner already said in the comments for PR11143, that this should not merely be documented. -- nosy: +davin ___ Python tracker

[issue35267] reproducible deadlock with multiprocessing.Pool

Change by Windson Yang : -- keywords: +patch pull_requests: +10374 stage: -> patch review ___ Python tracker ___ ___

[issue35267] reproducible deadlock with multiprocessing.Pool

Change by STINNER Victor : -- nosy: +pablogsal ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue35267] reproducible deadlock with multiprocessing.Pool

Change by STINNER Victor : -- nosy: +vstinner ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue35267] reproducible deadlock with multiprocessing.Pool

Jonathan Gossage added the comment: I think documentation is sufficient but I would like it to state the pitfalls available if apply_async is not synchronized correctly which will happen whenever the output does not fit the pipe buffer. -- ___

[issue35267] reproducible deadlock with multiprocessing.Pool

Windson Yang added the comment: As Jonathan Gossage said, I think it may break some code to fix this issue, maybe we could just add a warning on the document? -- ___ Python tracker

[issue35267] reproducible deadlock with multiprocessing.Pool

Change by Jonathan Gossage : Added file: https://bugs.python.org/file47980/lock1.result.txt ___ Python tracker ___ ___ Python-bugs-list

[issue35267] reproducible deadlock with multiprocessing.Pool

Jonathan Gossage added the comment: This is a great example of abusing the multi-processing API and thus creating timing errors that lead to locks not being released. What is happening is that the example attempts to transmit data that is too big for the underlying pipe and this creates the

[issue35267] reproducible deadlock with multiprocessing.Pool

dzhu added the comment: Given the hairiness of the deadlock, I think I would rather let someone who has more experience with the codebase in general handle it, but I can come back to it if it doesn't get addressed. -- ___ Python tracker

[issue35267] reproducible deadlock with multiprocessing.Pool

Windson Yang added the comment: I will work on it if no one wants to create a PR for this next week. -- ___ Python tracker ___ ___

[issue35267] reproducible deadlock with multiprocessing.Pool

Change by Neil Conway : -- nosy: +nconway ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue35267] reproducible deadlock with multiprocessing.Pool

Change by Ned Deily : -- nosy: +pitrou ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue35267] reproducible deadlock with multiprocessing.Pool

Windson Yang added the comment: Hello, dzhu. I can reproduce on my OSX, since you already dive into the code, do you have any idea to fix or improve it? -- nosy: +Windson Yang ___ Python tracker

[issue35267] reproducible deadlock with multiprocessing.Pool

New submission from dzhu : The attached snippet causes a deadlock just about every time it's run (tested with 3.6.7/Ubuntu, 3.7.1/Arch, 3.6.7/OSX, and 3.7.1/OSX -- deadlock seems to be less frequent on the last, but still common). The issue appears to be something like the following sequence