[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)

Ned Deily added the comment: In Python 3.8, the default start method for multiprocessing when run on macOS is now "spawn" instead of "fork". "Changed in version 3.8: On macOS, the spawn start method is now the default. The fork start method should be considered unsafe as it can lead to

[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)

Ronald Oussoren added the comment: That's annoying, I cannot reproduce the issue on macOS 10.13. I don't have access to my test VM running 10.11 at the moment, I'll work on documenting this when I get back home (and do have access to 10.11 again). BTW. I don't know if providing a reliable

[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)

Terry J. Reedy added the comment: Ronald Oussoren> I can provide a patch, but that will likely be during EuroPython as I’m currently taking a roundabout route towards Edinburgh. Can you add a test of the procedure you document for using _tkinter with multiprocessing? --

[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)

Antoine Pitrou added the comment: It will definitely break *some* user code. Also, everyone not using tkinter (which I think is the majority of users) isn't affected by this. -- ___ Python tracker

[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)

Ronald Oussoren added the comment: I can provide a patch, but that will likely be during EuroPython as I’m currently taking a roundabout route towards Edinburgh. Btw. I’m not sure yet how impactful changing the default spawning mechanism would be for 3.8. How likely is that to break user

[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)

Antoine Pitrou added the comment: macOS users, feel free to propose a doc PR for multiprocessing. -- assignee: -> docs@python components: +Documentation nosy: +docs@python versions: +Python 3.7, Python 3.8 -Python 3.5 ___ Python tracker

[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)

Terry J. Reedy added the comment: I was uncertain from Ethan's post what does work best. "endless errors with both 'spawn", "Only 'fork' works...but without graphics", "method='spawn'and ... does work". Do the 1st and 3rd sentences mean that spawn + graphics fails without the proper

[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)

Ronald Oussoren added the comment: As I mentioned in msg314304 this is almost certainly expected behavior on macOS because Apple’s GUI frameworks are not fork()-safe. There is nothing we can do about this other than changing the default spawn mechanism for multiprocessing on macOS. -- On

[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)

Terry J. Reedy added the comment: On MacOS, 3.7.0 is compiled for and the installer loads tcl/tk 8.6.8. The same is true for the 3.6.6 64-bit installer. Do tkinter and multiprocessing work together better with these installations? I want to consider using multiprocessing and pipes instead

[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)

Ethan Welty added the comment: @ronaldoussoren: The order of the imports made no difference. Even with the call at the top, I got endless errors with both 'spawn' and 'forkserver', with or without importing a graphics backend. Only 'fork' works, and only if a graphics

[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)

Ronald Oussoren added the comment: @ezwelty: The import of multiprocessing and the call to set_start_method should be at the very start of the code, before other imports, to avoid the annoying Apple behavior I mentioned. --

[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)

Ethan Welty added the comment: Terry Reedy: I just tried your suggestion of a main clause (code below) and it made no difference. ``` import _tkinter import numpy as np # multiprocessing.set_start_method("spawn") import multiprocessing def parallel_matmul(x): R =

[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)

Terry J. Reedy added the comment: The examples in the multiprocessing doc all put all multiprocessing calls within a "if __name__ == '__main__':" statement. I know that this is necessary on Windows, but don't know if or when it helps on other OSes. -- nosy:

[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)

Ronald Oussoren added the comment: I cannot reproduce the problem on macOS 10.13.3 with python 3.6 (but haven't spent much time on this). The combination of GUI code with multiprocessing is a known source of problems on macOS, the system libraries are hostile to that.

[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)

Change by Ethan Welty : -- components: +macOS ___ Python tracker ___ ___

[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)

Ethan Welty added the comment: I've tried with additional backends: WX, WXAgg, WXCairo, Qt5Agg (in matplotlib speak). With these, I can at least import matplotlib.pyplot, but as soon as say matplotlib.pyplot.plot is called and the backend is loaded, the code breaks

[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)

Ethan Welty added the comment: I have tried running the script with: - command line (python ): Works without (breaks with) `import _tkinter`. - basic python console (python): Same as with command line. - ipython: Fails with or without `import _tkinter`. However, if for

[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)

Ronald Oussoren added the comment: I'm fairly sure this is expected behavior on macOS: _tkinter loads the Tk library, which loads Apple GUI frameworks. Those frameworks are not save w.r.t. fork(2) and that can lead to all kinds of unwanted behavior (although I would

[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)

Ned Deily added the comment: Are you by any chance trying to run this under IDLE? -- nosy: +ned.deily ___ Python tracker ___

[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)

Change by Ethan Welty : -- versions: -Python 3.8 ___ Python tracker ___ ___

[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)

New submission from Ethan Welty : Merely importing tkinter breaks the use of parallel code on my system (Mac OSX 10.11.6, tested on Python 2.7.13 / 2.7.14 / 3.5.0 / 3.6.4, all barebones distributions installed with pyenv). I've tested this with both multiprocessing and