[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)
Ethan Welty <ethan.we...@gmail.com> added the comment:
@ronaldoussoren: The order of the imports made no difference. Even with the
call at the top, I got endless errors with both 'spawn' and 'forkserver', with
or without importing a graphics backend. Only 'fork' works, and only if a
graphics package is not imported.
** HOWEVER **
Setting method='spawn' or 'forkserver' and force=True at top, or calling
multiprocessing.set_start_method in __main__ does work (when run as scripts
from command line). It's a shame to have to give up the convenience of 'fork',
but this is a start.
```
import multiprocessing
multiprocessing.set_start_method("spawn", force=True)
import numpy as np
# import _tkinter
def parallel_matmul(x):
R = np.random.randn(3, 3)
return np.matmul(R, x)
if __name__ == '__main__':
# multiprocessing.set_start_method("spawn", force=False)
pool = multiprocessing.Pool(4)
results = pool.map(parallel_matmul, [np.random.randn(3, 5000) for i in
range(2)])
```
--
___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue33111>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)
Ethan Welty <ethan.we...@gmail.com> added the comment:
Terry Reedy: I just tried your suggestion of a main clause (code below) and it
made no difference.
```
import _tkinter
import numpy as np
# multiprocessing.set_start_method("spawn")
import multiprocessing
def parallel_matmul(x):
R = np.random.randn(3, 3)
return np.matmul(R, x)
if __name__ == '__main__':
pool = multiprocessing.Pool(4)
results = pool.map(parallel_matmul, [np.random.randn(3, 5000) for i in
range(2)])
```
Ronald Oussoren: Good to know that perhaps upgrading to the latest OSX would
resolve this issue. I tried 'spawn' and 'forkserver' with
multiprocessing.set_start_method() (see above) but both result in an endless
stream of errors (with or without importing a graphics backend), although that
may be my not knowing how to properly use them.
--
___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue33111>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)
Change by Ethan Welty <ethan.we...@gmail.com>: -- components: +macOS ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue33111> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)
Ethan Welty <ethan.we...@gmail.com> added the comment: I've tried with additional backends: WX, WXAgg, WXCairo, Qt5Agg (in matplotlib speak). With these, I can at least import matplotlib.pyplot, but as soon as say matplotlib.pyplot.plot is called and the backend is loaded, the code breaks (error message for WXAgg below). Is multiprocessing in an interactive shell simply not meant to be supported on MacOS? Error in atexit._run_exitfuncs: Traceback (most recent call last): File "/Users/Admin/.pyenv/versions/3.6.4/Python.framework/Versions/3.6/lib/python3.6/site-packages/matplotlib/_pylab_helpers.py", line 78, in destroy_all manager.destroy() File "/Users/Admin/.pyenv/versions/3.6.4/Python.framework/Versions/3.6/lib/python3.6/site-packages/matplotlib/backends/backend_wx.py", line 1303, in destroy self.frame.Destroy() File "/Users/Admin/.pyenv/versions/3.6.4/Python.framework/Versions/3.6/lib/python3.6/site-packages/matplotlib/backends/backend_wx.py", line 1256, in Destroy if not self.IsBeingDeleted(): RuntimeError: wrapped C/C++ object of type FigureFrameWxAgg has been deleted -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue33111> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)
Ethan Welty <ethan.we...@gmail.com> added the comment: I have tried running the script with: - command line (python ): Works without (breaks with) `import _tkinter`. - basic python console (python): Same as with command line. - ipython: Fails with or without `import _tkinter`. However, if for example I make the matrix in the scripts smaller or use only functions that don't resort to multithreading, code runs successfully in parallel in ipython. - IDLE: Perhaps unsurprisingly, the code fails with or without `import _tkinter` (never terminates, or prompts endless "Your program is still running! Do you want to kill it?") Suspecting a problem with Tcl/Tk (Apple's original 8.5.9), I rebuilt python pointing to tcl-tk installed with brew (8.6.8), which I checked with IDLE and _tkinter.TCL_VERSION() / _tkinter.TK_VERSION(). However, this did not fix the problem. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue33111> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)
Change by Ethan Welty <ethan.we...@gmail.com>: -- versions: -Python 3.8 ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue33111> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33111] Merely importing tkinter breaks parallel code (multiprocessing, sharedmem)
New submission from Ethan Welty <ethan.we...@gmail.com>: Merely importing tkinter breaks the use of parallel code on my system (Mac OSX 10.11.6, tested on Python 2.7.13 / 2.7.14 / 3.5.0 / 3.6.4, all barebones distributions installed with pyenv). I've tested this with both multiprocessing and sharedmem (see minimal scripts below). The issue seems to apply only to functions that evoke multithreading within their respective package (e.g. `numpy.matmul()`, `cv2.SIFT.detectAndCompute()`). If I make the matrix in the scripts below much smaller (e.g. change `5000` to `5`), avoiding internal multithreading, the scripts work. ## with `multiprocessing` ```python import numpy as np import multiprocessing import _tkinter def parallel_matmul(x): R = np.random.randn(3, 3) return np.matmul(R, x) pool = multiprocessing.Pool(4) results = pool.map(parallel_matmul, [np.random.randn(3, 5000) for i in range(2)]) ``` > *Code never exits and Python has to be force quit* ## with `sharedmem` ```python import numpy as np import sharedmem import _tkinter def parallel_matmul(x): R = np.random.randn(3, 3) return np.matmul(R, x) with sharedmem.MapReduce() as pool: results = pool.map(parallel_matmul, [np.random.randn(3, 5000) for i in range(2)]) ``` > sharedmem.sharedmem.SlaveException: slave process 1 killed by signal 11 -- components: Tkinter messages: 314160 nosy: ezwelty priority: normal severity: normal status: open title: Merely importing tkinter breaks parallel code (multiprocessing, sharedmem) type: crash versions: Python 2.7, Python 3.5, Python 3.6, Python 3.8 ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue33111> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
