[issue32485] Multiprocessing dict sharing between forked processes
André Neto <andre.c.n...@gmail.com> added the comment: You are right, it does not segfault (sorry for the abuse of language). It raises an exception while accessing the shared dictionary. The exception varies but typically is: Traceback (most recent call last): File "multiprocessbug.py", line 156, in test_manyForkedProcessesSingleThreaded(inst1, inst2, nRuns, nProcesses) File "multiprocessbug.py", line 77, in test_manyForkedProcessesSingleThreaded run(inst1, nRuns) File "multiprocessbug.py", line 29, in run inst.run() File "multiprocessbug.py", line 18, in run if (self.d.has_key(self.key)): File "", line 2, in has_key File "/usr/local/lib/python2.7/multiprocessing/managers.py", line 759, in _callmethod kind, result = conn.recv() cPickle.UnpicklingError: invalid load key, '#'. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32485> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32485] Multiprocessing dict sharing between forked processes
New submission from André Neto <andre.c.n...@gmail.com>: I'm working on a project where I need to share state between several processes (forked using gunicorn). I'm using dictionaries obtained from a multiprocessing SyncManager to achieve this. The issue is that if I have multiple forked processes concurrently accessing to different dictionaries created by the same SyncManager, the code will randomly crash while accessing (read-only and read-write) to any of the dictionaries. This behaviour is independent of the way I access the dictionary (i.e. it happens both using the dictionary functions (e.g. has_key) or the built-in keywords (e.g. key in dict). The attached snippet demonstrates the issue (tested only in python2.7): the function test_manyForkedProcessesSingleThreaded will crash. The issue is also being discussed here: https://stackoverflow.com/questions/48052148/python-multiprocessing-dict-sharing-between-processes -- files: multiprocessing_crash.py messages: 309402 nosy: André Neto priority: normal severity: normal status: open title: Multiprocessing dict sharing between forked processes type: crash versions: Python 2.7 Added file: https://bugs.python.org/file47361/multiprocessing_crash.py ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32485> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
