[issue34572] C unpickling bypasses import thread safety

Valentyn Tymofieiev added the comment: While investigating[1], I observe that certain unpickling operations, for example, Unpickler.find_class, remain not thread-safe in Python 3.7.5 and earlier versions that I tried. I have not tried 3.8, but cannot reproduce this error on Python 2. For

[issue34572] C unpickling bypasses import thread safety

Change by Valentyn Tymofieiev : Added file: https://bugs.python.org/file48736/find_class_deadlock.py ___ Python tracker ___ ___

[issue34572] C unpickling bypasses import thread safety

Brett Cannon added the comment: 3.6 and 3.5 are in security mode, so unless there's a security risk due to this bug the fix will not be backported to those older versions (https://devguide.python.org/#status-of-python-branches). -- ___ Python

[issue34572] C unpickling bypasses import thread safety

Guenther Starnberger added the comment: For this issue only 3.7 and 3.8 are listed as affected versions, but it appears to be also reproducible on the latest 3.5 and 3.6 releases. I've attached a script that can be used to reproduce the issue on those earlier releases (it consistently fails

[issue34572] C unpickling bypasses import thread safety

Antoine Pitrou added the comment: This is pushed to 3.7 and master now. Thank you Tim for the report and the fix! -- resolution: -> fixed stage: patch review -> resolved status: open -> closed versions: -Python 3.6 ___ Python tracker

[issue34572] C unpickling bypasses import thread safety

Antoine Pitrou added the comment: New changeset 3129432845948fdcab1e05042e99a19e9e2c3c71 by Antoine Pitrou (Miss Islington (bot)) in branch '3.7': bpo-34572: change _pickle unpickling to use import rather than retrieving from sys.modules (GH-9047) (GH-11921)

[issue34572] C unpickling bypasses import thread safety

Antoine Pitrou added the comment: New changeset 4371c0a9c0848f7a0947d43f26f234842b41efdf by Antoine Pitrou (tjb900) in branch 'master': bpo-34572: change _pickle unpickling to use import rather than retrieving from sys.modules (GH-9047)

[issue34572] C unpickling bypasses import thread safety

Change by miss-islington : -- pull_requests: +11946 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue34572] C unpickling bypasses import thread safety

Eric Snow added the comment: Thanks, Antoine. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue34572] C unpickling bypasses import thread safety

Antoine Pitrou added the comment: Opened issue35943 for PyImport_GetModule(). -- ___ Python tracker ___ ___ Python-bugs-list

[issue34572] C unpickling bypasses import thread safety

Antoine Pitrou added the comment: I agree that more generally PyImport_GetModule() should be fixed. But it should be done carefully so as to not to lose the performance benefit of doing it. I think we should open a separate issue about that. PS: one possibility is to reuse the

[issue34572] C unpickling bypasses import thread safety

Eric Snow added the comment: Perhaps PyImport_GetModule() should aquire-release the module's lock before the lookup? This would effectively be a call to _lock_unlock_module() in importlib._bootstrap. The alternative is to encourage using PyImport_Import() instead, like the PR has done.

[issue34572] C unpickling bypasses import thread safety

Change by Antoine Pitrou : -- nosy: +brett.cannon, eric.snow, ncoghlan ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue34572] C unpickling bypasses import thread safety

Antoine Pitrou added the comment: Interesting I could not reproduce here, even by throwing Pandas into the mix and spawning 1024 workers... -- nosy: +pitrou ___ Python tracker

[issue34572] C unpickling bypasses import thread safety

Change by SilentGhost : -- keywords: +needs review nosy: +alexandre.vassalotti ___ Python tracker ___ ___ Python-bugs-list mailing

[issue34572] C unpickling bypasses import thread safety

Tim Burgess added the comment: Hi! Just wondering if there is anything I can do to move this along? -- ___ Python tracker ___ ___

[issue34572] C unpickling bypasses import thread safety

Change by Tim Burgess : -- keywords: +patch pull_requests: +8509 stage: -> patch review ___ Python tracker ___ ___

[issue34572] C unpickling bypasses import thread safety

New submission from Tim Burgess : Retrieving and using a module directly from sys.modules (from C in this case) leads to a race condition where the module may be importing on another thread but has not yet been initialised. For slow filesystems or large modules (e.g. numpy) this seems to