[issue42406] Importing multiprocessing breaks pickle.whichmodule

[Roundup Robot]

Change by Roundup Robot :


--
nosy: +python-dev
nosy_count: 5.0 -> 6.0
pull_requests: +26885
pull_request: https://github.com/python/cpython/pull/28489

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42406] Importing multiprocessing breaks pickle.whichmodule

[Renato Cunha]

Renato Cunha  added the comment:

No worries, Gregory. Glad I could help. :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42406] Importing multiprocessing breaks pickle.whichmodule

[Gregory P. Smith]

Gregory P. Smith  added the comment:

thanks Renato!

--
resolution:  -> fixed
stage: patch review -> commit review
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42406] Importing multiprocessing breaks pickle.whichmodule

[miss-islington]

miss-islington  added the comment:


New changeset fcf7391f598e027a083173662a3ecf7865251291 by Miss Islington (bot) 
in branch '3.9':
bpo-42406: Fix whichmodule() with multiprocessing (GH-23403)
https://github.com/python/cpython/commit/fcf7391f598e027a083173662a3ecf7865251291


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42406] Importing multiprocessing breaks pickle.whichmodule

[miss-islington]

miss-islington  added the comment:


New changeset b1c48e513624641d6472262c33d09624170ea1f5 by Miss Islington (bot) 
in branch '3.8':
bpo-42406: Fix whichmodule() with multiprocessing (GH-23403)
https://github.com/python/cpython/commit/b1c48e513624641d6472262c33d09624170ea1f5


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42406] Importing multiprocessing breaks pickle.whichmodule

[miss-islington]

Change by miss-islington :


--
pull_requests: +22442
pull_request: https://github.com/python/cpython/pull/23561

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42406] Importing multiprocessing breaks pickle.whichmodule

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 4.0 -> 5.0
pull_requests: +22441
pull_request: https://github.com/python/cpython/pull/23560

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42406] Importing multiprocessing breaks pickle.whichmodule

[Gregory P. Smith]

Gregory P. Smith  added the comment:


New changeset 86684319d3dad8e1a7b0559727a48e0bc50afb01 by Renato Cunha in 
branch 'master':
bpo-42406: Fix whichmodule() with multiprocessing (GH-23403)
https://github.com/python/cpython/commit/86684319d3dad8e1a7b0559727a48e0bc50afb01


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42406] Importing multiprocessing breaks pickle.whichmodule

[Gregory P. Smith]

Gregory P. Smith  added the comment:

gross / nice find :)

--
assignee:  -> gregory.p.smith
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42406] Importing multiprocessing breaks pickle.whichmodule

[E. Paine]

Change by E. Paine :


--
nosy: +davin, pitrou
versions:  -Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42406] Importing multiprocessing breaks pickle.whichmodule

[Renato Cunha]

Change by Renato Cunha :


--
keywords: +patch
pull_requests: +22295
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/23403

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42406] Importing multiprocessing breaks pickle.whichmodule

[Renato Cunha]

Change by Renato Cunha :


--
components: +Library (Lib) -Interpreter Core

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42406] Importing multiprocessing breaks pickle.whichmodule

[Renato Cunha]

New submission from Renato Cunha :

Importing multiprocessing prior to other modules that define `ufunc`s breaks 
pickle.whichmodule for those functions.

Example:

Python 3.8.6 (default, Sep 30 2020, 04:00:38)
[GCC 10.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import multiprocessing
>>> from scipy.special import gdtrix
>>> from pickle import whichmodule
>>> whichmodule(gdtrix, gdtrix.__name__)
'__mp_main__'

Now, if we change the order of imports, whichmodule works fine:

Python 3.8.6 (default, Sep 30 2020, 04:00:38)
[GCC 10.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from scipy.special import gdtrix
>>> from pickle import whichmodule
>>> import multiprocessing
>>> whichmodule(gdtrix, gdtrix.__name__)
'scipy.special._ufuncs'

This happens because multiprocessing creates an alias to `__main__` when 
imported:

Lib/multiprocessing/__init__.py
37:sys.modules['__mp_main__'] = sys.modules['__main__']

But doing so changes the ordering of `sys.modules`, and `whichmodule` returns 
`__mp_main__` instead of the correct `scipy.special._ufuncs`.

This bug has arisen in the context of using `dill` (which imports 
multiprocessing) for serialization and is further documented at 
https://github.com/uqfoundation/dill/issues/392.

I intend to submit a patch/PR with a fix soon.

--
components: Interpreter Core
messages: 381410
nosy: renatolfc
priority: normal
severity: normal
status: open
title: Importing multiprocessing breaks pickle.whichmodule
type: behavior
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com