[issue18473] some objects pickled by Python 3.x are not unpicklable in Python 2.x because of incorrect REVERSE_IMPORT_MAPPING
Roundup Robot added the comment: New changeset 8d86dfe53b97 by Serhiy Storchaka in branch '3.4': Issue #18473: Fixed 2to3 and 3to2 compatible pickle mappings. https://hg.python.org/cpython/rev/8d86dfe53b97 New changeset 5980e81219ed by Serhiy Storchaka in branch 'default': Issue #18473: Fixed 2to3 and 3to2 compatible pickle mappings. https://hg.python.org/cpython/rev/5980e81219ed -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18473 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18473] some objects pickled by Python 3.x are not unpicklable in Python 2.x because of incorrect REVERSE_IMPORT_MAPPING
STINNER Victor added the comment:
Buildbots are unhappy. Example:
http://buildbot.python.org/all/builders/x86%20Ubuntu%20Shared%203.x/builds/11437/steps/test/logs/stdio
==
ERROR: test_name_mapping (test.test_pickle.CompatPickleTests)
[(('tkinter.filedialog', 'FileDialog'), ('FileDialog', 'FileDialog'))]
--
Traceback (most recent call last):
File
/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_pickle.py, line
241, in getmodule
return sys.modules[module]
KeyError: 'tkinter.filedialog'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File
/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_pickle.py, line
286, in test_name_mapping
attr = getattribute(module3, name3)
File
/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_pickle.py, line
247, in getattribute
obj = getmodule(module)
File
/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_pickle.py, line
243, in getmodule
__import__(module)
File
/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/tkinter/__init__.py, line
38, in module
import _tkinter # If this fails your Python may not be configured for Tk
ImportError: No module named '_tkinter'
--
nosy: +haypo
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18473
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18473] some objects pickled by Python 3.x are not unpicklable in Python 2.x because of incorrect REVERSE_IMPORT_MAPPING
STINNER Victor added the comment: A different error: http://buildbot.python.org/all/builders/AMD64%20Snow%20Leop%203.x/builds/2880/steps/test/logs/stdio == FAIL: test_class_nested_enum_and_pickle_protocol_four (test.test_enum.TestEnum) -- Traceback (most recent call last): File /Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/test/test_enum.py, line 580, in test_class_nested_enum_and_pickle_protocol_four protocol=(0, 3)) File /Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/test/test_enum.py, line 82, in test_pickle_exception dumps(obj, protocol=protocol) AssertionError: PicklingError not raised -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18473 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18473] some objects pickled by Python 3.x are not unpicklable in Python 2.x because of incorrect REVERSE_IMPORT_MAPPING
STINNER Victor added the comment: A third kind of error: http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%203.x/builds/5992/steps/test/logs/stdio == ERROR: test_import (test.test_pickle.CompatPickleTests) -- Traceback (most recent call last): File C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\test_pickle.py, line 241, in getmodule return sys.modules[module] KeyError: 'multiprocessing.popen_fork' During handling of the above exception, another exception occurred: Traceback (most recent call last): File C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\test_pickle.py, line 266, in test_import getmodule(module) File C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\test_pickle.py, line 243, in getmodule __import__(module) File C:\buildbot.python.org\3.x.kloth-win64\build\lib\multiprocessing\popen_fork.py, line 13, in module class Popen(object): File C:\buildbot.python.org\3.x.kloth-win64\build\lib\multiprocessing\popen_fork.py, line 25, in Popen def poll(self, flag=os.WNOHANG): AttributeError: module 'os' has no attribute 'WNOHANG' -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18473 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18473] some objects pickled by Python 3.x are not unpicklable in Python 2.x because of incorrect REVERSE_IMPORT_MAPPING
Roundup Robot added the comment: New changeset 5c5eb374e296 by Serhiy Storchaka in branch '3.4': Issue #18473: Fixed pickle compatibility tests for optional modules. https://hg.python.org/cpython/rev/5c5eb374e296 New changeset 29b2b2d8e36f by Serhiy Storchaka in branch 'default': Issue #18473: Fixed pickle compatibility tests for optional modules. https://hg.python.org/cpython/rev/29b2b2d8e36f -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18473 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18473] some objects pickled by Python 3.x are not unpicklable in Python 2.x because of incorrect REVERSE_IMPORT_MAPPING
Serhiy Storchaka added the comment: Fixed import errors. A second kind of errors was related to issue23611. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18473 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18473] some objects pickled by Python 3.x are not unpicklable in Python 2.x because of incorrect REVERSE_IMPORT_MAPPING
Changes by Serhiy Storchaka storch...@gmail.com: -- resolution: - fixed stage: patch review - resolved status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18473 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18473] some objects pickled by Python 3.x are not unpicklable in Python 2.x because of incorrect REVERSE_IMPORT_MAPPING
STINNER Victor added the comment: Le mardi 31 mars 2015, Roundup Robot rep...@bugs.python.org a écrit : Roundup Robot added the comment: New changeset 5c5eb374e296 by Serhiy Storchaka in branch '3.4': Issue #18473: Fixed pickle compatibility tests for optional modules. https://hg.python.org/cpython/rev/5c5eb374e296 New changeset 29b2b2d8e36f by Serhiy Storchaka in branch 'default': Issue #18473: Fixed pickle compatibility tests for optional modules. https://hg.python.org/cpython/rev/29b2b2d8e36f Why do you catch AttributeError on import? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18473 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18473] some objects pickled by Python 3.x are not unpicklable in Python 2.x because of incorrect REVERSE_IMPORT_MAPPING
Serhiy Storchaka added the comment: Why do you catch AttributeError on import? Because importing multiprocessing.popen_fork raises AttributeError on Windows. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18473 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18473] some objects pickled by Python 3.x are not unpicklable in Python 2.x because of incorrect REVERSE_IMPORT_MAPPING
Serhiy Storchaka added the comment: Updated patch fixes more bugs. Added support of unicode, izip_longest, abstract collections, ChainMap, multiprocessing exceptions, some socket and multiprocessing functions and types, xml.etree.ElementTree (C implementation). Added support of urllib and urllib2 (only public interface). Fixed reverse mapping of OSError subclasses, str, bz2, different dbm implementations, pickle. Added special sanity tests for _compat_pickle mappings (test that mappings are not ambiguous and mainly reversible, that 3.x names exist, etc). -- Added file: http://bugs.python.org/file38669/pickle_fix_import_2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18473 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18473] some objects pickled by Python 3.x are not unpicklable in Python 2.x because of incorrect REVERSE_IMPORT_MAPPING
Serhiy Storchaka added the comment: Ping. -- keywords: +needs review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18473 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18473] some objects pickled by Python 3.x are not unpicklable in Python 2.x because of incorrect REVERSE_IMPORT_MAPPING
Serhiy Storchaka added the comment: Here is a patch which fixes multiple reversible mappings. Mappings for StringIO and cStringIO is removed at all because cStringIO.StringIO is not pickleable on 2.7 at all and pickled StringIO.StringIO is not compatible with 3.x (issue22890). -- Added file: http://bugs.python.org/file37210/pickle_fix_import.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18473 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18473] some objects pickled by Python 3.x are not unpicklable in Python 2.x because of incorrect REVERSE_IMPORT_MAPPING
Changes by Serhiy Storchaka storch...@gmail.com: -- stage: needs patch - patch review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18473 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18473] some objects pickled by Python 3.x are not unpicklable in Python 2.x because of incorrect REVERSE_IMPORT_MAPPING
Serhiy Storchaka added the comment: I think that both UserDict.UserDict and UserDict.IterableUserDict should be mapped to collections.UserDict. And reverse mapping should map collections.UserDict to UserDict.IterableUserDict. There are similar issues with other multiple to single mappings (e.g. to io, dbm, http.server). -- nosy: +serhiy.storchaka versions: +Python 3.5 -Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18473 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18473] some objects pickled by Python 3.x are not unpicklable in Python 2.x because of incorrect REVERSE_IMPORT_MAPPING
Doug Royal added the comment: This patch only addresses the proven errors with UserList, UserString, and collections. -- keywords: +patch nosy: +doug.royal Added file: http://bugs.python.org/file37145/fix_issue18473.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18473 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18473] some objects pickled by Python 3.x are not unpicklable in Python 2.x because of incorrect REVERSE_IMPORT_MAPPING
Changes by Alexandre Vassalotti alexan...@peadrop.com: -- assignee: - alexandre.vassalotti priority: normal - high stage: - needs patch type: - behavior versions: +Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18473 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18473] some objects pickled by Python 3.x are not unpicklable in Python 2.x because of incorrect REVERSE_IMPORT_MAPPING
Changes by Terry J. Reedy tjre...@udel.edu: -- nosy: +alexandre.vassalotti, pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18473 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18473] some objects pickled by Python 3.x are not unpicklable in Python 2.x because of incorrect REVERSE_IMPORT_MAPPING
New submission from Mikhail Korobov: I think REVERSE_IMPORT_MAPPING is wrong (see http://hg.python.org/cpython/file/7272ef213b7c/Lib/_compat_pickle.py#l80 ). It relies on dictionary items order and maps 'collections' module to either UserString or UserList. This makes it impossible to unpickle collections module classes pickled by Python 3.x when using Python 2.x. import collections dct = collections.defaultdict() pickle.dumps(dct, protocol=1) b'cUserString\ndefaultdict\nq\x00)Rq\x01.' pickle.dumps(dct, protocol=2, fix_imports=False) b'ccollections\ndefaultdict\nq\x00)Rq\x01.' I think pickling of instances of classes from other modules could also suffer. I don't think it is a good idea to map io to cStringIO or StringIO, or to map http.server to either BaseHTTPServer, SimpleHTTPServer or CGIHTTPServer. Originally found this issue here: https://github.com/nltk/nltk/pull/441/files#L2R67 -- components: Library (Lib) messages: 193160 nosy: kmike priority: normal severity: normal status: open title: some objects pickled by Python 3.x are not unpicklable in Python 2.x because of incorrect REVERSE_IMPORT_MAPPING versions: Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18473 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18473] some objects pickled by Python 3.x are not unpicklable in Python 2.x because of incorrect REVERSE_IMPORT_MAPPING
Mikhail Korobov added the comment: my copy-paste went wrong, last statement in the example should be pickle.dumps(dct, protocol=1, fix_imports=False) b'ccollections\ndefaultdict\nq\x00)Rq\x01.' -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18473 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
