[issue22955] Pickling of methodcaller, attrgetter, and itemgetter

Serhiy Storchaka added the comment: This is new feature in 3.5. -- versions: -Python 3.4 ___ Python tracker ___

[issue22955] Pickling of methodcaller, attrgetter, and itemgetter

Jason Curtis added the comment: This is still an issue with operator.attrgetter in 3.4.3, even after clearing sys.modules['_operator']: $ python3 Python 3.4.3 (default, Oct 14 2015, 20:28:29) [GCC 4.8.4] on linux Type "help", "copyright", "credits" or "license" for more information. >>>

[issue22955] Pickling of methodcaller, attrgetter, and itemgetter

Changes by Serhiy Storchaka storch...@gmail.com: -- resolution: - fixed stage: patch review - resolved status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22955 ___

[issue22955] Pickling of methodcaller, attrgetter, and itemgetter

Roundup Robot added the comment: New changeset 2688655e431a by Serhiy Storchaka in branch 'default': Issue #22955: Fixed reference leak in attrgetter.repr(). https://hg.python.org/cpython/rev/2688655e431a -- ___ Python tracker rep...@bugs.python.org

[issue22955] Pickling of methodcaller, attrgetter, and itemgetter

Roundup Robot added the comment: New changeset 435bc22f39e3 by Serhiy Storchaka in branch 'default': Issue #22955: attrgetter, itemgetter and methodcaller objects in the operator https://hg.python.org/cpython/rev/435bc22f39e3 -- nosy: +python-dev ___

[issue22955] Pickling of methodcaller, attrgetter, and itemgetter

Roundup Robot added the comment: New changeset c93e5ba1cc20 by Serhiy Storchaka in branch 'default': Issue #22955: Fixed test_operator. It left Python implementation in https://hg.python.org/cpython/rev/c93e5ba1cc20 -- ___ Python tracker

[issue22955] Pickling of methodcaller, attrgetter, and itemgetter

Serhiy Storchaka added the comment: methodcaller with keyword arguments pickled with pickle_getter_and_caller3.patch needs Python 3.5 to unpickle. Following patch pickles it in backward compatible way. -- Added file: http://bugs.python.org/file39395/pickle_getter_and_caller4.patch

[issue22955] Pickling of methodcaller, attrgetter, and itemgetter

Serhiy Storchaka added the comment: Here is revised Josh's patch. Added tests for consistency between both implementations, fixed inconsistencies and bugs. I still hesitate about pickling format of methodcaller. First, there is asymmetry between positional and keyword arguments. Second, for

[issue22955] Pickling of methodcaller, attrgetter, and itemgetter

Serhiy Storchaka added the comment: functools.partial is a somewhat less than ideal comparison. The pure-Python version is not picklable, the Python and C versions return different things (the Python version is a function returning a function, the C version is a regular class and returns

[issue22955] Pickling of methodcaller, attrgetter, and itemgetter

Zachary Ware added the comment: Serhiy Storchaka added the comment: Reimplementations of the pure-Python itemgetter and attrgetter to automatically pickleable Python classes have a disadvantage. It makes the pickling incompatible between Python and C versions. This means that itemgetter

[issue22955] Pickling of methodcaller, attrgetter, and itemgetter

Zachary Ware added the comment: Serhiy: functools.partial is a somewhat less than ideal comparison. The pure-Python version is not picklable, the Python and C versions return different things (the Python version is a function returning a function, the C version is a regular class and returns

[issue22955] Pickling of methodcaller, attrgetter, and itemgetter

Raymond Hettinger added the comment: I'd prefer to just reimplement itemgetter and attrgetter to make them picklable rather than adding pickling methods to them; see attached patch. That isn't the usual approach. The pickling methods are there for a reason. I prefer to leave the existing

[issue22955] Pickling of methodcaller, attrgetter, and itemgetter

Raymond Hettinger added the comment: Please remember that a potential new pickling feature is the least import part of the design of methodcaller, itemgetter, and attrgetter. Pickle support should be driven by the design rather become a predominant consideration. One other note: the OP's

[issue22955] Pickling of methodcaller, attrgetter, and itemgetter

Serhiy Storchaka added the comment: Instead, it is the picking and unpickling tools themselves that tend to have crummy error messages when presented with objects that weren't specially designed with pickle support. See issue22995 about this. --

[issue22955] Pickling of methodcaller, attrgetter, and itemgetter

Zachary Ware added the comment: I'd prefer to just reimplement itemgetter and attrgetter to make them picklable rather than adding pickling methods to them; see attached patch. I also posted a few comments, but I just went ahead and addressed them myself in this patch. I'm not qualified to

[issue22955] Pickling of methodcaller, attrgetter, and itemgetter

Changes by Zachary Ware zachary.w...@gmail.com: Removed file: http://bugs.python.org/file37319/issue22955.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22955 ___

[issue22955] Pickling of methodcaller, attrgetter, and itemgetter

Changes by Zachary Ware zachary.w...@gmail.com: Added file: http://bugs.python.org/file37320/issue22955.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22955 ___

[issue22955] Pickling of methodcaller, attrgetter, and itemgetter

Serhiy Storchaka added the comment: operator.methodcaller is similar to functools.partial which is pickleable and can be used as a sample. In C implementation some code can be shared between __repr__ and __reduce__ methods. As for tests, different protocols should be tested. Also should be

[issue22955] Pickling of methodcaller, attrgetter, and itemgetter

Zachary Ware added the comment: Note that pickling of the pure Python version of methodcaller works as expected: Python 3.4.2 (default, Nov 20 2014, 12:40:10) [GCC 4.8.3] on linux Type help, copyright, credits or license for more information. import sys sys.modules['_operator'] = None

[issue22955] Pickling of methodcaller, attrgetter, and itemgetter

Raymond Hettinger added the comment: +1 for adding pickling support to Python 3.5. I don't see much of a need for any revision to 3.4. -- nosy: +rhettinger versions: -Python 3.4 ___ Python tracker rep...@bugs.python.org

[issue22955] Pickling of methodcaller, attrgetter, and itemgetter

Josh Rosenberg added the comment: I've made a patch that I believe should cover all three cases, including tests. In addition to the pickling behavior, I've made two other changes: 1. methodcaller verifies during construction that the name is a string (PyUnicode), and interns it; attrgetter

[issue22955] Pickling of methodcaller, attrgetter, and itemgetter

Changes by Josh Rosenberg shadowranger+pyt...@gmail.com: -- versions: -Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22955 ___ ___

[issue22955] Pickling of methodcaller, attrgetter, and itemgetter

Josh Rosenberg added the comment: Don't bother reviewing just yet. There is an issue with attrgetter's pickling (which the unit tests caught), and I need to update the pure Python modules to match. -- ___ Python tracker rep...@bugs.python.org

[issue22955] Pickling of methodcaller, attrgetter, and itemgetter

Josh Rosenberg added the comment: Okay, this one passes the tests for the built-in module. I'm not sure what's going wrong with the pure Python module. I'm getting the error: _pickle.PicklingError: Can't pickle class 'operator.attrgetter': it's not the same object as operator.attrgetter

[issue22955] Pickling of methodcaller, attrgetter, and itemgetter

Changes by Josh Rosenberg shadowranger+pyt...@gmail.com: Removed file: http://bugs.python.org/file37313/pickle_getter_and_caller.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22955 ___

[issue22955] Pickling of methodcaller, attrgetter, and itemgetter

Josh Rosenberg added the comment: Ah, solved it (I think). The bootstrapper used to import the Python and C versions of the module leaves sys.modules unpopulated (Does pickle itself may populate it when it finds no module of that name?). I added a setUp method to the unittest class for