[issue7689] Pickling of classes with a metaclass and copy_reg
Antoine Pitrou pit...@free.fr added the comment: +def __eq__(self, other): +r = (type(self) == type(other)) +if r: +return r I think this should be if not r. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7689 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7689] Pickling of classes with a metaclass and copy_reg
Craig Citro craigci...@gmail.com added the comment: Antoine -- why do you want to switch if r for if not r? If we did, the test would just confirm that the unpicked object was of the same type as the original; if we were going to do that, we might as well just replace the whole `__cmp__` function with just `return cmp(type(self), type(other))`. On the flipside, I could see an argument for adding *more* content to the test, but that seemed like overkill. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7689 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7689] Pickling of classes with a metaclass and copy_reg
Antoine Pitrou pit...@free.fr added the comment: Craig: I'm talking about the __eq__ version (durban's patch). The __cmp__ version is probably fine. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7689 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7689] Pickling of classes with a metaclass and copy_reg
Craig Citro craigci...@gmail.com added the comment: Antoine -- ah, that makes sense. Is that the only blocker? I've let this patch rot on the vine a long time; if so, I'll happily switch `__eq__` back to `__cmp__` and re-post if it'll get submitted. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7689 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7689] Pickling of classes with a metaclass and copy_reg
Antoine Pitrou pit...@free.fr added the comment: No need, I'll do it myself. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7689 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7689] Pickling of classes with a metaclass and copy_reg
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset 760ac320fa3d by Antoine Pitrou in branch '3.2': Issue #7689: Allow pickling of dynamically created classes when their http://hg.python.org/cpython/rev/760ac320fa3d New changeset 46c026a5ccb9 by Antoine Pitrou in branch 'default': Issue #7689: Allow pickling of dynamically created classes when their http://hg.python.org/cpython/rev/46c026a5ccb9 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7689 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7689] Pickling of classes with a metaclass and copy_reg
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset 64053bd79590 by Antoine Pitrou in branch '2.7': Issue #7689: Allow pickling of dynamically created classes when their http://hg.python.org/cpython/rev/64053bd79590 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7689 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7689] Pickling of classes with a metaclass and copy_reg
Antoine Pitrou pit...@free.fr added the comment: This is fixed now, thank you! -- resolution: - fixed stage: patch review - committed/rejected status: open - closed versions: -Python 3.1 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7689 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7689] Pickling of classes with a metaclass and copy_reg
Brent Payne brent.pa...@gmail.com added the comment: will the 2.7 patch also be incorporated into a 2.7 release? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7689 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7689] Pickling of classes with a metaclass and copy_reg
Antoine Pitrou pit...@free.fr added the comment: Yes, it will. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7689 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7689] Pickling of classes with a metaclass and copy_reg
Nick Coghlan ncogh...@gmail.com added the comment: Specifically, 2.7.3. A date for that has not yet been set, but somewhere in the December/January time frame is likely. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7689 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7689] Pickling of classes with a metaclass and copy_reg
Changes by Brent Payne brent.pa...@gmail.com: -- nosy: +Brent.Payne ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7689 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7689] Pickling of classes with a metaclass and copy_reg
Lance Hepler nlhep...@gmail.com added the comment: Hello all, sorry to be a bother, but what's the progress on this issue? I have a codebase that requires resolution of this issue to enable multiprocessing. What are the remaining outstanding problems herein preventing the attached patches from being merged? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7689 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7689] Pickling of classes with a metaclass and copy_reg
Changes by Lance Hepler nlhep...@gmail.com: -- nosy: +nlhepler ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7689 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7689] Pickling of classes with a metaclass and copy_reg
Changes by Antoine Pitrou pit...@free.fr: -- nosy: +ncoghlan ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7689 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7689] Pickling of classes with a metaclass and copy_reg
Daniel Urban urban.dani...@gmail.com added the comment: Attaching an updated patch for py3k. Not an expert, but the Python parts of your patch look good to me. Me neither, but the C parts also look good to me. The tests fail without the patch, succeed with it. Note, that it is possible, that the copy module also should be fixed similarly (but currently that can't even copy non-dynamic classes with a metaclass, see issue11480). -- components: +Extension Modules versions: +Python 3.3 Added file: http://bugs.python.org/file21101/issue7689.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7689 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7689] Pickling of classes with a metaclass and copy_reg
Changes by François Bissey francois.bis...@canterbury.ac.nz: -- nosy: +fbissey ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7689 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7689] Pickling of classes with a metaclass and copy_reg
Changes by Daniel Urban urban.dani...@gmail.com: -- nosy: +durban ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7689 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7689] Pickling of classes with a metaclass and copy_reg
Éric Araujo mer...@netwok.org added the comment: I was mistaken: bug fixes go in stable releases too. -- versions: +Python 2.7, Python 3.1 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7689 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7689] Pickling of classes with a metaclass and copy_reg
Éric Araujo mer...@netwok.org added the comment: Not an expert, but the Python parts of your patch look good to me. -- nosy: +pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7689 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7689] Pickling of classes with a metaclass and copy_reg
Gerald Dalley dall...@deshaw.com added the comment: Another use case: for distributed processing, it's handy to be able to pickle interactive functions and functions that are part of a script. The user can then remotely execute a broader set of functions than can be pickled by default. This is especially helpful for interactive exploration of data. Pickling most types of functions is possible by serializing a function's bytecode, etc. and registering a handler with copy_reg. Unfortunately, this handler is ignored under some conditions (e.g. copy_reg is ignored when attempting to serialize top-level functions in a script) but the copy_reg handler does get used for lambdas and inner functions. This patch looks like it will allow FunctionType's pickle handler to be overridden in all cases. -- nosy: +dalleyg ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7689 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7689] Pickling of classes with a metaclass and copy_reg
Éric Araujo mer...@netwok.org added the comment: Thanks for your report and patch. I’m editing the versions field: New features and bug fixes go to the active branch, py3k (future 3.2), while only security and documentation fixes are allowed to go in stable branches (2.6 and 3.1). Current trunk (2.7) is frozen, since it’s in release candidate stage; if your bug is fixed in py3k really soon, we’ll ask the release manager Benjamin Peterson if he agrees to let it into 2.7. Thanks again! -- keywords: +needs review nosy: +merwok stage: - patch review versions: -Python 2.5, Python 2.6, Python 2.7, Python 3.1 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7689 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7689] Pickling of classes with a metaclass and copy_reg
Changes by Bryan Silverthorn bc...@cornell.edu: -- nosy: +bsilverthorn ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7689 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7689] Pickling of classes with a metaclass and copy_reg
Changes by Antoine Pitrou pit...@free.fr: -- nosy: +alexandre.vassalotti ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7689 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7689] Pickling of classes with a metaclass and copy_reg
New submission from Craig Citro craigci...@gmail.com: Currently, it's impossible to use the usual pickle mechanisms to pickle a dynamically created class, even if the user requests a different pickling mechanism via copy_reg. The attached patch makes this customization possible by simply transposing two blocks of code. Longer explanation: Classes are pickled by name, which is of course problematic when trying to pickle a dynamically created class. The natural solution would be to create a __reduce__ method on the metaclass. However, as mentioned in issue 494904, this won't work -- the issue is that for any class C with a metaclass M, C.__reduce__ is the unbound method for instances of class C, as opposed to the bound method M.__reduce__ for C (i.e. viewing C as an instance of M). Guido's patch on that ticket does the sensible thing, which is force the class to be pickled by name -- this is fine, until you want to pickle a class that's created at runtime. The copy_reg module exists to handle custom pickling of objects, which is exactly what's needed here. However, the code from #494904 that checks for instances of a metaclass does this *just before* it looks at the copy_reg dispatch table. The patches just reorder these tests in pickle.py and cPickle.c, so that one can register a custom pickler for instances of a metaclass. Comments: First, let me preemptively say that I do indeed have a use case for this fix. We ran into this bug working on Sage (http://www.sagemath.org), where we create lots of dynamic classes at runtime (to model mathematical relationships between different kinds of objects). There's a healthy mix of dynamic and non-dynamic classes floating around, and we want the user to be able to pickle them without having to know anything about how the class was created. We could create our own pickling function, but we'd much rather just use the default Python mechanisms, especially since it's such a small fix. Second, this patch is fairly safe, in the sense that it's pretty unlikely it can break any existing code. The only way it could break is if someone created a dynamic class, registered a pickler for it with copy_reg, and was depending on calls to pickle to fail. Third, there's a few extra lines of code in the chunk the patch moves around coming from ticket #502085. These are there to deal with versions of Boost circa 2002. Is it worth removing these? The attached patch is against trunk (r77421). At least as I write this, the same patch should apply against the py3k branch (up to renaming copy_reg to copyreg, anyway), but I'm happy to do the legwork of rebasing it as needed. Also, I'd happily review something else in exchange for a review of this. We'll be keeping this patch around in Sage until it gets merged, so the sooner the better. Authors: Nicolas Thiery (nthi...@users.sf.net) first hunted this down and wrote a patch for cPickle.c. I did the (fairly trivial) work of mirroring the fix in pickle.py, and added the tests in pickletester. -- components: Library (Lib) files: dynamic_class_copyreg.patch keywords: patch messages: 97702 nosy: craigcitro severity: normal status: open title: Pickling of classes with a metaclass and copy_reg type: behavior versions: Python 2.5, Python 2.6, Python 2.7, Python 3.1, Python 3.2 Added file: http://bugs.python.org/file15853/dynamic_class_copyreg.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7689 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
