New submission from Oleg Broytman: Hello! I've found a problematic behaviour of pickle when it pickles (direct or indirect) weak proxies to self. I suspect pickle cannot detect reference loops with weak proxies. I managed to narrow the case to the following example:
import pickle, weakref class TestPickle(object): def __init__(self): self.recursive = weakref.proxy(self) def __getstate__(self): print "__getstate__", id(self) return self.__dict__.copy() def __setstate__(self, d): print "__setstate__", id(self) self.__dict__.update(d) print "- 1 -" test = TestPickle() print "- 2 -" data = pickle.dumps(test, pickle.HIGHEST_PROTOCOL) print "- 3 -" test2 = pickle.loads(data) print "- 4 -" print "Result:", id(test2) print "- 5 -" It prints: - 1 - - 2 - __getstate__ 3075348620 __getstate__ 3075348620 - 3 - __setstate__ 3075348844 __setstate__ 3075349004 - 4 - Result: 3075349004 - 5 - That is, __getstate__ is called twice for the same object. And what is worse, __setstate__ is called twice for different objects. The resulting unpickled object is the last one, but in the library that I have been debugging creation of two different objects during unpickling is a bug. I can fix it by avoiding pickling the proxy and recreating the proxy on unpickling: import pickle, weakref class TestPickle(object): def __init__(self): self.recursive = weakref.proxy(self) def __getstate__(self): print "__getstate__", id(self) d = self.__dict__.copy() del d['recursive'] return d def __setstate__(self, d): print "__setstate__", id(self) self.__dict__.update(d) self.recursive = weakref.proxy(self) print "- 1 -" test = TestPickle() print "- 2 -" data = pickle.dumps(test, pickle.HIGHEST_PROTOCOL) print "- 3 -" test2 = pickle.loads(data) print "- 4 -" print "Result:", id(test2) print "- 5 -" - 1 - - 2 - __getstate__ 3075070092 - 3 - __setstate__ 3075070188 - 4 - Result: 3075070188 - 5 - But I wonder if it's a bug that should be fixed? If it's an expected behaviour it perhaps should be documented as a warning in docs for pickle or weakref or both. ---------- components: Library (Lib) messages: 190105 nosy: phd priority: normal severity: normal status: open title: pickle + weakref.proxy(self) type: behavior versions: Python 2.6, Python 2.7 _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue18068> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com