New submission from Thomas Miedema:
Pickling a namedtuple Point(x=10, y=20, z=30) in Python 2.7.2 with protocol
level 0 would result in something like the following output:
ccopy_reg
_reconstructor
p0
(c__main__
Point
p1
c__builtin__
tuple
p2
(I10
I20
I30
tp3
tp4
Rp5
.
In Python 2.7.3, the same namedtuple dumps to:
ccopy_reg
_reconstructor
p0
(c__main__
Point
p1
c__builtin__
tuple
p2
(I10
I20
I30
tp3
tp4
Rp5
ccollections
OrderedDict
p6
((lp7
(lp8
S'x'
p9
aI10
aa(lp10
S'y'
p11
aI20
aa(lp12
S'z'
p13
aI30
aatp14
Rp15
b.
Note the OrderedDictionary at the end. All data, the field names and the
values, are duplicated, which can result in very large pickled files when using
nested namedtuples.
Loading both dumps with CPython 2.7.3 works. This is why this bug was not
noticed any earlier. Loading the second dump with CPython or pypy 2.7.2 does
not work however. CPython 2.7.3 broke forward compatibility.
Attached is a patch with a fix. The patch makes pickled namedtuples forward
compatibile with 2.7.2. This patch does not break backward compability with
2.7.3, since the extra OrderedDict data contained the same information as the
tuple.
Introduced:
http://hg.python.org/cpython/diff/26d5f022eb1a/Lib/collections.py
Also relevant:
http://bugs.python.org/issue3065
--
components: Library (Lib)
files: namedtuple_pickle_fix.patch
keywords: patch
messages: 167215
nosy: rhettinger, thomie
priority: normal
severity: normal
status: open
title: Fix pickling of named tuples in 2.7.3
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file26662/namedtuple_pickle_fix.patch
___
Python tracker
<http://bugs.python.org/issue15535>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com