[issue22783] Pickle: use NEWOBJ instead of NEWOBJ_EX if possible

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Updated patch addresses Antoine's comments. Also added few comments explaining 
binary data.

--
Added file: http://bugs.python.org/file37470/pickle_newobj_2.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22783
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22783] Pickle: use NEWOBJ instead of NEWOBJ_EX if possible

[Roundup Robot]

Roundup Robot added the comment:

New changeset 2ffaac4c8e53 by Serhiy Storchaka in branch 'default':
Issue #22783: Pickling now uses the NEWOBJ opcode instead of the NEWOBJ_EX
https://hg.python.org/cpython/rev/2ffaac4c8e53

--
nosy: +python-dev

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22783
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22783] Pickle: use NEWOBJ instead of NEWOBJ_EX if possible

[Serhiy Storchaka]

Changes by Serhiy Storchaka storch...@gmail.com:


--
assignee:  - serhiy.storchaka
resolution:  - fixed
stage: patch review - resolved
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22783
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22783] Pickle: use NEWOBJ instead of NEWOBJ_EX if possible

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Ping.

--
keywords: +needs review

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22783
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22783] Pickle: use NEWOBJ instead of NEWOBJ_EX if possible

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Some examples (with issue19858 for protocol 4 optimization).

Unpatched:

 len(pickle.dumps([P(12, 34) for i in range(1000)], 3))
17258
 len(pickletools.optimize(pickle.dumps([P(12, 34) for i in range(1000)], 3)))
8018
 len(pickle.dumps([P(i, -i) for i in range(1000)], 3))
20999
 len(pickletools.optimize(pickle.dumps([P(i, -i) for i in range(1000)], 3)))
11759

 len(pickle.dumps([P(12, 34) for i in range(1000)], 4))
12031
 len(pickletools.optimize(pickle.dumps([P(12, 34) for i in range(1000)], 4)))
9028
 len(pickle.dumps([P(i, -i) for i in range(1000)], 4))
15772
 len(pickletools.optimize(pickle.dumps([P(i, -i) for i in range(1000)], 4)))
12769

Patched:

 len(pickle.dumps([P(12, 34) for i in range(1000)], 4))
10031
 len(pickletools.optimize(pickle.dumps([P(12, 34) for i in range(1000)], 4)))
8028
 len(pickle.dumps([P(i, -i) for i in range(1000)], 4))
13772
 len(pickletools.optimize(pickle.dumps([P(i, -i) for i in range(1000)], 4)))
11769

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22783
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22783] Pickle: use NEWOBJ instead of NEWOBJ_EX if possible

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

Currently in pickle with protocol 4 the NEWOBJ_EX opcode is used to reconstruct 
objects with __getnewargs__() or __getnewargs_ex__() even if there no keyword 
arguments.

 import pickle, pickletools, collections
 P = collections.namedtuple('P', 'x y')
 pickletools.dis(pickle.dumps(P(12, 34), 3))
0: \x80 PROTO  3
2: cGLOBAL '__main__ P'
   14: qBINPUT 0
   16: KBININT112
   18: KBININT134
   20: \x86 TUPLE2
   21: qBINPUT 1
   23: \x81 NEWOBJ
   24: qBINPUT 2
   26: .STOP
highest protocol among opcodes = 2
 pickletools.dis(pickle.dumps(P(12, 34), 4))
0: \x80 PROTO  4
2: \x95 FRAME  28
   11: \x8c SHORT_BINUNICODE '__main__'
   21: \x94 MEMOIZE
   22: \x8c SHORT_BINUNICODE 'P'
   25: \x94 MEMOIZE
   26: \x93 STACK_GLOBAL
   27: \x94 MEMOIZE
   28: KBININT112
   30: KBININT134
   32: \x86 TUPLE2
   33: \x94 MEMOIZE
   34: }EMPTY_DICT
   35: \x94 MEMOIZE
   36: \x92 NEWOBJ_EX
   37: \x94 MEMOIZE
   38: .STOP
highest protocol among opcodes = 4

These EMPTY_DICT//MEMOIZE//NEWOBJ_EX can be replaced by single NEWOBJ. Actually 
there is a regression in protocol 4 against protocol 3, so may be following 
patch should be applied on 3.4 too.

Here is a patch which uses __newobj__ instead of __newobj_ex__ if there are no 
keywords arguments. Also it reduce the number of lines of C code and adds tests 
for classes with __getnewargs__() and __getnewargs_ex__().

--
components: Library (Lib)
files: pickle_newobj.patch
keywords: patch
messages: 230490
nosy: alexandre.vassalotti, pitrou, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Pickle: use NEWOBJ instead of NEWOBJ_EX if possible
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file37107/pickle_newobj.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22783
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com