Serhiy Storchaka added the comment:
Here is a patch which addresses Antoine's comments. Also added dis() output for
binary example in comments.
--
Added file: http://bugs.python.org/file37469/pickle_optimize_memoize_2.patch
___
Python tracker
Roundup Robot added the comment:
New changeset c49b7acba06f by Serhiy Storchaka in branch '3.4':
Issue #19858: pickletools.optimize() now aware of the MEMOIZE opcode, can
https://hg.python.org/cpython/rev/c49b7acba06f
New changeset e7dd739b4b4e by Serhiy Storchaka in branch 'default':
Issue
Serhiy Storchaka added the comment:
Many thanks for all your reviews Antoine!
--
assignee: - serhiy.storchaka
resolution: - fixed
stage: patch review - resolved
status: open - closed
___
Python tracker rep...@bugs.python.org
Serhiy Storchaka added the comment:
Ping.
--
keywords: +needs review
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19858
___
___
Python-bugs-list
Serhiy Storchaka added the comment:
Here is a patch which makes pickletools.optimize() to work with the MEMOIZE
opcode.
As side effect it also renumbers memoized values, this allows to use short
BINPUT and BINGET instead of LONG_BINPUT and LONG_BINGET.
--
keywords: +patch
stage:
Alexandre Vassalotti added the comment:
Ah, I almost forgot! I did implement the verification in pickletools.dis() for
MEMOIZE:
http://hg.python.org/cpython/file/2612ea573ff7/Lib/pickletools.py#l2420
--
___
Python tracker rep...@bugs.python.org
Serhiy Storchaka added the comment:
I afraid this is not a feature, but a bug. pickletools.optimize() can produce
invalid pickled data when *PUT and MEMOIZE opcodes are used.
import pickle, pickletools
p = b'\x80\x04]\x94(\x8c\x04spamq\x01\x8c\x03ham\x94h\x02e.'
pickle.loads(p)
['spam',
Alexandre Vassalotti added the comment:
Well, that can only happen if MEMOIZE and PUT are both used together, which
won't happen with the Pickler classes we support. The easiest thing to do here
is to disable pickletools.optimize on protocol 4.
--
Changes by Antoine Pitrou pit...@free.fr:
--
nosy: +tim.peters
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19858
___
___
Python-bugs-list
Antoine Pitrou added the comment:
By the way, I was curious if there were many users of pickletools.optimize(),
so I did a search and most matches seem to be copies of the stdlib source tree:
Tim Peters added the comment:
Is it _documented_ that MEMOIZE and PUT can't be used together? If not, it
should be documented; and pickletools dis() and optimize() should verify that
this restriction is honored in their inputs.
--
___
Python
Alexandre Vassalotti added the comment:
MEMOIZE and PUT can be used together. They just need to not step on each other
toes when they write to the memo table. As specified by PEP 3154, the memo
index used by MEMOIZE is the number of elements currently in the memo table.
This obviously means
New submission from Alexandre Vassalotti:
PEP 3154 introduced the MEMOIZE opcode which lowered the overhead of
memoization compared to the PUT opcodes which were previously used.
We should update pickletools.optimize to remove superfluous uses of this new
opcode.
--
components:
13 matches
Mail list logo
