[issue32503] Avoid creating small frames in pickle protocol 4

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32503] Avoid creating small frames in pickle protocol 4

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 1211c9a9897a174b7261ca258cabf289815a40d8 by Serhiy Storchaka in 
branch 'master':
bpo-32503: Avoid creating too small frames in pickles. (#5127)
https://github.com/python/cpython/commit/1211c9a9897a174b7261ca258cabf289815a40d8


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32503] Avoid creating small frames in pickle protocol 4

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

In the current form the change is trivial, just an additional check. Actually 
it fixes a regression introduced in issue31993. Currently even empty frames can 
be produced (when fast=True).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32503] Avoid creating small frames in pickle protocol 4

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I don't think the overall gain is meaningful.  I'd rather not add too many 
special cases in the framing code.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32503] Avoid creating small frames in pickle protocol 4

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

PR 5127 makes frames be created only when the size of the payload is not less 
than 4. Since the minimal size of 3 chunks is 3 bytes this is the absolute 
minimum of frame size.

It would be better to count the number of chunks instead of bytes, but this 
will complicate implementations, especially Python implementation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32503] Avoid creating small frames in pickle protocol 4

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +4988
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32503] Avoid creating small frames in pickle protocol 4

[Serhiy Storchaka]

New submission from Serhiy Storchaka :

Pickle protocol 4 uses framing for reducing the overhead of calling the read() 
method for small chunks of data. Most read chunks are small -- opcodes, small 
integers, short strings, etc, and calling read() for every 1 or 4 bytes is too 
expensive. But using framing itself adds an overhead. It increases the size of 
pickled data by 9 bytes. A frame  itself needs 3 reads -- the opcode, the frame 
size, and a payload. Thus it doesn't make sense to create a frame containing 
less than 3 chunks of data.

For example after issue31993 pickling the list [b'a'*7, b'b'*7] with 
the Python implementation produces a data containing 3 frames of sizes 3, 1 and 
3. Using frames here is completely redundant.

--
components: Library (Lib)
messages: 309564
nosy: Olivier.Grisel, alexandre.vassalotti, pitrou, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Avoid creating small frames in pickle protocol 4
type: performance
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com