New submission from Serhiy Storchaka <storchaka+cpyt...@gmail.com>: Pickle protocol 4 uses framing for reducing the overhead of calling the read() method for small chunks of data. Most read chunks are small -- opcodes, small integers, short strings, etc, and calling read() for every 1 or 4 bytes is too expensive. But using framing itself adds an overhead. It increases the size of pickled data by 9 bytes. A frame itself needs 3 reads -- the opcode, the frame size, and a payload. Thus it doesn't make sense to create a frame containing less than 3 chunks of data.
For example after issue31993 pickling the list [b'a'*70000, b'b'*70000] with the Python implementation produces a data containing 3 frames of sizes 3, 1 and 3. Using frames here is completely redundant. ---------- components: Library (Lib) messages: 309564 nosy: Olivier.Grisel, alexandre.vassalotti, pitrou, serhiy.storchaka priority: normal severity: normal status: open title: Avoid creating small frames in pickle protocol 4 type: performance versions: Python 3.7 _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32503> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
