On Mon, 18 Nov 2013 18:50:01 -0600 Tim Peters <tim.pet...@gmail.com> wrote: > [Antoine] > > Yet another possibility: keep framing but use a variable-length > > encoding for the frame size: > > > > - first byte: bits 7-5: N (= frame size bytes length - 1) > > - first byte: bits 4-0: first 5 bits of frame size > > - remaning N bytes: remaining bits of frame size > > > > With this scheme, very small pickles have a one byte overhead; small > > ones a two byte overhead; and the max frame size is 2**61 rather than > > 2**64, which should still be sufficient. > > > > And the frame size is read using either one or two read() calls, which > > is efficient. > > That would be a happy compromise :-) > > I'm unclear on how that would work for, e.g., encoding 40 = > 0b000101000. That has 6 significant bits. Would you store 0 in the > leading byte and 40 in the second byte? That would work.
Yeah, I haven't decided whether it would be big-endian or little-endian. It doesn't matter much. big-endian sounds a bit easier to decode and encode (no bit shifts needed), it's also less consistent with the rest of the pickle opcodes. > 2**61 = 2,305,843,009,213,693,952 is a lot of bytes, especially for a pickle > ;-) Let's call it a biggle :-) Regards Antoine. _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
