[Tim]
>> But I wonder why it isn't done with a new framing opcode instead (say,
>> FRAME followed by 8-byte count). I suppose that would be like the
>> "prefetch" idea, except that framing opcodes would be mandatory
>> (instead of optional) in proto 4. Why I initially like that:
>>
>> - Uniform decoding loop ("the next thing" _always_ starts with an opcode).
> But it's not actually uniform. A frame isn't a normal opcode, it's a
> large section of bytes that contains potentially many opcodes.
>
> The framing layer is really below the opcode layer, it makes also sense
> to implement it like that.
That makes sense to me.
> (I also tried to implement Serhiy's PREFETCH idea, but it didn't bring
> any actual simplification)
But it has a different kind of advantage: PREFETCH was optional. As
Guido said, it's annoying to bloat the size of small pickles (which
may, although individually small, occur in great numbers) by 8 bytes
each. There's really no point to framing small chunks of data, right?
Which leads to another idea: after the PROTO opcode, there is, or is
not, an optional PREFETCH opcde with an 8-byte argument. If the
PREFETCH opcode exists, then it gives the number of bytes up to and
including the pickle's STOP opcode. So there's exactly 0 or 1
PREFETCH opcodes per pickle.
Is there an advantage to spraying multiple 8-byte "frame counts"
throughout a pickle stream? 8 bytes is surely enough to specify the
size of any single pickle for half a generation ;-) to come.
>> When slinging 8-byte counts, _some_ sanity-checking seems like a good idea
>> ;-)
> I don't know. It's not much worse (for denial of service opportunities)
> than a 4-byte count, which already exists in earlier protocols.
I'm not thinking of DOS at all, just general sanity as data objects
get larger & larger. Pickles have almost no internal checks now. But
I've seen my share of corrupted pickles! About the only thing that
catches them early is hitting a byte that isn't a legitimate pickle
opcode. That _used_ to be a much stronger check than it is now,
because the 8-bit opcode space was sparsely populated at first. But,
over time, more and more opcodes get added, so the chance of mistaking
a garbage byte for a legit opcode has increased correspondingly.
A PREFETCH opcode with a "bytes until STOP" makes for a decent bad ;-)
sanity check too ;-)
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
