This is a neat idea, but this will only work for parsing framed
binary protocols. For example, if you protocol prefixes all packets
with a length field, you can write an efficient read buffer and
use your proposal to decode all of message's fields in one shot.
Which is good.
Not all protocols use framing though. For instance, your proposal
won't help to write Thrift or Postgres protocols parsers.
Overall, I'm not sure that this is worth the hassle. With proposal:
data, = struct.unpack('!H$', buf)
buf = buf[2+len(data):]
with the current struct module:
len, = struct.unpack('!H', buf)
data = buf[2:2+len]
buf = buf[2+len:]
Another thing: struct.calcsize won't work with structs that use
variable length fields.
Yury
On 2017-01-18 5:24 AM, Elizabeth Myers wrote:
Hello,
I've noticed a lot of binary protocols require variable length
bytestrings (with or without a null terminator), but it is not easy to
unpack these in Python without first reading the desired length, or
reading bytes until a null terminator is reached.
I've noticed the netstruct library
(https://github.com/stendec/netstruct) has a format specifier, $, which
assumes the previous type to pack/unpack is the string's length. This is
an interesting idea in of itself, but doesn't handle the null-terminated
string chase. I know $ is similar to pascal strings, but sometimes you
need more than 255 characters :p.
For null-terminated strings, it may be simpler to have a specifier for
those. I propose 0, but this point can be bikeshedded over endlessly if
desired ;) (I thought about using n/N but they're :P).
It's worth noting that (maybe one of?) Perl's equivalent to the struct
module, whose name escapes me atm, has a module which can handle this
case. I can't remember if it handled variable length or zero-terminated
though; maybe it did both. Perl is more or less my 10th language. :p
This pain point is an annoyance imo and would greatly simplify a lot of
code if implemented, or something like it. I'd be happy to take a look
at implementing it if the idea is received sufficiently warmly.
--
Elizabeth
_______________________________________________
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
