Ugh, gmail screwed up my paste: https://gist.github.com/michelp/8482432
-Michel On Fri, Jan 17, 2014 at 1:51 PM, Michel Pelletier < [email protected]> wrote: > It wasn't the recv that was truncating the string, it was the send. > > For string that contain null chars, you'll have to use zframe send/recv. > Here's your example working with those functions: > > File Edit Options Buffers Tools Python Help > > from pyczmq import zmq, zctx, zsocket, zstr, zframe > > ctx = zctx.new() > rep = zsocket.new(ctx, zmq.REP) > zsocket.bind(rep, "tcp://127.0.0.1:5253") > > req = zsocket.new(ctx, zmq.REQ) > zsocket.connect(req, "tcp://127.0.0.1:5253") > > out_data = "blah blah hey what is \0 doing here?" > out_frame = zframe.new(out_data) > zframe.send(out_frame, req, 0) > > in_frame = zframe.recv(rep) > in_data = bytearray(zframe.data(in_frame)) > assert in_data == out_data, "expected %r, got %r" % (out_data, in_data) > > > > > > > > > > > > > On Fri, Jan 17, 2014 at 1:20 PM, Michel Pelletier < > [email protected]> wrote: > >> Ah you've got a good catch there, browsing the docs cffi does support >> variadic function calls >> https://cffi.readthedocs.org/en/release-0.8/#variadic-function-calls >> >> I'll study this after work tonight, I'm sure we can address the issue. >> >> As for the null char in the middle of a received string, I'll have to >> think about that one. I'm open to suggestions. >> >> -Michel >> >> >> On Fri, Jan 17, 2014 at 12:55 PM, Greg Ward <[email protected]> wrote: >> >>> On 17 January 2014, Michel Pelletier said: >>> > zstr however is how you turn messages into strings. It is not an >>> > implementation of a string type. It is critical to the functionality >>> of >>> > pyczmq (assuming you want string output from the library). >>> >>> I was going to start another thread about this... but what the hell, >>> as long as it came up, here are the two problems I've spotted with >>> pyczmq.zstr.send(). >>> >>> 1. zstr_send() is printf()-style, pyczmq.zstr.send() is not >>> ----------------------------------------------------------- >>> >>> The C API for zstr_send() is >>> >>> zstr_send(void *zocket, const char *format, ...) >>> >>> which means calling it like >>> >>> char *data = [...get a string from somewhere...] >>> zstr_send(mysocket, data); >>> >>> is wrong, because 'data' might happen to contain a "%s" or "%d" by >>> chance. Presumably the innards of zstr_send() will look for further >>> args on the stack, find gibberish, and send gibberish over the wire. >>> Or crash. The right way to do this in C is >>> >>> zstr_send(mysocket, "%s", data); >>> >>> But the corresponding Python function is >>> >>> def send(sock, string): >>> return C.zstr_send(sock, string) >>> >>> so if 'string' happens to contain "%s" or "%d", bad stuff happens. But >>> since pyczmq.zstr.send() doesn't expose a format string, there's no >>> way to do the right thing. >>> >>> Here's an example: >>> >>> $ cat zstr-bug1.py >>> from pyczmq import zmq, zctx, zsocket, zstr >>> >>> ctx = zctx.new() >>> rep = zsocket.new(ctx, zmq.REP) >>> zsocket.bind(rep, "tcp://127.0.0.1:5253") >>> >>> req = zsocket.new(ctx, zmq.REQ) >>> zsocket.connect(req, "tcp://127.0.0.1:5253") >>> >>> out_data = "blah blah hey what is %s doing here?" >>> zstr.send(req, out_data) >>> >>> in_data = zstr.recv(rep) >>> assert in_data == out_data, "expected %r, got %r" % (out_data, in_data) >>> >>> $ python zstr-bug1.py >>> zsh: segmentation fault (core dumped) python zstr-bug1.py >>> >>> Here are the innermost stack frames from that segfault: >>> >>> """ >>> #0 0x00007f3c06a4bf90 in _IO_vfprintf_internal (s=s@entry >>> =0x7fffd8435470, >>> format=<optimized out>, >>> format@entry=0x7f3c07124fa4 "blah blah hey what is %s doing here?", >>> ap=ap@entry=0x7fffd84355e8) at vfprintf.c:1655 >>> #1 0x00007f3c06b0f6c0 in ___vsnprintf_chk ( >>> s=s@entry=0x1120750 "blah blah hey what is ", maxlen=<optimized >>> out>, >>> maxlen@entry=256, flags=flags@entry=1, slen=slen@entry=256, >>> format=format@entry=0x7f3c07124fa4 "blah blah hey what is %s doing >>> here?", >>> args=args@entry=0x7fffd84355e8) at vsnprintf_chk.c:63 >>> #2 0x00007f3c04af0d1b in vsnprintf (__ap=0x7fffd84355e8, >>> __fmt=0x7f3c07124fa4 "blah blah hey what is %s doing here?", __n=256, >>> __s=0x1120750 "blah blah hey what is ") >>> at /usr/include/x86_64-linux-gnu/bits/stdio2.h:77 >>> #3 zsys_vprintf ( >>> format=0x7f3c07124fa4 "blah blah hey what is %s doing here?", >>> argptr=argptr@entry=0x7fffd8435638) at zsys.c:386 >>> #4 0x00007f3c04af0499 in zstr_send (zocket=0x11c2d30, format=<optimized >>> out>) >>> at zstr.c:112 >>> #5 0x00007f3c05521adc in ffi_call_unix64 () >>> from /usr/lib/x86_64-linux-gnu/libffi.so.6 >>> """ >>> >>> 2. zstr_send() is not binary safe >>> --------------------------------- >>> >>> The fact that CZMQ's zstr_send() is not binary safe is just fine: it >>> says right in the docs that it's a convenience for sending C strings >>> around. So the fact that pyczmq.zstr.send() is not binary safe is not >>> surprising (although it is annoying -- I'm used to Python strings, not >>> C strings). But since you say it's "critical to the functionality of >>> pyczmq", maybe I should worry about it. Anyways, here's an example: >>> >>> $ cat zstr-bug2.py >>> from pyczmq import zmq, zctx, zsocket, zstr >>> >>> ctx = zctx.new() >>> rep = zsocket.new(ctx, zmq.REP) >>> zsocket.bind(rep, "tcp://127.0.0.1:5253") >>> >>> req = zsocket.new(ctx, zmq.REQ) >>> zsocket.connect(req, "tcp://127.0.0.1:5253") >>> >>> out_data = "blah blah hey what is \0 doing here?" >>> zstr.send(req, out_data) >>> >>> in_data = zstr.recv(rep) >>> assert in_data == out_data, "expected %r, got %r" % (out_data, in_data) >>> >>> $ python zstr-bug2.py >>> Traceback (most recent call last): >>> File "zstr-bug2.py", line 14, in <module> >>> assert in_data == out_data, "expected %r, got %r" % (out_data, >>> in_data) >>> AssertionError: expected 'blah blah hey what is \x00 doing here?', got >>> 'blah blah hey what is ' >>> >>> So: until I get to the point of needing secure authentication, pyczmq >>> just gets in my way and makes life harder. Darn. >>> >>> Greg >>> _______________________________________________ >>> zeromq-dev mailing list >>> [email protected] >>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>> >> >> >
_______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
