Am 18.04.2014 um 12:37 schrieb Goswin von Brederlow <[email protected]>:
> On Fri, Apr 18, 2014 at 01:23:59AM +0200, Michael Haberler wrote: >> >> Am 18.04.2014 um 00:33 schrieb Goswin von Brederlow <[email protected]>: >> >>> On Thu, Apr 17, 2014 at 10:54:59PM +0200, Michael Haberler wrote: >>>> Goswin, >>>> >>>> Am 17.04.2014 um 11:24 schrieb Goswin von Brederlow <[email protected]>: >>>> >>>>> On Wed, Apr 16, 2014 at 08:07:27PM +0200, Michael Haberler wrote: >>>>>> we're using zeroMQ in machinekit (formerly LinuxCNC) >>>>>> >>>>>> to retain the end-to-end principle, we need to be able to send multipart >>>>>> messages to and from RT threads >>>>>> >>>>>> that isnt possible with libzmq code since it's C++ and uses dynamic >>>>>> memory allocation >>>>>> >>>>>> the code which we'll be using is based on a lock-free ringbuffer in >>>>>> shared memory (ring_*), and a multipart framing layer on top of that >>>>>> (bring_*): http://psha.org.ru/cgit/psha/ring/?h=wip-buffer >>>>>> >>>>>> a proxy thread at the RT boundary translates from zmq to ringbuffer >>>>>> framing and back >>>>>> >>>>>> just in case somebody has a similar requirement >>>>>> >>>>>> - Michael >>>>> >>>>> doesn't zero-copy fix that? >>>> >>>> I dont understand what are you are suggesting, could you elaborate? >>>> >>>> - Michael >>> >>> You have your data in some buffer somewhere in memory. Then when you >>> can call zmq_msg_init_data() to make a message out of it without >>> copying the data: >> >> I think I should have mentioned the environment restrictions: >> >> The application is a cooperative ensemble of normal userland threads, and RT >> threads. >> >> All these threads share is a shm segment; nothing else. RT threads may live >> in the kernel, as a hypervisor thread like in Xenomai, or as a 'posix thread >> on steroids' with RT-PREEMPT. Or even on a core running an entirely >> different OS, or a bare metal code blob. The application works with either, >> which is why this framing was needed. >> >> The fact that you have some data outside such a shared memory segment has no >> bearing; it's not accessible. >> >> That said, the methods in the repo I pointed to support zero-copy writing at >> the frame level _within_ the shm segment. >> >> - Michael >> >>> >>> The zmq_msg_init_data() function shall initialise the message object >>> referenced by msg to represent the content referenced by the buffer >>> located at address data, size bytes long. No copy of data shall be >>> performed and ØMQ shall take ownership of the supplied buffer. >>> >>> If provided, the deallocation function ffn shall be called once the >>> data buffer is no longer required by ØMQ, with the data and hint >>> arguments supplied to zmq_msg_init_data(). >>> >>> The deallocation function allows you to reuse buffers that zmq has >>> finished with so you don't need to dynamically allocate them. And the >>> data isn't copied so the zmq_msg functions run in constant time no >>> matter how large the buffer. >>> >>> MfG >>> Goswin > > The point would be to use buffers in the shared memory segment from > the start so you can pass them from the normal threads to the RT > threads / kernel / barebone blob without the need of a proxy thread > that copies them. I would love to have that. But it is tough going in allocation and referencing; the key decision to make is: can one assume a shm segment is mapped to the same address for all parties that's a risky one, but if one says 'yes' to this, it should be straighforward except for a shm-based memory allocator. if the answer is 'no' (ours is 'definitely not' for the environment at hand), one needs to go the route of base/offset pointer representation. That's going to be hairy. a related issue is signaling - signalfd's work fine between arbitrary userland processes (see https://github.com/mhaberler/libancillary) but it I found it tough to have a RT-compatible version of this which covers RTAI and Xenomai as well in a common API (the application is portable across all RT kernels). For now we use polling and may get smarter downstream. - Michael _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
