Re: [zeromq-dev] Question about zero-copying, local and remote clients
Another good read on the topic: https://web.archive.org/web/20120929202131/http://lwn.net/Articles/179492/ On Wed, Sep 14, 2016 at 1:22 PM, Michel Pelletier < pelletier.mic...@gmail.com> wrote: > > > On Tue, Sep 13, 2016 at 12:37 PM, Mateusz Jemielity < > m.jemiel...@is-wireless.com> wrote: > >> Hi, >> >> >> >> I have a question about 0MQ’s internal use of zero-copy. >> >> >> > > >> Does 0MQ somehow know that server and client 1 are on the same machine? >> If it knew, then it could, for example, use shared memory to transfer the >> messages, without actually using tcp stack. >> >> I guess I could explicitly use ipc transport for processes on same host >> and do another bind in the server, but can I do it in such kinda-generic, >> single-bind way? >> >> > Pipe level (IPC) use of zero copy features is OS specific and therefore > not portable, there is a good thread here where Linus explains features and > the issues: > > https://web.archive.org/web/20130521163124/http://kerneltrap.org/node/6505 > > Quote: > > "There's a _huge_ downside to specialized interfaces. Admittedly, splice() > is a lot less specialized (ie it works in a much wider variety of loads), > but it's still very much a "corner-case" thing. You can always do the same > thing splice() does with a read/write pair instead, and be portable." > > I don't know if 0MQ has flags to enable splice() level optimization or > not, but it would be an interesting feature if it did, presumably improving > throughput for IPC endpoints by removing the copy and vm bookkeeping > requirements as all splice does is increment ref-counts for existing mapped > pages. > > -Michel > ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Question about zero-copying, local and remote clients
On Tue, Sep 13, 2016 at 12:37 PM, Mateusz Jemielity < m.jemiel...@is-wireless.com> wrote: > Hi, > > > > I have a question about 0MQ’s internal use of zero-copy. > > > > Does 0MQ somehow know that server and client 1 are on the same machine? If > it knew, then it could, for example, use shared memory to transfer the > messages, without actually using tcp stack. > > I guess I could explicitly use ipc transport for processes on same host > and do another bind in the server, but can I do it in such kinda-generic, > single-bind way? > > Pipe level (IPC) use of zero copy features is OS specific and therefore not portable, there is a good thread here where Linus explains features and the issues: https://web.archive.org/web/20130521163124/http://kerneltrap.org/node/6505 Quote: "There's a _huge_ downside to specialized interfaces. Admittedly, splice() is a lot less specialized (ie it works in a much wider variety of loads), but it's still very much a "corner-case" thing. You can always do the same thing splice() does with a read/write pair instead, and be portable." I don't know if 0MQ has flags to enable splice() level optimization or not, but it would be an interesting feature if it did, presumably improving throughput for IPC endpoints by removing the copy and vm bookkeeping requirements as all splice does is increment ref-counts for existing mapped pages. -Michel ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Question about zero-copying, local and remote clients
If you ask for TCP, you are going to get TCP. But you can however bind the same socket to multiple endpoints, one IPC, one TCP, etc. Note that "true" zero copy happens with ZMQ_PAIR sockets over inproc:// transport. IPC uses unix file sockets, so there is at least a write syscall and thus a copy. On 13 September 2016 at 20:37, Mateusz Jemielity wrote: > Hi, > > > > I have a question about 0MQ’s internal use of zero-copy. > > > > Suppose that I have a client-server setup using req/rep sockets that are > sending large messages between themselves. There are multiple clients, some > may run on the same host as the server, others are on remote hosts. All > endpoints use tcp, so I’d have: > > Server: zmq_bind(server_socket, “tcp://*:5000”); > > Client 1 (on same host): zmq_connect(client1_socket, > “tcp://127.0.0.1:5000”); > > Client 2 (on remote host): zmq_connect(client2_socket, > “tcp://192.168.1.2:5000”); > > > > Does 0MQ somehow know that server and client 1 are on the same machine? If > it knew, then it could, for example, use shared memory to transfer the > messages, without actually using tcp stack. > > I guess I could explicitly use ipc transport for processes on same host and > do another bind in the server, but can I do it in such kinda-generic, > single-bind way? > > > > If 0MQ doesn’t do this internally, can I get requester’s endpoint string in > the server? Or at least some indication of endpoint being local or remote? > If I had the endpoint, then I could parse it for “127.0.0.1” or “localhost”, > etc. and then set up a scheme for sending some shared memory parameters > instead of a giant message. > > > > Finally, whetever’s the answer for req/rep, does it apply for pub/sub, etc? > > > > Regards, > > Mateusz > > > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Question about zero-copying, local and remote clients
Hi, I have a question about 0MQ's internal use of zero-copy. Suppose that I have a client-server setup using req/rep sockets that are sending large messages between themselves. There are multiple clients, some may run on the same host as the server, others are on remote hosts. All endpoints use tcp, so I'd have: Server: zmq_bind(server_socket, "tcp://*:5000"); Client 1 (on same host): zmq_connect(client1_socket, "tcp://127.0.0.1:5000"); Client 2 (on remote host): zmq_connect(client2_socket, "tcp://192.168.1.2:5000"); Does 0MQ somehow know that server and client 1 are on the same machine? If it knew, then it could, for example, use shared memory to transfer the messages, without actually using tcp stack. I guess I could explicitly use ipc transport for processes on same host and do another bind in the server, but can I do it in such kinda-generic, single-bind way? If 0MQ doesn't do this internally, can I get requester's endpoint string in the server? Or at least some indication of endpoint being local or remote? If I had the endpoint, then I could parse it for "127.0.0.1" or "localhost", etc. and then set up a scheme for sending some shared memory parameters instead of a giant message. Finally, whetever's the answer for req/rep, does it apply for pub/sub, etc? Regards, Mateusz ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
