I actually replaced it with a PUSH/PULL after Pieter's response and it appears to be significantly faster now. You are correct that I do not need the reply from the worker.
Thanks! Erik On Sat, Feb 2, 2013 at 1:14 AM, Jean-François Smigielski < [email protected]> wrote: > Hello Erik and folks, > > Have you tried spawning multiple acceptor threads ? > > I am a ZeroMQ newbie and I might be wrong but.. > > - with only one acceptor, with need an intermediate device? you could > connect workers sockets to the socket bond in the acceptor. > - have you tried replacing the REQ/REP pattern by a PUSH/PULL pattern? > Do you really need the pointer to be returned to the acceptor thread? I > feel you could parallelize better your jobs in the workers, without waiting > for a reply for each connection. What I read in your exemple is that you > are benchmark accept() more than ZMQ. > > > -- > Jean-François SMIGIELSKI > +33 (0) 625 135 563 > > > 2013/2/2 Erik Fears <[email protected]> > >> I'm using zeromq 3.2.2, inproc sockets, and the ZMQ_QUEUE device to >> dispatch incoming >> TCP connections from an acceptor thread to 8 worker threads. >> >> I'm finding that I can only send about 8500 messages/second over the >> queue to worker threads. >> >> I've tried reducing the amount of work done outside zmq as much as >> possible to isolate the performance >> problem. >> >> Any ideas? Is the queue code expected to be this slow? >> >> Here's the setup: >> >> 1. There's a pthread that starts that has the purpose of being the queue. >> Here's how it's started: >> >> void *frontend_socket = zmq_socket(zmq_context(), ZMQ_ROUTER); >> void *backend_socket = zmq_socket(zmq_context(), ZMQ_DEALER); >> zmq_bind(frontend_socket, "inproc://listener-queue-frontend"); >> zmq_bind(backend_socket, "inproc://listener-queue-backend"); >> boost::thread dispatch_thread(boost::bind(&dispatch_thread_main, >> frontend_socket, backend_socket)); >> >> >> void >> dispatch_thread_main(void *frontend_socket, void *backend_socket) >> { >> zmq_device(ZMQ_QUEUE, frontend_socket, backend_socket); >> zmq_close(frontend_socket); >> zmq_close(backend_socket); >> } >> >> >> 2. There's a second thread that's purpose is to receive new connections >> (accept()), and then >> send them to the queue. Here's what the code looks like: >> >> void *dispatch_socket = zmq_socket(zmq_context(), ZMQ_REQ); >> >> if (zmq_connect(dispatch_socket, "inproc://listener-queue-frontend") >> != 0) { >> // handle error >> } >> >> while (int new_client_fd = accept(listen_fd, (sockaddr*) &client_addr, >> &client_addr_len)) >> { >> >> //Sends to dispatcher and should immediately unblock >> zmq_send_ptr<int>(dispatch_socket, &new_client_fd); >> >> //Block zmq_recv_ptr will block until the reply >> int *fd = zmq_recv_ptr<int>(dispatch_socket); >> assert(fd == &new_client_fd); >> } >> >> >> 3. N number of works (e.g. 8) are started that execute the following >> code. All I'm doing right now >> is calling close() on the accepted socket in the worker thread. >> >> void *dispatch_socket = zmq_socket(zmq_context(), ZMQ_REP); >> >> if (zmq_connect(dispatch_socket, >> "inproc://listener-queue-backend") != 0) { >> //handle error >> } >> >> int *client_fd = NULL; >> while ((client_fd = zmq_recv_ptr<int>(dispatch_socket, 0)) != >> nullptr) { >> close(*client_fd); >> zmq_send_ptr<int>(dispatch_socket.get(), client_fd); >> } >> >> >> 4. zmq_recv_ptr/send_ptr are just some helper functions designed to >> implement 0copy message >> passing of pointers. They use zmq_msg_recv/send, and >> zmq_msg_init_data. I don't think these >> are the source of any performance issues. >> >> >> >> _______________________________________________ >> zeromq-dev mailing list >> [email protected] >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> >> > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > >
_______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
