Hello Erik and folks, Have you tried spawning multiple acceptor threads ?
I am a ZeroMQ newbie and I might be wrong but.. - with only one acceptor, with need an intermediate device? you could connect workers sockets to the socket bond in the acceptor. - have you tried replacing the REQ/REP pattern by a PUSH/PULL pattern? Do you really need the pointer to be returned to the acceptor thread? I feel you could parallelize better your jobs in the workers, without waiting for a reply for each connection. What I read in your exemple is that you are benchmark accept() more than ZMQ. -- Jean-François SMIGIELSKI +33 (0) 625 135 563 2013/2/2 Erik Fears <[email protected]> > I'm using zeromq 3.2.2, inproc sockets, and the ZMQ_QUEUE device to > dispatch incoming > TCP connections from an acceptor thread to 8 worker threads. > > I'm finding that I can only send about 8500 messages/second over the queue > to worker threads. > > I've tried reducing the amount of work done outside zmq as much as > possible to isolate the performance > problem. > > Any ideas? Is the queue code expected to be this slow? > > Here's the setup: > > 1. There's a pthread that starts that has the purpose of being the queue. > Here's how it's started: > > void *frontend_socket = zmq_socket(zmq_context(), ZMQ_ROUTER); > void *backend_socket = zmq_socket(zmq_context(), ZMQ_DEALER); > zmq_bind(frontend_socket, "inproc://listener-queue-frontend"); > zmq_bind(backend_socket, "inproc://listener-queue-backend"); > boost::thread dispatch_thread(boost::bind(&dispatch_thread_main, > frontend_socket, backend_socket)); > > > void > dispatch_thread_main(void *frontend_socket, void *backend_socket) > { > zmq_device(ZMQ_QUEUE, frontend_socket, backend_socket); > zmq_close(frontend_socket); > zmq_close(backend_socket); > } > > > 2. There's a second thread that's purpose is to receive new connections > (accept()), and then > send them to the queue. Here's what the code looks like: > > void *dispatch_socket = zmq_socket(zmq_context(), ZMQ_REQ); > > if (zmq_connect(dispatch_socket, "inproc://listener-queue-frontend") > != 0) { > // handle error > } > > while (int new_client_fd = accept(listen_fd, (sockaddr*) &client_addr, > &client_addr_len)) > { > > //Sends to dispatcher and should immediately unblock > zmq_send_ptr<int>(dispatch_socket, &new_client_fd); > > //Block zmq_recv_ptr will block until the reply > int *fd = zmq_recv_ptr<int>(dispatch_socket); > assert(fd == &new_client_fd); > } > > > 3. N number of works (e.g. 8) are started that execute the following code. > All I'm doing right now > is calling close() on the accepted socket in the worker thread. > > void *dispatch_socket = zmq_socket(zmq_context(), ZMQ_REP); > > if (zmq_connect(dispatch_socket, > "inproc://listener-queue-backend") != 0) { > //handle error > } > > int *client_fd = NULL; > while ((client_fd = zmq_recv_ptr<int>(dispatch_socket, 0)) != > nullptr) { > close(*client_fd); > zmq_send_ptr<int>(dispatch_socket.get(), client_fd); > } > > > 4. zmq_recv_ptr/send_ptr are just some helper functions designed to > implement 0copy message > passing of pointers. They use zmq_msg_recv/send, and zmq_msg_init_data. > I don't think these > are the source of any performance issues. > > > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > >
_______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
