Sure, but when I have 3 parallel clients (300k messages), I am observing packet loss.
Sincerely, Vishal On Thu, Dec 11, 2014 at 5:43 PM, André Caron <[email protected]> wrote: > Throughput is bound by multiple factors. In your case, my guess is that > your bottleneck is the throughput of a single TCP socket on your system. I > doubt that you can reach millions of messages per second with only 3 > clients. If you add more clients, you should see the server's throughput > increase well over 30k. > > Cheers, > > André > > On Thu, Dec 11, 2014 at 4:40 AM, Vishal Ahuja <[email protected]> wrote: > >> I removed the printfs, and increased the TCP send and receive buffers to >> the max possible. When I have two clients (running in parallel) sending >> 100k messages, there is no packet loss. But when I increase the number of >> clients to three, then each client receives around 93k packets. Also, the >> majordomo specification (http://rfc.zeromq.org/spec:7) says that the >> broker should be able to switch millions of messages per second, whereas I >> seeing a throughput of around 30k messages per second. Can someone tell how >> to improve the throughput? >> >> Sincerely, >> >> Vishal >> >> On Thu, Dec 11, 2014 at 10:37 AM, Andre Caron <[email protected]> >> wrote: >> >>> The broker uses a router socket, which silently drops outbound messages >>> when the sndhwm is reached. If your client is reading more slowly than the >>> broker is writing, you'll lose messages. >>> >>> In your case, the slow part is writing to the terminal. Try removing >>> the printf in the recv loop and you should lose fewer messages. >>> >>> André >>> >>> > On Dec 10, 2014, at 9:40 PM, Vishal Ahuja <[email protected]> wrote: >>> > >>> > While reading the zeromq guide, I came across client code (pertaining >>> to the majordomo pattern) which sends 100k requests in a loop, and then >>> receives the reply in a second loop. >>> > >>> > #include "../include/mdp.h" >>> > #include <time.h> >>> > >>> > >>> > int main (int argc, char *argv []) >>> > { >>> > int verbose = (argc > 1 && streq (argv [1], "-v")); >>> > mdp_client_t *session = mdp_client_new >>> ("tcp://localhost:5555", verbose); >>> > int count; >>> > for (count = 0; count < 100000; count++) { >>> > zmsg_t *request = zmsg_new (); >>> > zmsg_pushstr (request, "Hello world"); >>> > mdp_client_send (session, "echo", &request); >>> > } >>> > printf("sent all\n"); >>> > >>> > for (count = 0; count < 100000; count++) { >>> > zmsg_t *reply = mdp_client_recv (session,NULL,NULL); >>> > if (reply) >>> > zmsg_destroy (&reply); >>> > else >>> > break; // Interrupted by Ctrl-C >>> > printf("reply received:%d\n", count); >>> > } >>> > printf ("%d replies received\n", count); >>> > mdp_client_destroy (&session); >>> > return 0; >>> > } >>> > >>> > I have added a counter to count the number of replies that the worker >>> (test_worker.c) sends to the broker, and another counter in mdp_broker.c to >>> count the number of replies the broker sends to a client. Both of these >>> count up to 100k, but the client is receiving only around 37k replies. >>> > >>> > If the number of client requests is set to around 40k, then it >>> receives all the replies. Can someone please tell me why packets are lost >>> when the client sends more than 40k asynchronous requests? >>> > >>> > I tried setting the HWM to 100k for the broker socket, but the problem >>> persists: >>> > >>> > static broker_t * >>> > s_broker_new (int verbose) >>> > { >>> > broker_t *self = (broker_t *) zmalloc (sizeof (broker_t)); >>> > int64_t hwm = 100000; >>> > // Initialize broker state >>> > self->ctx = zctx_new (); >>> > self->socket = zsocket_new (self->ctx, ZMQ_ROUTER); >>> > zmq_setsockopt(self->socket, ZMQ_SNDHWM, &hwm, sizeof(hwm)); >>> > >>> > zmq_setsockopt(self->socket, ZMQ_RCVHWM, &hwm, sizeof(hwm)); >>> > self->verbose = verbose; >>> > self->services = zhash_new (); >>> > self->workers = zhash_new (); >>> > self->waiting = zlist_new (); >>> > self->heartbeat_at = zclock_time () + HEARTBEAT_INTERVAL; >>> > return self; >>> > } >>> > >>> > Vishal >>> > _______________________________________________ >>> > zeromq-dev mailing list >>> > [email protected] >>> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>> _______________________________________________ >>> zeromq-dev mailing list >>> [email protected] >>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>> >> >> >> _______________________________________________ >> zeromq-dev mailing list >> [email protected] >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> >> > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > >
_______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
