While performance testing my code (which uses ZeroMQ) I’m observing *delayed message reception*.
*Test Scenario* Process1: Sends messages (each 76 bytes long) using zmq PUB/SUB (IPC Protocol) in an infinite while loop with 1msec delay. Process2: Receives messages from Process1 using zmq PUB/SUB (IPC protocol) in an infinite while loop. If either of the below two conditions are true on receiving the message I log to a file. - If the message delay (receive time - send time) is greater than 10msec. - If there is message loss. *Observation* After running the above test for 108 hours these are my observations: - No message loss - *Lots of message delays are seen.* Delay of more than 100msec is common, with max delay seen = 220msec. Delay appears to be random and not increasing incrementally or in sequence. If one message is delayed by 100msec the next 89 consecutive messages are also logged as they are received with delays 99, 98, 97 and so on till 10msec. From this I understand that once we see the delay for one packet all the subsequent packets are received at the same time. I suspected that Process2 was not getting scheduled frequently enough and increased the priority of this process to nice value -20 but it did not help. *I am not able to figure out why some messages are getting delayed. Does anyone have any ideas? Has anyone faced this sort of thing before? Any suggestions would be really helpful. Is there anything else I can try to get to the root cause of the message delay problem?* Note: Process1 and Process2 call my wrapper APIs which in turn calls the ZeroMq Apis. In the receive/send wrapper APIs there are no locks/mutex or any blocking statements. There are only a few parameter checks and socket validity checks before calling zmq_send/zmq_recv. Whether send/recv is blocking or non-blocking is decided by a parameter in wrapper API. Thanks, Divya
_______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
