Re: [zeromq-dev] Odd numbers with zeromq

2012-09-21 Thread Robert G. Jakabosky
On Wednesday 19, Christian Martinez wrote: Maybe I'm missing something here, but are people asserting here that one can't do a request reply MEP with various RPC technologies and exceed 1000 1KB messages a second? With only 1 client and 1 concurrent request, then yes. The msg/sec throughput

Re: [zeromq-dev] Odd numbers with zeromq

2012-09-20 Thread Pieter Hintjens
On Wed, Sep 19, 2012 at 7:25 PM, Christian Martinez ch...@microsoft.com wrote: Maybe I’m missing something here, but are people asserting here that one can’t do a request reply MEP with various RPC technologies and exceed 1000 1KB messages a second? Pretty much, yes. There is a discussion in

Re: [zeromq-dev] Odd numbers with zeromq

2012-09-19 Thread Paul Colomiets
HI Maninder, On Wed, Sep 19, 2012 at 6:21 AM, Maninder Batth whatpuzzle...@gmail.com wrote: Paul, With messages being sent one way, via pub and sub sockets, i am getting a very decent performance. About 80% of our network gets saturated. The code is zserver.cpp and zclient.cpp But if i

Re: [zeromq-dev] Odd numbers with zeromq

2012-09-19 Thread Christian Martinez
: Tuesday, September 18, 2012 10:03 PM To: ZeroMQ development list Subject: Re: [zeromq-dev] Odd numbers with zeromq excessive? you are going from a scheme where packets go out in bulk asynchronously, to a scheme where a single packet goes out, a process needs to woken up, processes a single packet

Re: [zeromq-dev] Odd numbers with zeromq

2012-09-19 Thread Andrew Hume
Hume Sent: Tuesday, September 18, 2012 10:03 PM To: ZeroMQ development list Subject: Re: [zeromq-dev] Odd numbers with zeromq excessive? you are going from a scheme where packets go out in bulk asynchronously, to a scheme where a single packet goes out, a process needs to woken up

Re: [zeromq-dev] Odd numbers with zeromq

2012-09-18 Thread Maninder Batth
Paul, Here is number of messages as seen by the server in one second. Each message is 1024 byte excluding tcp/ip and zmq headers. Based on these numbers and i am getting a throughput of 1.4 Gb/sec. Enclosed is the source code for the server and the client. Number of msgs in one second: 189448

Re: [zeromq-dev] Odd numbers with zeromq

2012-09-18 Thread Paul Colomiets
Hi Maninder, On Tue, Sep 18, 2012 at 10:21 PM, Maninder Batth whatpuzzle...@gmail.com wrote: Paul, Here is number of messages as seen by the server in one second. Each message is 1024 byte excluding tcp/ip and zmq headers. Based on these numbers and i am getting a throughput of 1.4 Gb/sec.

Re: [zeromq-dev] Odd numbers with zeromq

2012-09-18 Thread Maninder Batth
Paul, Thank you again for your help. Now with message copying, i am getting a throughput of .8Gb, which is what i would have expected on a 1Gb network. On Tue, Sep 18, 2012 at 4:20 PM, Paul Colomiets p...@colomiets.name wrote: Hi Maninder, On Tue, Sep 18, 2012 at 10:21 PM, Maninder Batth

Re: [zeromq-dev] Odd numbers with zeromq

2012-09-18 Thread Maninder Batth
Paul, With messages being sent one way, via pub and sub sockets, i am getting a very decent performance. About 80% of our network gets saturated. The code is zserver.cpp and zclient.cpp But if i configure the software such that client only sends the next message, after it has received a response

Re: [zeromq-dev] Odd numbers with zeromq

2012-09-18 Thread Andrew Hume
why would you expect different? REQ/REP involves a synchronous repsonse so there is no opportunity to do anything fast. On Sep 18, 2012, at 8:21 PM, Maninder Batth wrote: Paul, With messages being sent one way, via pub and sub sockets, i am getting a very decent performance. About 80% of

Re: [zeromq-dev] Odd numbers with zeromq

2012-09-18 Thread Bennie Kloosteman
If you wait for responses you have to wait on latency which can get much worse eg international 250ms your limit is 4 messages per second. You cannot do better That why people build asynch systems...and why Doom/Quake use to spam UDP position updates - without waiting for an answer. Ben On

Re: [zeromq-dev] Odd numbers with zeromq

2012-09-18 Thread Apostolis Xekoukoulotakis
If you want to acknowledge a msg but each msg is independent of each other, use Router/dealer. There is nothing else you can do. 2012/9/19 Bennie Kloosteman bkloo...@gmail.com If you wait for responses you have to wait on latency which can get much worse eg international 250ms your limit is

Re: [zeromq-dev] Odd numbers with zeromq

2012-09-18 Thread Maninder Batth
A difference of 110K to 1k a second seems excessive. But i am not saying it is wrong or right. I am just wondering a. Am i doing something silly from api usage perspective b. If someone has similar experience, (one way burst, vs send, recieve ack and then send next message), maybe they can share

Re: [zeromq-dev] Odd numbers with zeromq

2012-09-18 Thread Andrew Hume
excessive? you are going from a scheme where packets go out in bulk asynchronously, to a scheme where a single packet goes out, a process needs to woken up, processes a single packet, and then send a single packet, and then wake up the sending process. i think you're lucky to get 1k/s. a. no. it

Re: [zeromq-dev] Odd numbers with zeromq

2012-09-16 Thread Chuck Remes
On Sep 15, 2012, at 8:37 PM, Maninder Batth wrote: Paul, That was an excellent idea indeed. As i was simply following the first example in the guide and developed on top of it, i did not read the api. After going over the socket API, now, instead of how many messages can a pub send in a

Re: [zeromq-dev] Odd numbers with zeromq

2012-09-15 Thread Maninder Batth
Paul, That was an excellent idea indeed. As i was simply following the first example in the guide and developed on top of it, i did not read the api. After going over the socket API, now, instead of how many messages can a pub send in a second, i record, how many messages can a consumer get in one

Re: [zeromq-dev] Odd numbers with zeromq

2012-09-14 Thread Paul Colomiets
Hi , On Thu, Sep 13, 2012 at 7:33 PM, Maninder Batth whatpuzzle...@gmail.com wrote: 2. Clients which publish without needing ack: In this use case, a client publishes data as fast as it can in one direction and server simply discards the output. Enclosed are files zserver.cpp and zclient.cpp

Re: [zeromq-dev] Odd numbers with zeromq

2012-09-14 Thread Maninder Batth
That does not sound right. These are tcp sockets internally, so either the message should be sent OR an exception should be thrown. But, it should not silently discard messages. On Fri, Sep 14, 2012 at 3:13 AM, Paul Colomiets p...@colomiets.name wrote: Hi , On Thu, Sep 13, 2012 at 7:33

Re: [zeromq-dev] Odd numbers with zeromq

2012-09-14 Thread Maninder Batth
Regarding calculations, i am simply incrementing a counter after send() and message size is constant. On Fri, Sep 14, 2012 at 1:35 AM, Pieter Hintjens p...@imatix.com wrote: On Fri, Sep 14, 2012 at 1:33 AM, Maninder Batth whatpuzzle...@gmail.com wrote: With message size of 1KB, i am

Re: [zeromq-dev] Odd numbers with zeromq

2012-09-14 Thread Michel Pelletier
On Fri, Sep 14, 2012 at 9:55 AM, Maninder Batth whatpuzzle...@gmail.com wrote: That does not sound right. These are tcp sockets internally, so either the message should be sent OR an exception should be thrown. But, it should not silently discard messages. Different sockets have different

[zeromq-dev] Odd numbers with zeromq

2012-09-13 Thread Maninder Batth
I have 2 use cases. 1. Clients which wiat for ack to publish next message 2. Clients which publish without needing any ack Let me elaborate 2. *Clients which publish without needing ack: * In this use case, a client publishes data as fast as it can in one direction and server simply discards