Re: [zeromq-dev] PUB-SUB Performance with 5000 subscriber
if subscribers are not on network, would it be possible stream market data to 5k+ subscribers from one publisher ? I am planning to use NetMQ On Sun, Oct 28, 2018 at 1:27 PM Doron Somech wrote: > If you are on your own network and multicast is an option you can try > scatter-gather pattern and use multicast or PGM with pub-sub. > > > On Fri, Oct 26, 2018 at 7:35 AM Justin Karneges > wrote: > >> 5000*500kb = 2.5gb. That's a lot of data. What throughput are you >> expecting? >> >> On Thu, Oct 25, 2018, at 8:11 PM, Hiren Shah wrote: >> >> Notice there is significant increase in latency for bigger Message Size >> when Number of subscriber increases. >> >> 4k Message size gets to all Subscriber in < 1sec but 500k Message Size >> takes about 60sec. >> >> With Async IO and 5000 clients 500k message should be quite faster then >> what I am seeing. I feel like I am missing option which I am setting. >> >> >> *___* >> zeromq-dev mailing list >> zeromq-dev@lists.zeromq.org >> https://lists.zeromq.org/mailman/listinfo/zeromq-dev >> >> >> ___ >> zeromq-dev mailing list >> zeromq-dev@lists.zeromq.org >> https://lists.zeromq.org/mailman/listinfo/zeromq-dev >> > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org https://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] PUB-SUB Performance with 5000 subscriber
Somehow I had missed this option before ZMQ_IO_THREADS . Increasing IO threads have help improve performance/throughput On Sun, Oct 28, 2018 at 5:27 AM Doron Somech wrote: > If you are on your own network and multicast is an option you can try > scatter-gather pattern and use multicast or PGM with pub-sub. > > > On Fri, Oct 26, 2018 at 7:35 AM Justin Karneges > wrote: > >> 5000*500kb = 2.5gb. That's a lot of data. What throughput are you >> expecting? >> >> On Thu, Oct 25, 2018, at 8:11 PM, Hiren Shah wrote: >> >> Notice there is significant increase in latency for bigger Message Size >> when Number of subscriber increases. >> >> 4k Message size gets to all Subscriber in < 1sec but 500k Message Size >> takes about 60sec. >> >> With Async IO and 5000 clients 500k message should be quite faster then >> what I am seeing. I feel like I am missing option which I am setting. >> >> >> *___* >> zeromq-dev mailing list >> zeromq-dev@lists.zeromq.org >> https://lists.zeromq.org/mailman/listinfo/zeromq-dev >> >> >> ___ >> zeromq-dev mailing list >> zeromq-dev@lists.zeromq.org >> https://lists.zeromq.org/mailman/listinfo/zeromq-dev >> > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org https://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] PUB-SUB Performance with 5000 subscriber
If you are on your own network and multicast is an option you can try scatter-gather pattern and use multicast or PGM with pub-sub. On Fri, Oct 26, 2018 at 7:35 AM Justin Karneges wrote: > 5000*500kb = 2.5gb. That's a lot of data. What throughput are you > expecting? > > On Thu, Oct 25, 2018, at 8:11 PM, Hiren Shah wrote: > > Notice there is significant increase in latency for bigger Message Size > when Number of subscriber increases. > > 4k Message size gets to all Subscriber in < 1sec but 500k Message Size > takes about 60sec. > > With Async IO and 5000 clients 500k message should be quite faster then > what I am seeing. I feel like I am missing option which I am setting. > > > *___* > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > https://lists.zeromq.org/mailman/listinfo/zeromq-dev > ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org https://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] PUB-SUB Performance with 5000 subscriber
5000*500kb = 2.5gb. That's a lot of data. What throughput are you expecting? On Thu, Oct 25, 2018, at 8:11 PM, Hiren Shah wrote: > Notice there is significant increase in latency for bigger Message > Size when Number of subscriber increases.> > 4k Message size gets to all Subscriber in < 1sec but 500k Message Size > takes about 60sec.> > With Async IO and 5000 clients 500k message should be quite faster > then what I am seeing. I feel like I am missing option which I am > setting.> > > _ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > https://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org https://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] PUB-SUB Performance with 5000 subscriber
Notice there is significant increase in latency for bigger Message Size when Number of subscriber increases. 4k Message size gets to all Subscriber in < 1sec but 500k Message Size takes about 60sec. With Async IO and 5000 clients 500k message should be quite faster then what I am seeing. I feel like I am missing option which I am setting. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org https://lists.zeromq.org/mailman/listinfo/zeromq-dev
