Make a pull request :) On Wed, Oct 2, 2019, 20:27 Francesco <francesco.monto...@gmail.com> wrote:
> > > Il mer 2 ott 2019, 19:05 Doron Somech <somdo...@gmail.com> ha scritto: > >> >> You don't need to create multiple sockets, just call connect multiple >> times with same address. >> > Wow, really?? > I wish I had known that, I already changed quite a bit of code to use > multiple zmq sockets to make better use of background zmq threads!! > > I will try connecting multiple times... At this point I suggest modifying > the benchmark utility to just do this trick and update the performance > graphs in the wiki with new results! > > Francesco > > > On Wed, Oct 2, 2019, 19:45 Brett Viren via zeromq-dev < >> zeromq-dev@lists.zeromq.org> wrote: >> >>> Hi Francesco, >>> >>> I confirm your benchmark using two systems with the same 100 Gbps >>> Mellanox NICs but with an intervening Juniper QFX5200 switch (100 Gbps >>> ports). >>> >>> To reach ~25 Gbps with the largest message sizes required "jumbo frame" >>> MTU. The default mtu=1500 allows only ~20 Gbps. I also tried two more >>> doubling of zmsg size in the benchmark and these produce no significant >>> increase in throughput. OTOH, pinning the receiver (local_thr) to a CPU >>> gets it up to 33 Gbps. >>> >>> I note that iperf3 can achieve almost 40 Gbps (20 Gbps w MTU=1500). >>> Multiple simultaneous iperf3 tests can, in aggregate, use 90-100 Gbps. >>> >>> In both the ZMQ and singular iperf3 tests, it seems that CPU is the >>> bottleneck. For ZeroMQ the receiver's I/O thread is pegged at 100%. >>> With iperf3 it's that of the client/sender. The other ends in both >>> cases are at about 50%. >>> >>> The zguide suggests to use one I/O thread per GByte/s (faq says "Gbps") >>> so I tried the naive thing and hacked the ZMQ remote_thr.cpp and >>> local_thr.cpp so each use ten I/O threads. While I see all ten threads >>> in "top -H", still only one thread uses any CPU and it remains pegged at >>> 100% on the receiver (local_thr) and about 50% on the sender >>> (remote_thr). I think now that I misinterpreted this advice and it's >>> really relevant to the case of handling a very large number of >>> connections. >>> >>> >>> Any suggestions on how to let ZeroMQ get higher throughput at 100 Gbps? >>> If so, I'll give them a try. >>> >>> >>> Cheers, >>> -Brett. >>> >>> Francesco <francesco.monto...@gmail.com> writes: >>> >>> > Hi all, >>> > >>> > I placed here: >>> > http://zeromq.org/results:100gbe-tests-v432 >>> > the results I collected using 2 Mellanox ConnectX-5 linked by 100Gbps >>> > fiber cable. >>> > >>> > The results are not too much different from those at 10gpbs >>> > (http://zeromq.org/results:10gbe-tests-v432 )... the difference in TCP >>> > throughput is that >>> > - even using 100kB-long messages we still cannot saturate the link >>> > - latency is very much improved for messages > 10kB long >>> > >>> > Hopefully we will be able to improve performances in the future to >>> > improve these benchmarks... >>> > >>> > Francesco >>> > _______________________________________________ >>> > zeromq-dev mailing list >>> > zeromq-dev@lists.zeromq.org >>> > https://lists.zeromq.org/mailman/listinfo/zeromq-dev >>> _______________________________________________ >>> zeromq-dev mailing list >>> zeromq-dev@lists.zeromq.org >>> https://lists.zeromq.org/mailman/listinfo/zeromq-dev >>> >>
_______________________________________________ zeromq-dev mailing list zeromq-dev@lists.zeromq.org https://lists.zeromq.org/mailman/listinfo/zeromq-dev
