It’s a 10Gbit network. Each of the 50 interfaces (one per node) is sending 150Mbytes while it is receiving 150Mbytes of data.
Patrick >> On Aug 13, 2018, at 11:31 AM, Justin Azoff <justin.az...@gmail.com> wrote: >> >> On Mon, Aug 13, 2018 at 2:07 AM Patrick Reedy <preed...@gmail.com> wrote: >> >> I’m struggling with a pipeline stage that needs to complete quickly, in less >> than a second, but sometimes is taking 5 or 6 seconds. >> >> In this pipeline stage each of the 50 nodes have to send their to piece of >> data to all of the 100 main workers - so 50 nodes each sending 100 pieces of >> data with each piece about 1.5M bytes. The main workers need to collect all >> of the pieces of data before the pipeline can advance to the next stage. > > What kind of network interfaces do you have? When this stage runs how > much data total does each physical interface send/retrieve? Could you > be saturating the network? > > > > -- > - Justin > _______________________________________________ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > https://lists.zeromq.org/mailman/listinfo/zeromq-dev _______________________________________________ zeromq-dev mailing list zeromq-dev@lists.zeromq.org https://lists.zeromq.org/mailman/listinfo/zeromq-dev
