On Mon, 2006-06-05 at 11:12, hbchen wrote: > Hi, > I have a question about the IPoIB bandwidth performance. > I did netperf testing using Single GiGE, Myrinet D card, Myrinet 10G > ethernet card, > and Voltaire Infiniband 4X HCA400Ex (PCI-Express interface). > > > NIC (Jumbo enabled) Line bandwidth(LB) IPoverNIC bandwidth utilization > (IPoNIC/LB) > --------------------- ---------------- -------------- > ---------------------------------- > Single Gigabit NIC : 1Gb/sec=125MB/sec 120MB/sec 96% (PIC-X interface) > Myrinet D card : 250MB/sec 240~-245MB/sec 96% ~ 98% (PCI-X interface) > Myrinet 10G Ethernet: 10Gb/sec=1280MB/sec 980MB/sec 76.6% (My testing > using Linux 2.6.14.6) > (PCI-Express) 1225MB/sec 95.7% (Data from Myrinet website) > IB HCA4X(PCI-Express): 10Gb/sec=1280MB/sec 420MB/sec 32.8% (My testing > using Linux 2.6.14.6) > 474MB/sec 37% (the best from OpenIB mailing list) > (2.6.12-rc5 patch 1) > > Why the bandwidth utilization of IPoIB is so low compared to the others > NICs?
One thing to note is that the max utilization of 10G IB (4x) is 8G due to the signalling being included in this rate (unlike ethernet whose rate represents the data rate and does not include the signalling overhead). -- Hal > There must be a lot of room to improve the IPoIB software to reach 75%+ > bandwidth utilization. > > > HB Chen > Los Alamos National Lab > [EMAIL PROTECTED] > > _______________________________________________ > openib-general mailing list > [email protected] > http://openib.org/mailman/listinfo/openib-general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
