Re: [openib-general] Question about the IPoIB bandwidth performance ?

Mon, 05 Jun 2006 08:38:46 -0700

Hal Rosenstock wrote:
On Mon, 2006-06-05 at 11:12, hbchen wrote:
  
Hi,
I have a question about the IPoIB bandwidth performance.
I did netperf testing using Single GiGE, Myrinet D card, Myrinet 10G
ethernet card,
and Voltaire Infiniband 4X HCA400Ex (PCI-Express interface).


NIC (Jumbo enabled) Line bandwidth(LB) IPoverNIC bandwidth utilization
(IPoNIC/LB)
--------------------- ---------------- --------------
----------------------------------
Single Gigabit NIC : 1Gb/sec=125MB/sec 120MB/sec 96% (PIC-X interface)
Myrinet D card : 250MB/sec 240~-245MB/sec 96% ~ 98% (PCI-X interface)
Myrinet 10G Ethernet: 10Gb/sec=1280MB/sec 980MB/sec 76.6% (My testing
using Linux 2.6.14.6)
(PCI-Express) 1225MB/sec 95.7% (Data from Myrinet website)
IB HCA4X(PCI-Express): 10Gb/sec=1280MB/sec 420MB/sec 32.8% (My testing
using Linux 2.6.14.6)
474MB/sec 37% (the best from OpenIB mailing list)
(2.6.12-rc5 patch 1)

Why the bandwidth utilization of IPoIB is so low compared to the others
NICs?
    

One thing to note is that the max utilization of 10G IB (4x) is 8G due
to the signalling being included in this rate (unlike ethernet whose
rate represents the data rate and does not include the signalling
overhead).
Hal,
Even with this IB-4X = 8Gb/sec = 1024 MB/sec the IPoIB bandwidth utilization is still very low.
>> IPoIB=420MB/sec 
>> bandwidth utilization= 420/1024 = 41.01%


HB 



-- Hal

  
There must be a lot of room to improve the IPoIB software to reach 75%+
bandwidth utilization.


HB Chen
Los Alamos National Lab
[EMAIL PROTECTED]

_______________________________________________
openib-general mailing list
[email protected]
http://openib.org/mailman/listinfo/openib-general

To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general

    

  

_______________________________________________
openib-general mailing list
[email protected]
http://openib.org/mailman/listinfo/openib-general

To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general

Reply via email to