On Wed, Oct 23, 2013 at 6:07 PM, Morgan Yang <[email protected]>wrote:
> Hi All:**** > > ** ** > > I am doing testing with Mellanox’s CX3 40G NIC. **** > > ** ** > > The setup without OVS runs about 37~38 Gbps between multiple threads of > IPERF, and no errors are noticed on the Ethernet interfaces**** > > ** ** > > Host A Eth4(80.1.1.1) <-> Host B Eth4(80.2.1.1)**** > > ** ** > > After I introduce OVS (running 2.0.0), I still get 37~38 Gbps, but I begin > to see *errors and overruns* on the physical NIC**** > > ** ** > > ******** > > OVSFA(80.1.1.1) <-> Host A Eth4 <-> Host B Eth4 <-> OVSFB(80.2.1.1)**** > > ** ** > > root:~/mlnx# ovs-ofctl dump-ports ovsfa**** > > OFPST_PORT reply (xid=0x2): 2 ports**** > > port 1: rx pkts=7262324, bytes=451601757688, drop=0, errs=363, > frame=0, over=470, crc=0**** > > tx pkts=5624590, bytes=303750392, drop=0, errs=0, coll=0**** > > port LOCAL: rx pkts=5624537, bytes=303747170, drop=0, errs=0, frame=0, > over=0, crc=0**** > > tx pkts=7262379, bytes=451601761078, drop=0, errs=0, coll=0**** > > ** ** > > This is only showing up on the physical port and not on the LOCAL switch > port**** > > ** ** > > eth4 Link encap:Ethernet HWaddr 00:00:50:A4:5F:AC ** ** > > inet6 addr: fe80::200:50ff:fea4:5fac/64 Scope:Link**** > > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1**** > > RX packets:309077941 errors:363 dropped:0 overruns:470 frame:470 > **** > > TX packets:5624597 errors:0 dropped:0 overruns:0 carrier:0**** > > collisions:0 txqueuelen:1000 **** > > RX bytes:467899917070 (435.7 GiB) TX bytes:337492360 (321.8 MiB) > **** > > ** ** > > ovsfa Link encap:Ethernet HWaddr 00:00:50:A4:5F:AC **** > > inet addr:80.1.1.1 Bcast:80.255.255.255 Mask:255.0.0.0**** > > inet6 addr: fe80::1452:ceff:fe82:f794/64 Scope:Link**** > > UP BROADCAST RUNNING MTU:1500 Metric:1**** > > RX packets:7262782 errors:0 dropped:0 overruns:0 frame:0**** > > TX packets:5624537 errors:0 dropped:0 overruns:0 carrier:0**** > > collisions:0 txqueuelen:0 **** > > RX bytes:451601898904 (420.5 GiB) TX bytes:303747170 (289.6 MiB) > **** > > ** ** > > This gets really bad when I bring up multiple VM’s and run an aggregated > performance test**** > > ** ** > > When I bring up 8 VM’s, each VM will have a VNET to OVSFA. When I try to > run an aggregated iperf test, I can only get an aggregated throughput of > 20~21 Gbps**** > > ** ** > > Examining the port stats, it seems the physical link to OVSFA interface is > now a bottleneck (see port 1 stats)**** > > ** ** > > OFPST_PORT reply (xid=0x2): 10 ports**** > > port 14: rx pkts=993698, bytes=54019132, drop=0, errs=0, frame=0, > over=0, crc=0**** > > tx pkts=1001557, bytes=26452089418, drop=0, errs=0, coll=0**** > > port 16: rx pkts=1018316, bytes=55415244, drop=0, errs=0, frame=0, > over=0, crc=0**** > > tx pkts=1024662, bytes=24373022960, drop=0, errs=0, coll=0**** > > port 10: rx pkts=467571, bytes=25466650, drop=0, errs=0, frame=0, > over=0, crc=0**** > > tx pkts=470419, bytes=11241911122, drop=0, errs=0, coll=0**** > > port 1: rx pkts=14665755, bytes=623148271848, drop=0, errs=85734, > frame=0, over=76354, crc=1**** > > tx pkts=12988799, bytes=704654302, drop=0, errs=0, coll=0**** > > port 17: rx pkts=915354, bytes=49815016, drop=0, errs=0, frame=0, > over=0, crc=0**** > > tx pkts=919841, bytes=21658413792, drop=0, errs=0, coll=0**** > > port 12: rx pkts=1338285, bytes=72922342, drop=0, errs=0, frame=0, > over=0, crc=0**** > > tx pkts=1343566, bytes=27474695496, drop=0, errs=0, coll=0**** > > port 13: rx pkts=1338687, bytes=72961466, drop=0, errs=0, frame=0, > over=0, crc=0**** > > tx pkts=1343787, bytes=27303337788, drop=0, errs=0, coll=0**** > > port 11: rx pkts=277265, bytes=15097566, drop=0, errs=0, frame=0, > over=0, crc=0**** > > tx pkts=278658, bytes=6790283224, drop=0, errs=0, coll=0**** > > port 15: rx pkts=1015033, bytes=55206482, drop=0, errs=0, frame=0, > over=0, crc=0**** > > tx pkts=1021695, bytes=26252916396, drop=0, errs=0, coll=0**** > > port LOCAL: rx pkts=5624537, bytes=303747170, drop=0, errs=0, frame=0, > over=0, crc=0**** > > tx pkts=7262947, bytes=451601929606, drop=0, errs=0, coll=0**** > > ** ** > > Is there anything I can do to reduce errors and overruns? It seems the > QUEUE between Eth4 and the OVS interface gets congested and OVS can’t > switch traffic between too many VM’s at a very high speed. > What is the CPU usage like?
_______________________________________________ discuss mailing list [email protected] http://openvswitch.org/mailman/listinfo/discuss
