Abbreviated as TSO, TCP Segmentation Offload is a feature which enables the network stack to delegate the TCP segmentation to the NIC reducing the per packet CPU overhead.
A guest using vhost-user interface with TSO enabled can send TCP packets much bigger than the MTU, which saves CPU cycles normally used to break the packets down to MTU size and to calculate checksums. It also saves CPU cycles used to parse multiple packets/headers during the packet processing inside virtual switch. If the destination of the packet is another guest in the same host, then the same big packet can be sent through a vhost-user interface skipping the segmentation completely. However, if the destination is not local, the NIC hardware is instructed to do the TCP segmentation and checksum calculation. The first 2 patches are not really part of TSO support, but they are required to make sure everything works. There are good improvements sending to or receiving from veth pairs or tap devices as well. See the iperf3 results below: [*] veth with ethtool tx off. VM sending to: Default Enabled Enabled/Default Local BR 3 Gbits/sec 23 Gbits/sec 7x Net NS (veth) 3 Gbits/sec[*] 22 Gbits/sec 7x VM (same host) 2.5 Gbits/sec 24 Gbits/sec 9x Ext Host 10 Gbits/sec 35 Gbits/sec 3x Ext Host (vxlan) 8.8 Gbits/sec (not supported) Using VLAN: Local BR 3 Gbits/sec 23 Gbits/sec 7x VM (same host) 2.5 Gbits/sec 21 Gbits/sec 8x Ext Host 6.4 Gbits/sec 34 Gbits/sec 5x Using IPv6: Net NS (veth) 2.7 Gbits/sec[*] 22 Gbits/sec 8x VM (same host) 2.6 Gbits/sec 21 Gbits/sec 8x Ext Host 8.7 Gbits/sec 34 Gbits/sec 4x Conntrack: No packet changes: 1.41 Gbits/sec 33 Gbits/sec 23x VM receiving from: Local BR 2.5 Gbits/sec 2.4 Gbits/sec 1x Net NS (veth) 2.5 Gbits/sec[*] 9.3 Gbits/sec 3x VM (same host) 4.9 Gbits/sec 25 Gbits/sec 5x Ext Host 9.7 Gbits/sec 9.4 Gbits/sec 1x Ext Host (vxlan) 5.5 Gbits/sec (not supported) Using VLAN: Local BR 2.4 Gbits/sec 2.4 Gbits/sec 1x VM (same host) 3.8 Gbits/sec 24 Gbits/sec 8x Ext Host 9.5 Gbits/sec 9.5 Gbits/sec 1x Using IPv6: Net NS (veth) 2.2 Gbits/sec[*] 9 Gbits/sec 4x VM (same host) 4.5 Gbits/sec 24 Gbits/sec 5x Ext Host 8.9 Gbits/sec 8.9 Gbits/sec 1x Used iperf3 -u to test UDP traffic limited at default 1Mbits/sec and noticed no change with the exception for tunneled packets (not supported). Travis, AppVeyor, and Cirrus-ci passed. Flavio Leitner (3): dp-packet: preserve headroom when cloning a pkt batch vhost: Disable multi-segmented buffers netdev-dpdk: Add TCP Segmentation Offload support Documentation/automake.mk | 1 + Documentation/topics/index.rst | 1 + Documentation/topics/userspace-tso.rst | 98 +++++++ NEWS | 1 + lib/automake.mk | 2 + lib/conntrack.c | 29 +- lib/dp-packet.h | 192 +++++++++++- lib/ipf.c | 32 +- lib/netdev-dpdk.c | 355 ++++++++++++++++++++--- lib/netdev-linux-private.h | 5 + lib/netdev-linux.c | 386 ++++++++++++++++++++++--- lib/netdev-provider.h | 9 + lib/netdev.c | 78 ++++- lib/userspace-tso.c | 48 +++ lib/userspace-tso.h | 23 ++ vswitchd/bridge.c | 2 + vswitchd/vswitch.xml | 17 ++ 17 files changed, 1154 insertions(+), 125 deletions(-) create mode 100644 Documentation/topics/userspace-tso.rst create mode 100644 lib/userspace-tso.c create mode 100644 lib/userspace-tso.h -- 2.24.1 _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
