Hi, Flavio After I ran it repeatedly in different servers, I'm very sure it can't work on Ubuntu 16.04 kernel 4.15.0-55-generic and Upstream kernel 4.15.9, so can you tell me your kernel version when you ran run-iperf3.sh I provided ? I doubt this TSO patch for veth needs higher kernel version.
-----邮件原件----- 发件人: Flavio Leitner [mailto:f...@sysclose.org] 发送时间: 2020年2月20日 21:41 收件人: Yi Yang (杨燚)-云服务集团 <yangy...@inspur.com> 抄送: pkusunyif...@gmail.com; d...@openvswitch.org; i.maxim...@ovn.org; txfh2...@aliyun.com 主题: Re: 答复: [ovs-dev] [PATCH v4 0/3] Add support for TSO with DPDK On Thu, Feb 20, 2020 at 10:10:36AM +0000, Yi Yang (杨 D)-云服务集团 wrote: > Hi, Flavio > > I find this tso feature doesn't work normally on my Ubuntu 16.04, here > is my result. My kernel version is > > $ uname -a > Linux cmp008 4.15.0-55-generic #60~16.04.2-Ubuntu SMP Thu Jul 4 > 09:03:09 UTC > 2019 x86_64 x86_64 x86_64 GNU/Linux > $ I tested with 4.15.0 upstream and it worked. Can you do the same? > $ ./run-iperf3.sh > Connecting to host 10.15.1.3, port 5201 [ 4] local 10.15.1.2 port > 56466 connected to 10.15.1.3 port 5201 > [ ID] Interval Transfer Bandwidth Retr Cwnd > [ 4] 0.00-10.00 sec 7.05 MBytes 5.91 Mbits/sec 2212 5.66 KBytes > [ 4] 10.00-20.00 sec 7.67 MBytes 6.44 Mbits/sec 2484 5.66 KBytes > [ 4] 20.00-30.00 sec 7.77 MBytes 6.52 Mbits/sec 2500 5.66 KBytes > [ 4] 30.00-40.00 sec 7.77 MBytes 6.52 Mbits/sec 2490 5.66 KBytes > [ 4] 40.00-50.00 sec 7.76 MBytes 6.51 Mbits/sec 2500 5.66 KBytes > [ 4] 50.00-60.00 sec 7.79 MBytes 6.54 Mbits/sec 2504 5.66 KBytes > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bandwidth Retr > [ 4] 0.00-60.00 sec 45.8 MBytes 6.40 Mbits/sec 14690 > sender > [ 4] 0.00-60.00 sec 45.7 MBytes 6.40 Mbits/sec > receiver That looks like TSO packets are being dropped and the traffic is basically TCP retransmissions of MTU size. fbl > > Server output: > Accepted connection from 10.15.1.2, port 56464 [ 5] local 10.15.1.3 > port 5201 connected to 10.15.1.2 port 56466 > [ ID] Interval Transfer Bandwidth > [ 5] 0.00-10.00 sec 6.90 MBytes 5.79 Mbits/sec > [ 5] 10.00-20.00 sec 7.71 MBytes 6.47 Mbits/sec [ 5] > 20.00-30.00 sec 7.73 MBytes 6.48 Mbits/sec [ 5] 30.00-40.00 sec > 7.79 MBytes 6.53 Mbits/sec [ 5] 40.00-50.00 sec 7.79 MBytes 6.53 > Mbits/sec [ 5] 50.00-60.00 sec 7.79 MBytes 6.54 Mbits/sec > > > iperf Done. > $ > > But it does work for tap, I'm not sure if it is a kernel issue, which > kernel version are you using? I didn't use tpacket_v3 patch. Here is > my local ovs info. > > $ git log > commit 1223cf123ed141c0a0110ebed17572bdb2e3d0f4 > Author: Ilya Maximets <i.maxim...@ovn.org> > Date: Thu Feb 6 14:24:23 2020 +0100 > > netdev-dpdk: Don't enable offloading on HW device if not requested. > > DPDK drivers has different implementations of transmit functions. > Enabled offloading may cause driver to choose slower variant > significantly affecting performance if userspace TSO wasn't requested. > > Fixes: 29cf9c1b3b9c ("userspace: Add TCP Segmentation Offload support") > Reported-by: David Marchand <david.march...@redhat.com> > Acked-by: David Marchand <david.march...@redhat.com> > Acked-by: Flavio Leitner <f...@sysclose.org> > Acked-by: Kevin Traynor <ktray...@redhat.com> > Signed-off-by: Ilya Maximets <i.maxim...@ovn.org> > > commit 73858f9dbe83daf8cc8d4b604acc23eb62cc3f52 > Author: Flavio Leitner <f...@sysclose.org> > Date: Mon Feb 3 18:45:50 2020 -0300 > > netdev-linux: Prepend the std packet in the TSO packet > > Usually TSO packets are close to 50k, 60k bytes long, so to > to copy less bytes when receiving a packet from the kernel > change the approach. Instead of extending the MTU sized > packet received and append with remaining TSO data from > the TSO buffer, allocate a TSO packet with enough headroom > to prepend the std packet data. > > Fixes: 29cf9c1b3b9c ("userspace: Add TCP Segmentation Offload support") > Suggested-by: Ben Pfaff <b...@ovn.org> > Signed-off-by: Flavio Leitner <f...@sysclose.org> > Signed-off-by: Ben Pfaff <b...@ovn.org> > > commit 2297cbe6cc25b6b1862c499ce8f16f52f75d9e5f > Author: Flavio Leitner <f...@sysclose.org> > Date: Mon Feb 3 11:22:22 2020 -0300 > > netdev-linux-private: fix max length to be 16 bits > > The dp_packet length is limited to 16 bits, so document that > and fix the length value accordingly. > > Fixes: 29cf9c1b3b9c ("userspace: Add TCP Segmentation Offload support") > Signed-off-by: Flavio Leitner <f...@sysclose.org> > Signed-off-by: Ben Pfaff <b...@ovn.org> > > commit 3d6a6f450af5b7eaf4b532983cb14458ae792b72 > Author: David Marchand <david.march...@redhat.com> > Date: Tue Feb 4 22:28:26 2020 +0100 > > netdev-dpdk: Fix port init when lacking Tx offloads for TSO. > > The check on TSO capability did not ensure ip checksum, tcp checksum and > TSO tx offloads were available which resulted in a port init failure > (example below with a ena device): > > *2020-02-04T17:42:52.976Z|00084|dpdk|ERR|Ethdev port_id=0 requested Tx > offloads 0x2a doesn't match Tx offloads capabilities 0xe in > rte_eth_dev_configure()* > > Fixes: 29cf9c1b3b9c ("userspace: Add TCP Segmentation Offload > support") > > Reported-by: Ravi Kerur <rke...@gmail.com> > Signed-off-by: David Marchand <david.march...@redhat.com> > Acked-by: Kevin Traynor <ktray...@redhat.com> > Acked-by: Flavio Leitner <f...@sysclose.org> > Signed-off-by: Ilya Maximets <i.maxim...@ovn.org> > > commit 8e371aa497aa95e3562d53f566c2d634b4b0f589 > Author: Kirill A. Kornilov <kirill@tp> > Date: Mon Jan 13 12:29:10 2020 +0300 > > vswitchd: Add serial number configuration. > > Signed-off-by: Kirill A. Kornilov <korni...@zelax.ru> > Signed-off-by: Ben Pfaff <b...@ovn.org> > > I applied your tap patch. > > $ git diff > diff --git a/lib/netdev-linux.c b/lib/netdev-linux.c index > c6f3d27..74a5728 100644 > --- a/lib/netdev-linux.c > +++ b/lib/netdev-linux.c > @@ -1010,6 +1010,23 @@ netdev_linux_construct_tap(struct netdev *netdev_) > goto error_close; > } > > + if (userspace_tso_enabled()) { > + /* Old kernels don't support TUNSETOFFLOAD. If TUNSETOFFLOAD is > + * available, it will return EINVAL when a flag is unknown. > + * Therefore, try enabling offload with no flags to check > + * if TUNSETOFFLOAD support is available or not. */ > + if (ioctl(netdev->tap_fd, TUNSETOFFLOAD, 0) == 0 || errno != > EINVAL) { > + unsigned long oflags = TUN_F_CSUM | TUN_F_TSO4 | > + TUN_F_TSO6; > + > + if (ioctl(netdev->tap_fd, TUNSETOFFLOAD, oflags) == -1) { > + VLOG_WARN("%s: enabling tap offloading failed: %s", name, > + ovs_strerror(errno)); > + error = errno; > + goto error_close; > + } > + } > + } > + > netdev->present = true; > return 0; > > $ > > Here is performance result for tap. > > $ ./run-iperf3.sh > Connecting to host 10.15.1.3, port 5201 [ 4] local 10.15.1.2 port > 56480 connected to 10.15.1.3 port 5201 > [ ID] Interval Transfer Bandwidth Retr Cwnd > [ 4] 0.00-10.00 sec 19.4 GBytes 16.7 Gbits/sec 0 3.05 MBytes > [ 4] 10.00-20.00 sec 18.3 GBytes 15.7 Gbits/sec 0 3.05 MBytes > [ 4] 20.00-30.00 sec 17.6 GBytes 15.1 Gbits/sec 0 3.05 MBytes > [ 4] 30.00-40.00 sec 19.3 GBytes 16.6 Gbits/sec 0 3.05 MBytes > [ 4] 40.00-50.00 sec 18.8 GBytes 16.1 Gbits/sec 0 3.05 MBytes > [ 4] 50.00-60.00 sec 17.9 GBytes 15.4 Gbits/sec 0 3.05 MBytes > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bandwidth Retr > [ 4] 0.00-60.00 sec 111 GBytes 15.9 Gbits/sec 0 sender > [ 4] 0.00-60.00 sec 111 GBytes 15.9 Gbits/sec > receiver > > Server output: > Accepted connection from 10.15.1.2, port 56478 [ 5] local 10.15.1.3 > port 5201 connected to 10.15.1.2 port 56480 > [ ID] Interval Transfer Bandwidth > [ 5] 0.00-10.00 sec 19.3 GBytes 16.6 Gbits/sec > [ 5] 10.00-20.00 sec 18.3 GBytes 15.7 Gbits/sec [ 5] > 20.00-30.00 sec 17.6 GBytes 15.1 Gbits/sec [ 5] 30.00-40.00 sec > 19.3 GBytes 16.6 Gbits/sec [ 5] 40.00-50.00 sec 18.8 GBytes 16.1 > Gbits/sec [ 5] 50.00-60.00 sec 17.9 GBytes 15.4 Gbits/sec > > > iperf Done. > $ -- fbl _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev