On Sun, Feb 23, 2020 at 08:45:16AM +0000, Yi Yang (杨燚)-云服务集团 wrote: > Hi, Flavio > > Just let you know, your TSO support patch does need higher kernel version, it > will be great if you can add document to tell users which kernel version is > minimal requirement. I can confirm it can work after I used Ubuntu 18.04 and > use kernel 5.3.0-40-generic. > > vagrant@ubuntu1804:~$ uname -a > Linux ubuntu1804 5.3.0-40-generic #32~18.04.1-Ubuntu SMP Mon Feb 3 14:05:59 > UTC 2020 x86_64 x86_64 x86_64 GNU/Linux > vagrant@ubuntu1804:~$
OK, I was able to reproduce the bad tput with veth-ovs-veth with TSO on 4.15.0 upstream and I am going to dig deeper now. Thanks, fbl > > By the way, TPACKET_V3 also can support TSO without any new change needed, so > I think TPACKET_V3 can work normally with userspace-tso-enable=true as long > as your TSO patch can work normally for veth-to-veth case. > > I'll send out my tpacket patch v5 for review. > > -----邮件原件----- > 发件人: Yi Yang (杨燚)-云服务集团 > 发送时间: 2020年2月23日 11:05 > 收件人: 'f...@sysclose.org' <f...@sysclose.org> > 抄送: 'pkusunyif...@gmail.com' <pkusunyif...@gmail.com>; 'd...@openvswitch.org' > <d...@openvswitch.org>; 'i.maxim...@ovn.org' <i.maxim...@ovn.org>; > 'txfh2...@aliyun.com' <txfh2...@aliyun.com> > 主题: 答复: 答复: [ovs-dev] [PATCH v4 0/3] Add support for TSO with DPDK > 重要性: 高 > > Hi, Flavio > > After I ran it repeatedly in different servers, I'm very sure it can't work > on Ubuntu 16.04 kernel 4.15.0-55-generic and Upstream kernel 4.15.9, so can > you tell me your kernel version when you ran run-iperf3.sh I provided ? I > doubt this TSO patch for veth needs higher kernel version. > > -----邮件原件----- > 发件人: Flavio Leitner [mailto:f...@sysclose.org] > 发送时间: 2020年2月20日 21:41 > 收件人: Yi Yang (杨燚)-云服务集团 <yangy...@inspur.com> > 抄送: pkusunyif...@gmail.com; d...@openvswitch.org; i.maxim...@ovn.org; > txfh2...@aliyun.com > 主题: Re: 答复: [ovs-dev] [PATCH v4 0/3] Add support for TSO with DPDK > > On Thu, Feb 20, 2020 at 10:10:36AM +0000, Yi Yang (杨 D)-云服务集团 wrote: > > Hi, Flavio > > > > I find this tso feature doesn't work normally on my Ubuntu 16.04, here > > is my result. My kernel version is > > > > $ uname -a > > Linux cmp008 4.15.0-55-generic #60~16.04.2-Ubuntu SMP Thu Jul 4 > > 09:03:09 UTC > > 2019 x86_64 x86_64 x86_64 GNU/Linux > > $ > > I tested with 4.15.0 upstream and it worked. Can you do the same? > > > $ ./run-iperf3.sh > > Connecting to host 10.15.1.3, port 5201 [ 4] local 10.15.1.2 port > > 56466 connected to 10.15.1.3 port 5201 > > [ ID] Interval Transfer Bandwidth Retr Cwnd > > [ 4] 0.00-10.00 sec 7.05 MBytes 5.91 Mbits/sec 2212 5.66 KBytes > > [ 4] 10.00-20.00 sec 7.67 MBytes 6.44 Mbits/sec 2484 5.66 KBytes > > [ 4] 20.00-30.00 sec 7.77 MBytes 6.52 Mbits/sec 2500 5.66 KBytes > > [ 4] 30.00-40.00 sec 7.77 MBytes 6.52 Mbits/sec 2490 5.66 KBytes > > [ 4] 40.00-50.00 sec 7.76 MBytes 6.51 Mbits/sec 2500 5.66 KBytes > > [ 4] 50.00-60.00 sec 7.79 MBytes 6.54 Mbits/sec 2504 5.66 KBytes > > - - - - - - - - - - - - - - - - - - - - - - - - - > > [ ID] Interval Transfer Bandwidth Retr > > [ 4] 0.00-60.00 sec 45.8 MBytes 6.40 Mbits/sec 14690 > > sender > > [ 4] 0.00-60.00 sec 45.7 MBytes 6.40 Mbits/sec > > receiver > > That looks like TSO packets are being dropped and the traffic is basically > TCP retransmissions of MTU size. > > fbl > > > > > > Server output: > > Accepted connection from 10.15.1.2, port 56464 [ 5] local 10.15.1.3 > > port 5201 connected to 10.15.1.2 port 56466 > > [ ID] Interval Transfer Bandwidth > > [ 5] 0.00-10.00 sec 6.90 MBytes 5.79 Mbits/sec > > [ 5] 10.00-20.00 sec 7.71 MBytes 6.47 Mbits/sec [ 5] > > 20.00-30.00 sec 7.73 MBytes 6.48 Mbits/sec [ 5] 30.00-40.00 sec > > 7.79 MBytes 6.53 Mbits/sec [ 5] 40.00-50.00 sec 7.79 MBytes 6.53 > > Mbits/sec [ 5] 50.00-60.00 sec 7.79 MBytes 6.54 Mbits/sec > > > > > > iperf Done. > > $ > > > > But it does work for tap, I'm not sure if it is a kernel issue, which > > kernel version are you using? I didn't use tpacket_v3 patch. Here is > > my local ovs info. > > > > $ git log > > commit 1223cf123ed141c0a0110ebed17572bdb2e3d0f4 > > Author: Ilya Maximets <i.maxim...@ovn.org> > > Date: Thu Feb 6 14:24:23 2020 +0100 > > > > netdev-dpdk: Don't enable offloading on HW device if not requested. > > > > DPDK drivers has different implementations of transmit functions. > > Enabled offloading may cause driver to choose slower variant > > significantly affecting performance if userspace TSO wasn't requested. > > > > Fixes: 29cf9c1b3b9c ("userspace: Add TCP Segmentation Offload support") > > Reported-by: David Marchand <david.march...@redhat.com> > > Acked-by: David Marchand <david.march...@redhat.com> > > Acked-by: Flavio Leitner <f...@sysclose.org> > > Acked-by: Kevin Traynor <ktray...@redhat.com> > > Signed-off-by: Ilya Maximets <i.maxim...@ovn.org> > > > > commit 73858f9dbe83daf8cc8d4b604acc23eb62cc3f52 > > Author: Flavio Leitner <f...@sysclose.org> > > Date: Mon Feb 3 18:45:50 2020 -0300 > > > > netdev-linux: Prepend the std packet in the TSO packet > > > > Usually TSO packets are close to 50k, 60k bytes long, so to > > to copy less bytes when receiving a packet from the kernel > > change the approach. Instead of extending the MTU sized > > packet received and append with remaining TSO data from > > the TSO buffer, allocate a TSO packet with enough headroom > > to prepend the std packet data. > > > > Fixes: 29cf9c1b3b9c ("userspace: Add TCP Segmentation Offload support") > > Suggested-by: Ben Pfaff <b...@ovn.org> > > Signed-off-by: Flavio Leitner <f...@sysclose.org> > > Signed-off-by: Ben Pfaff <b...@ovn.org> > > > > commit 2297cbe6cc25b6b1862c499ce8f16f52f75d9e5f > > Author: Flavio Leitner <f...@sysclose.org> > > Date: Mon Feb 3 11:22:22 2020 -0300 > > > > netdev-linux-private: fix max length to be 16 bits > > > > The dp_packet length is limited to 16 bits, so document that > > and fix the length value accordingly. > > > > Fixes: 29cf9c1b3b9c ("userspace: Add TCP Segmentation Offload support") > > Signed-off-by: Flavio Leitner <f...@sysclose.org> > > Signed-off-by: Ben Pfaff <b...@ovn.org> > > > > commit 3d6a6f450af5b7eaf4b532983cb14458ae792b72 > > Author: David Marchand <david.march...@redhat.com> > > Date: Tue Feb 4 22:28:26 2020 +0100 > > > > netdev-dpdk: Fix port init when lacking Tx offloads for TSO. > > > > The check on TSO capability did not ensure ip checksum, tcp checksum and > > TSO tx offloads were available which resulted in a port init failure > > (example below with a ena device): > > > > *2020-02-04T17:42:52.976Z|00084|dpdk|ERR|Ethdev port_id=0 requested Tx > > offloads 0x2a doesn't match Tx offloads capabilities 0xe in > > rte_eth_dev_configure()* > > > > Fixes: 29cf9c1b3b9c ("userspace: Add TCP Segmentation Offload > > support") > > > > Reported-by: Ravi Kerur <rke...@gmail.com> > > Signed-off-by: David Marchand <david.march...@redhat.com> > > Acked-by: Kevin Traynor <ktray...@redhat.com> > > Acked-by: Flavio Leitner <f...@sysclose.org> > > Signed-off-by: Ilya Maximets <i.maxim...@ovn.org> > > > > commit 8e371aa497aa95e3562d53f566c2d634b4b0f589 > > Author: Kirill A. Kornilov <kirill@tp> > > Date: Mon Jan 13 12:29:10 2020 +0300 > > > > vswitchd: Add serial number configuration. > > > > Signed-off-by: Kirill A. Kornilov <korni...@zelax.ru> > > Signed-off-by: Ben Pfaff <b...@ovn.org> > > > > I applied your tap patch. > > > > $ git diff > > diff --git a/lib/netdev-linux.c b/lib/netdev-linux.c index > > c6f3d27..74a5728 100644 > > --- a/lib/netdev-linux.c > > +++ b/lib/netdev-linux.c > > @@ -1010,6 +1010,23 @@ netdev_linux_construct_tap(struct netdev *netdev_) > > goto error_close; > > } > > > > + if (userspace_tso_enabled()) { > > + /* Old kernels don't support TUNSETOFFLOAD. If TUNSETOFFLOAD is > > + * available, it will return EINVAL when a flag is unknown. > > + * Therefore, try enabling offload with no flags to check > > + * if TUNSETOFFLOAD support is available or not. */ > > + if (ioctl(netdev->tap_fd, TUNSETOFFLOAD, 0) == 0 || errno != > > EINVAL) { > > + unsigned long oflags = TUN_F_CSUM | TUN_F_TSO4 | > > + TUN_F_TSO6; > > + > > + if (ioctl(netdev->tap_fd, TUNSETOFFLOAD, oflags) == -1) { > > + VLOG_WARN("%s: enabling tap offloading failed: %s", name, > > + ovs_strerror(errno)); > > + error = errno; > > + goto error_close; > > + } > > + } > > + } > > + > > netdev->present = true; > > return 0; > > > > $ > > > > Here is performance result for tap. > > > > $ ./run-iperf3.sh > > Connecting to host 10.15.1.3, port 5201 [ 4] local 10.15.1.2 port > > 56480 connected to 10.15.1.3 port 5201 > > [ ID] Interval Transfer Bandwidth Retr Cwnd > > [ 4] 0.00-10.00 sec 19.4 GBytes 16.7 Gbits/sec 0 3.05 MBytes > > [ 4] 10.00-20.00 sec 18.3 GBytes 15.7 Gbits/sec 0 3.05 MBytes > > [ 4] 20.00-30.00 sec 17.6 GBytes 15.1 Gbits/sec 0 3.05 MBytes > > [ 4] 30.00-40.00 sec 19.3 GBytes 16.6 Gbits/sec 0 3.05 MBytes > > [ 4] 40.00-50.00 sec 18.8 GBytes 16.1 Gbits/sec 0 3.05 MBytes > > [ 4] 50.00-60.00 sec 17.9 GBytes 15.4 Gbits/sec 0 3.05 MBytes > > - - - - - - - - - - - - - - - - - - - - - - - - - > > [ ID] Interval Transfer Bandwidth Retr > > [ 4] 0.00-60.00 sec 111 GBytes 15.9 Gbits/sec 0 sender > > [ 4] 0.00-60.00 sec 111 GBytes 15.9 Gbits/sec > > receiver > > > > Server output: > > Accepted connection from 10.15.1.2, port 56478 [ 5] local 10.15.1.3 > > port 5201 connected to 10.15.1.2 port 56480 > > [ ID] Interval Transfer Bandwidth > > [ 5] 0.00-10.00 sec 19.3 GBytes 16.6 Gbits/sec > > [ 5] 10.00-20.00 sec 18.3 GBytes 15.7 Gbits/sec [ 5] > > 20.00-30.00 sec 17.6 GBytes 15.1 Gbits/sec [ 5] 30.00-40.00 sec > > 19.3 GBytes 16.6 Gbits/sec [ 5] 40.00-50.00 sec 18.8 GBytes 16.1 > > Gbits/sec [ 5] 50.00-60.00 sec 17.9 GBytes 15.4 Gbits/sec > > > > > > iperf Done. > > $ > > > > -- > fbl -- fbl _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
