Hi, all Current tap, internal and system interfaces aren't handled by pmd_thread, so the performance can't be boosted too high, I have a very simple test just by setting is_pmd to true for them, the below is my data for veth (using TPACKET_V3), you can see pmd_thread is much better than ovs_vswitchd obviously, compared with my previous data 1.98Gbps, my question is if we can set is_pmd to true by default, I'll set is_pmd to true in next version if no objection.
$ sudo ip netns exec ns01 iperf3 -t 60 -i 10 -c 10.15.1.3 --get-server-output Connecting to host 10.15.1.3, port 5201 [ 4] local 10.15.1.2 port 59590 connected to 10.15.1.3 port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 4] 0.00-10.00 sec 3.59 GBytes 3.09 Gbits/sec 0 3.04 MBytes [ 4] 10.00-20.00 sec 3.57 GBytes 3.06 Gbits/sec 0 3.04 MBytes [ 4] 20.00-30.00 sec 3.60 GBytes 3.09 Gbits/sec 0 3.04 MBytes [ 4] 30.00-40.00 sec 3.56 GBytes 3.06 Gbits/sec 0 3.04 MBytes [ 4] 40.00-50.00 sec 3.64 GBytes 3.12 Gbits/sec 0 3.04 MBytes [ 4] 50.00-60.00 sec 3.62 GBytes 3.11 Gbits/sec 0 3.04 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-60.00 sec 21.6 GBytes 3.09 Gbits/sec 0 sender [ 4] 0.00-60.00 sec 21.6 GBytes 3.09 Gbits/sec receiver Server output: ----------------------------------------------------------- Accepted connection from 10.15.1.2, port 59588 [ 5] local 10.15.1.3 port 5201 connected to 10.15.1.2 port 59590 [ ID] Interval Transfer Bandwidth [ 5] 0.00-10.00 sec 3.57 GBytes 3.07 Gbits/sec [ 5] 10.00-20.00 sec 3.57 GBytes 3.06 Gbits/sec [ 5] 20.00-30.00 sec 3.60 GBytes 3.09 Gbits/sec [ 5] 30.00-40.00 sec 3.56 GBytes 3.06 Gbits/sec [ 5] 40.00-50.00 sec 3.64 GBytes 3.12 Gbits/sec [ 5] 50.00-60.00 sec 3.62 GBytes 3.11 Gbits/sec iperf Done. eipadmin@cmp008:~$ -----邮件原件----- 发件人: Ben Pfaff [mailto:[email protected]] 发送时间: 2020年1月22日 3:26 收件人: [email protected] 抄送: [email protected]; [email protected]; Yi Yang (杨�D)-云服务集 团 <[email protected]> 主题: Re: [PATCH] Use TPACKET_V1/V2/V3 to accelerate veth for DPDK datapath On Tue, Jan 21, 2020 at 02:49:47AM -0500, [email protected] wrote: > From: Yi Yang <[email protected]> > > We can avoid high system call overhead by using TPACKET_V1/V2/V3 and > use DPDK-like poll to receive and send packets (Note: send still needs > to call sendto to trigger final packet transmission). > > I can see about 30% improvement compared to last recvmmsg optimization > if I use TPACKET_V3. TPACKET_V1/V2 is worse than TPACKET_V3, but it > still can improve about 20%. > > For veth, it is 1.47 Gbps before this patch, it is about 1.98 Gbps > after applied this patch. But it is about 4.00 Gbps if we use > af_packet for veth, the bottle neck lies in ovs-vswitchd thread, it > will handle too many things for every loop (as below) , so it can't > work very efficintly as pmd_thread. > > memory_run(); > bridge_run(); > unixctl_server_run(unixctl); > netdev_run(); > > memory_wait(); > bridge_wait(); > unixctl_server_wait(unixctl); > netdev_wait(); > poll_block(); > > In the next step, it will be better if let pmd_thread to handle tap > and veth interface. > > Signed-off-by: Yi Yang <[email protected]> > Co-authored-by: William Tu <[email protected]> > Signed-off-by: William Tu <[email protected]> Thanks for the patch! I am a bit concerned about version compatibility issues here. There are two relevant kinds of versions. The first is the version of the kernel/library headers. This patch works pretty hard to adapt to the headers that are available at compile time, only dealing with the versions of the protocols that are available from the headers. This approach is sometimes fine, but an approach can be better is to simply declare the structures or constants that the headers lack. This is often pretty easy for Linux data structures. OVS does this for some structures that it cares about with the headers in ovs/include/linux. This approach has two advantages: the OVS code (outside these special declarations) doesn't have to care whether particular structures are declared, because they are always declared, and the OVS build always supports a particular feature regardless of the headers of the system on which it was built. The second kind of version is the version of the system that OVS runs on. Unless a given feature is one that is supported by every version that OVS cares about, OVS needs to test at runtime whether the feature is supported and, if not, fall back to the older feature. I don't see that in this code. Instead, it looks to me like it assumes that if the feature was available at build time, then it is available at runtime. This is not a good way to do things, since we want people to be able to get builds from distributors such as Red Hat or Debian and then run those builds on a diverse collection of kernels. One specific comment I have here is that, in acinclude.m4, it would be better to use AC_CHECK_TYPE or AC_CHECK_TYPES thatn OVS_GREP_IFELSE. The latter is for testing for kernel builds only; we can't use the normal AC_* tests for those because we often can't successfully build kernel headers using the compiler and flags that Autoconf sets up for building OVS. Thanks, Ben.
_______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
