Yes, I have done review on this patch and gave comments back on it a while ago. I also have done a lot of testing as a result of the performance analysis I made for the PV and PVP cases. These tests includes full integration of the feature. Meaning the hw-offload follows the add/deletion of flows in the megaflow cache. So, add and deletion I a single/multi-queue setup has been tested. Furthermore, I'm working on this patchset for continued development, so it is continuously being used by me as a basic functionality. So far, it has been running stable.
Finn >-----Original Message----- >From: Yuanhan Liu [mailto:y...@fridaylinux.org] >Sent: 18. januar 2018 04:39 >To: Ian Stokes <ian.sto...@intel.com> >Cc: Finn Christensen <f...@napatech.com>; d...@openvswitch.org; Darrell Ball ><db...@vmware.com>; Chandran Sugesh <sugesh.chand...@intel.com>; >Simon Horman <simon.hor...@netronome.com>; Thomas Monjalon ><tho...@monjalon.net>; Olga Shern <ol...@mellanox.com>; Shahaf Shuler ><shah...@mellanox.com> >Subject: Re: [PATCH v5 0/5] OVS-DPDK flow offload with rte_flow > >I was told more tests and reviews are needed from yesterday's meeting. > >For testing, as you can see below, Finn already have done some performance >testing covering quite few testcases. I have also done some very basic >performance testing, basically just with rxonly. > >Besides that, I have actually spent a lot of effort for doing functional >testing, >to make sure it doesn't break anything. The limited testing >including: > >- 1 queue >- 2 queues >- 1 flow >- 1000 flows >- adding/deleting flows without traffic >- adding/deleting flows with traffic >- keep adding and then deleting flows with traffic > >For review, I think Finn from Napatech should also have done it. He has >provided some valuable comments after all :) > >Thanks. > > --yliu > >On Fri, Dec 22, 2017 at 11:28:45AM +0000, Finn Christensen wrote: >> Hi, >> >> The addition of the offload thread is an great addition, furthermore, RSS >action usage for target queues, works now well with our NICs, I just had to do >some additional drive work, to make it all work. >> >> The support for RSS removed the problem with rte-flow queue selection, >but RSS in general added an additional challenge as now multiple pmd's may >request the same flow offload (as Yuanhan pointed out). The solution has >worked without issues for me. I have run tests with 1 and 2 queues in a PV >and PVP setup going over virtio. >> Yuanhan mentioned tests in a P2P setup with high throughput acceleration. I >have focused on a PVP scenario, which, in addition, will crosses virtio, and >most importantly, has acceleration in Rx direction only (seen from VM). It >still >gives fairly good performance improvements. >> >> Here are my initial test results. >> >> Percentage improvements: >> 1 RX/Tx queue: >> 1 megaflow - 10K flowsPV: 73% >> 10 megaflows - 100K flowsPVP: 50%PV: 56% >> 100 megaflows - 1M flowsPVP: 42%PV: 49% >> 1000 megaflows - 10M flowsPVP: 40%PV: 42% >> >> With RSS: 2 Rx/Tx queue pairs: >> 1 megaflow - 10K flowsPV: 55% >> 10 megaflows - 100K flowsPVP: 36%PV: 38% >> 100 megaflows - 1M flowsPVP: 27%PV: 24% >> 1000 megaflows - 10M flowsPVP: 21%PV: 24% >> >> PVP for 1 megaflow offload is omitted because my VM-app imposed a >bottle-neck. >> >> Regards, >> Finn >> >> -----Original Message----- >> From: Yuanhan Liu [mailto:y...@fridaylinux.org] >> Sent: 20. december 2017 15:45 >> To: d...@openvswitch.org >> Cc: Finn Christensen <f...@napatech.com>; Darrell Ball >> <db...@vmware.com>; Chandran Sugesh <sugesh.chand...@intel.com>; >> Simon Horman <simon.hor...@netronome.com>; Ian Stokes >> <ian.sto...@intel.com>; Yuanhan Liu <y...@fridaylinux.org> >> Subject: [PATCH v5 0/5] OVS-DPDK flow offload with rte_flow >> >> Hi, >> >> Here is a joint work from Mellanox and Napatech, to enable the flow hw >> offload with the DPDK generic flow interface (rte_flow). >> >> The basic idea is to associate the flow with a mark id (a unit32_t >> number). >> Later, we then get the flow directly from the mark id, which could bypass >> some heavy CPU operations, including but not limiting to mini flow >> extract, >> emc lookup, dpcls lookup, etc. >> >> The association is done with CMAP in patch 1. The CPU workload >bypassing >> is done in patch 2. The flow offload is done in patch 3, which mainly >> does >> two things: >> >> - translate the ovs match to DPDK rte flow patterns >> - bind those patterns with a RSS + MARK action. >> >> Patch 5 makes the offload work happen in another thread, for leaving the >> datapath as light as possible. >> >> A PHY-PHY forwarding with 1000 mega flows (udp,tp_src=1000-1999) and 1 >> million streams (tp_src=1000-1999, tp_dst=2000-2999) show more than >> 260% performance boost. >> >> Note that it's disabled by default, which can be enabled by: >> >> $ ovs-vsctl set Open_vSwitch . other_config:hw-offload=true >> >> >> v5: - fixed an issue that it took too long if we do flow add/remove >> repeatedly. >> - removed an unused mutex lock >> - turned most of the log level to DBG >> - rebased on top of the latest code >> >> v4: - use RSS action instead of QUEUE action with MARK >> - make it work with multiple queue (see patch 1) >> - rebased on top of latest code >> >> v3: - The mark and id association is done with array instead of CMAP. >> - Added a thread to do hw offload operations >> - Removed macros completely >> - dropped the patch to set FDIR_CONF, which is a workround some >> Intel NICs. >> - Added a debug patch to show all flow patterns we have created. >> - Misc fixes >> >> v2: - workaround the queue action issue >> - fixed the tcp_flags being skipped issue, which also fixed the >> build warnings >> - fixed l2 patterns for Intel nic >> - Converted some macros to functions >> - did not hardcode the max number of flow/action >> - rebased on top of the latest code >> >> Thanks. >> >> --yliu >> >> >> --- >> Finn Christensen (1): >> netdev-dpdk: implement flow offload with rte flow >> >> Yuanhan Liu (4): >> dpif-netdev: associate flow with a mark id >> dpif-netdev: retrieve flow directly from the flow mark >> netdev-dpdk: add debug for rte flow patterns >> dpif-netdev: do hw flow offload in a thread >> >> lib/dp-packet.h | 13 + >> lib/dpif-netdev.c | 473 ++++++++++++++++++++++++++++++++++- >> lib/flow.c | 155 +++++++++--- >> lib/flow.h | 1 + >> lib/netdev-dpdk.c | 732 >> +++++++++++++++++++++++++++++++++++++++++++++++++++++- >> lib/netdev.h | 6 + >> 6 files changed, 1341 insertions(+), 39 deletions(-) >> >> -- >> 2.7.4 >> >> Disclaimer: This email and any files transmitted with it may contain >confidential information intended for the addressee(s) only. The information >is not to be surrendered or copied to unauthorized persons. If you have >received this communication in error, please notify the sender immediately >and delete this e-mail from your system. _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
