Hi Yuanhan, Thank you for working on this. I will be reviewing this patch when I am back office after my Xmas/New Year vacation. I do have some comments as well as the questions on how this proposal is going to work with the different hardwares, full acceleration, and other partial offload options. Lets have that conversation after the holidays and see whats the best way to get it work for most of the cases.
Regards _Sugesh > -----Original Message----- > From: Finn Christensen [mailto:f...@napatech.com] > Sent: Friday, December 22, 2017 11:29 AM > To: Yuanhan Liu <y...@fridaylinux.org>; d...@openvswitch.org > Cc: Darrell Ball <db...@vmware.com>; Chandran, Sugesh > <sugesh.chand...@intel.com>; Simon Horman > <simon.hor...@netronome.com>; Stokes, Ian <ian.sto...@intel.com> > Subject: RE: [PATCH v5 0/5] OVS-DPDK flow offload with rte_flow > > Hi, > > The addition of the offload thread is an great addition, furthermore, RSS > action > usage for target queues, works now well with our NICs, I just had to do some > additional drive work, to make it all work. > > The support for RSS removed the problem with rte-flow queue selection, but RSS > in general added an additional challenge as now multiple pmd's may request the > same flow offload (as Yuanhan pointed out). The solution has worked without > issues for me. I have run tests with 1 and 2 queues in a PV and PVP setup > going > over virtio. > Yuanhan mentioned tests in a P2P setup with high throughput acceleration. I > have focused on a PVP scenario, which, in addition, will crosses virtio, and > most > importantly, has acceleration in Rx direction only (seen from VM). It still > gives > fairly good performance improvements. > > Here are my initial test results. > > Percentage improvements: > 1 RX/Tx queue: > 1 megaflow - 10K flowsPV: 73% > 10 megaflows - 100K flowsPVP: 50%PV: 56% > 100 megaflows - 1M flowsPVP: 42%PV: 49% > 1000 megaflows - 10M flowsPVP: 40%PV: 42% > > With RSS: 2 Rx/Tx queue pairs: > 1 megaflow - 10K flowsPV: 55% > 10 megaflows - 100K flowsPVP: 36%PV: 38% > 100 megaflows - 1M flowsPVP: 27%PV: 24% > 1000 megaflows - 10M flowsPVP: 21%PV: 24% > > PVP for 1 megaflow offload is omitted because my VM-app imposed a bottle- > neck. > > Regards, > Finn > > -----Original Message----- > From: Yuanhan Liu [mailto:y...@fridaylinux.org] > Sent: 20. december 2017 15:45 > To: d...@openvswitch.org > Cc: Finn Christensen <f...@napatech.com>; Darrell Ball > <db...@vmware.com>; Chandran Sugesh <sugesh.chand...@intel.com>; > Simon Horman <simon.hor...@netronome.com>; Ian Stokes > <ian.sto...@intel.com>; Yuanhan Liu <y...@fridaylinux.org> > Subject: [PATCH v5 0/5] OVS-DPDK flow offload with rte_flow > > Hi, > > Here is a joint work from Mellanox and Napatech, to enable the flow hw > offload with the DPDK generic flow interface (rte_flow). > > The basic idea is to associate the flow with a mark id (a unit32_t > number). > Later, we then get the flow directly from the mark id, which could bypass > some heavy CPU operations, including but not limiting to mini flow > extract, > emc lookup, dpcls lookup, etc. > > The association is done with CMAP in patch 1. The CPU workload bypassing > is done in patch 2. The flow offload is done in patch 3, which mainly does > two things: > > - translate the ovs match to DPDK rte flow patterns > - bind those patterns with a RSS + MARK action. > > Patch 5 makes the offload work happen in another thread, for leaving the > datapath as light as possible. > > A PHY-PHY forwarding with 1000 mega flows (udp,tp_src=1000-1999) and 1 > million streams (tp_src=1000-1999, tp_dst=2000-2999) show more than > 260% performance boost. > > Note that it's disabled by default, which can be enabled by: > > $ ovs-vsctl set Open_vSwitch . other_config:hw-offload=true > > > v5: - fixed an issue that it took too long if we do flow add/remove > repeatedly. > - removed an unused mutex lock > - turned most of the log level to DBG > - rebased on top of the latest code > > v4: - use RSS action instead of QUEUE action with MARK > - make it work with multiple queue (see patch 1) > - rebased on top of latest code > > v3: - The mark and id association is done with array instead of CMAP. > - Added a thread to do hw offload operations > - Removed macros completely > - dropped the patch to set FDIR_CONF, which is a workround some > Intel NICs. > - Added a debug patch to show all flow patterns we have created. > - Misc fixes > > v2: - workaround the queue action issue > - fixed the tcp_flags being skipped issue, which also fixed the > build warnings > - fixed l2 patterns for Intel nic > - Converted some macros to functions > - did not hardcode the max number of flow/action > - rebased on top of the latest code > > Thanks. > > --yliu > > > --- > Finn Christensen (1): > netdev-dpdk: implement flow offload with rte flow > > Yuanhan Liu (4): > dpif-netdev: associate flow with a mark id > dpif-netdev: retrieve flow directly from the flow mark > netdev-dpdk: add debug for rte flow patterns > dpif-netdev: do hw flow offload in a thread > > lib/dp-packet.h | 13 + > lib/dpif-netdev.c | 473 ++++++++++++++++++++++++++++++++++- > lib/flow.c | 155 +++++++++--- > lib/flow.h | 1 + > lib/netdev-dpdk.c | 732 > +++++++++++++++++++++++++++++++++++++++++++++++++++++- > lib/netdev.h | 6 + > 6 files changed, 1341 insertions(+), 39 deletions(-) > > -- > 2.7.4 > > Disclaimer: This email and any files transmitted with it may contain > confidential > information intended for the addressee(s) only. The information is not to be > surrendered or copied to unauthorized persons. If you have received this > communication in error, please notify the sender immediately and delete this > e- > mail from your system. _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
