Hi Yuanhan, Thank you for sending out the patch series.
We are also looking into something similar to enable full offload in OVS-DPDK. It is based on ' http://dpdk.org/ml/archives/dev/2017-September/074746.html' and some other rte_flow extension in DPDK. It is noted that the patch series doesn't work very well for some of our requirements. Please find below for the high level comments. I have also provided specific comments on the following patches. 1) Looks to me the patch series is to enable/use just one functionality of NIC(mark action). In a multiple hardware environment it is necessary to have a feature discovery mechanism to define what needs to be installed in the hardware based on its capabilities, for eg: MARK+QUEUE, MARK only, number of supported flow entries, supported flow fields and etc. This is very important to support different hardware NICs and make flow install easy. In our implementation we have a feature discovery at the OVS init. It will also populate the OVSDB to expose the device capability to higher management layers. The new table introduced in OVSDB is like below. <table name="hw_offload"> <p> Hardware switching configuration and capabilities. </p> <column name="name"> The name of hardware acceleration device. </column> <column name="dev_id" type='{"type": "integer", "minInteger": 0, "maxInteger": 7}'> The integer device id of hardware accelerated NIC. </column> <column name="pci_id" type='{"type": "string"}'> The PCI ID of the hardware acceleration device. The broker id/PF id. </column> <column name="features" key="n_vhost_ports" type='{"type": "integer"}'> The number of supported vhost ports in the hardware switch. </column> </table> The features column can be extended with more fields as necessary. IMO the proposed partial offload doesn't need populating the OVSDB, however its necessary to have some kind of feature discovery at init. 2) I feel its better to keep the hardware offload functionalities in netdev as much as possible similar to kernel implementation. I see changes in upcall and dpif. 3) The cost of flow install . PMDs are blocked when a hardware flow install is happening. Its an issue when there are lot of short lived flows are getting installed in the DP. One option to handle this would be move the flow install into revalidate. The advantage of this approach would be hardware offload would happen only when a flow is being used at least for sometime. Something like how revalidator thread handle the flow modify operation. 4) AFAIK, these hardware programmability for a NIC/not for a specific port. i.e the FDIR/RSS hash configuration are device specific. This will be an issue if a NIC shared between kernel and DPDK drivers? Regards _Sugesh > -----Original Message----- > From: [email protected] [mailto:ovs-dev- > [email protected]] On Behalf Of Yuanhan Liu > Sent: Tuesday, September 5, 2017 10:23 AM > To: [email protected] > Subject: [ovs-dev] [PATCH v2 0/8] OVS-DPDK flow offload with rte_flow > > Hi, > > Here is a joint work from Mellanox and Napatech, to enable the flow hw > offload with the DPDK generic flow interface (rte_flow). > > The basic idea is to associate the flow with a mark id (a unit32_t number). > Later, we then get the flow directly from the mark id, bypassing the heavy > emc processing, including miniflow_extract. > > The association is done with CMAP in patch 1. It also reuses the flow APIs > introduced while adding the tc offloads. The emc bypassing is done in patch > 2. The flow offload is done in patch 4, which mainly does two > things: > > - translate the ovs match to DPDK rte flow patterns > - bind those patterns with a MARK action. > > Afterwards, the NIC will set the mark id in every pkt's mbuf when it matches > the flow. That's basically how we could get the flow directly from the > received mbuf. > > While testing with PHY-PHY forwarding with one core and one queue, I got > about 54% performance boost. For PHY-vhost forwarding, I got about 41% > performance boost. The reason it's lower than v1 is I added the logic to get > the correct tcp_flags, which examines all packets recieved. > > The major issue mentioned in last version is also workarounded: the queue > index is never set to 0 blindly anymore, but set to the rxq that first > receives > the upcall pkt. > > Note that it's disabled by default, which can be enabled by: > > $ ovs-vsctl set Open_vSwitch . other_config:hw-offload=true > > > v2: - workaround the queue action issue > - fixed the tcp_flags being skipped issue, which also fixed the > build warnings > - fixed l2 patterns for Intel nic > - Converted some macros to functions > - did not hardcode the max number of flow/action > - rebased on top of the lastest code > > Thanks. > > --yliu > > > --- > Finn Christensen (3): > netdev-dpdk: implement flow put with rte flow > netdev-dpdk: retry with queue action > netdev-dpdk: set FDIR config > > Shachar Beiser (1): > dpif-netdev: record rx queue id for the upcall > > Yuanhan Liu (4): > dpif-netdev: associate flow with a mark id > dpif-netdev: retrieve flow directly from the flow mark > netdev-dpdk: convert ufid to dpdk flow > netdev-dpdk: remove offloaded flow on deletion > > lib/dp-packet.h | 14 ++ > lib/dpif-netdev.c | 132 +++++++++++-- > lib/flow.c | 78 ++++++++ > lib/flow.h | 1 + > lib/netdev-dpdk.c | 574 > +++++++++++++++++++++++++++++++++++++++++++++++++++++- > lib/netdev.c | 1 + > lib/netdev.h | 7 + > 7 files changed, 795 insertions(+), 12 deletions(-) > > -- > 2.7.4 > > _______________________________________________ > dev mailing list > [email protected] > https://mail.openvswitch.org/mailman/listinfo/ovs-dev _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
