It seems that this patch implements a partial hw-offload using DPDK RTE FLOW
api.
Using a flow_tag to circumvent the EMC, and get the flow from NIC-delivered
flow_tag on input.
In general I think that RTE FLOW is capable of doing more, like full flow
offload, and therefore we should aim at a more full implementation of RTE FLOW
usage in OVS. We may not always be in the output datapath in OVS, but OVS
should still be fully updated and in control.
At least we will need that.
The RTE FLOW feature: rte_flow_query() can be used to retrieve NIC flow
statistics. And that is what we need here.
Furthermore, we would also like to have the possibility to offload virtual
ports (at least at some point later), so that the in_port may not be the native
pmd port_id. This feature is also available in RTE FLOW - item type:
RTE_FLOW_ITEM_TYPE_PORT
And again, this RTE FLOW feature is more or less, what is needed for this.
Besides that, I have some questions about this implementation. It is probably
me that have not read the code carefully enough, but maybe you could clarify
them for me:
You build an item array out of the flow wildcard mask:
...
if (memcmp(&mask->dl_dst,ð_mac,sizeof(struct eth_addr))!=0
|| memcmp(&mask->dl_src,ð_mac,sizeof(struct eth_addr))!=0) {
VLOG_INFO("rte_item_eth\n");
item_any_flow[ii++].set = rte_item_set_eth;
*buf_size += sizeof(struct rte_flow_item_eth);
*buf_size += sizeof(struct rte_flow_item_eth);
if (mask->nw_src!=0 || mask->nw_dst!=0) {
VLOG_INFO("rte_item_ip\n");
item_any_flow[ii++].set = rte_item_set_ip;
*buf_size += sizeof(struct rte_flow_item_ipv4);
*buf_size += sizeof(struct rte_flow_item_ipv4);
if (mask->tp_dst!=0 || mask->tp_src!=0) {
item_any_flow[ii++].set = rte_item_set_udp;
*buf_size += sizeof(struct rte_flow_item_udp);
*buf_size += sizeof(struct rte_flow_item_udp);
}
}
}
Taken from the function hw_pipeline_item_array_build().
I would like to know how you make sure that all important bits in the flow/mask
does match the flow that you are about to program into the NIC?
Theoretically, would it be possible to program an UDP RTE FLOW filter in NIC by
using a TCP flow?
It seems as if the flows are never programmed into NIC:
From hw_pipeline_thread():
...
while (1) {
// listen to read_socket :
// call the rte_flow_create ( flow , wildcard mask)
ret = hw_pipeline_msg_queue_dequeue(msgq,&ptr_rule);
if (ret != 0) {
continue;
}
if (ptr_rule.mode == HW_PIPELINE_REMOVE_RULE) {
ret =hw_pipeline_remove_flow(dp,&ptr_rule.data.rm_flow);
if (OVS_UNLIKELY(ret)) {
VLOG_ERR(" hw_pipeline_remove_flow failed to remove flow \n");
}
}
ptr_rule.mode = HW_PIPELINE_NO_RULE;
}
Shouldn't hw_pipeline_insert_flow() function be called?
Finn Christensen
-----Original Message-----
From: [email protected] [mailto:[email protected]]
On Behalf Of Shachar Beiser
Sent: 6. juli 2017 06:33
To: [email protected]
Subject: Re: [ovs-dev] [RFC] [PATCH 00/11] Data Path Classifier Offloading
Hi ,
I would like to clarify since I did not add the label [RFC] to the subject
of my patches , but only to the cover letter that All the patches I have sent
were sent as a RFC:
[RFC] [PATCH 00/11] Data Path Classifier Offloading [RFC][PATCH 1/11]
ovs/dp-cls ....
[RFC][PATCH 2/11] ovs/dp-cls ....
[RFC] [PATCH 3/11] ovs/dp-cls ....
[RFC] [PATCH 4/11] ovs/dp-cls ....
[RFC] [PATCH 5/11] ovs/dp-cls ....
[RFC][PATCH 6/11] ovs/dp-cls ....
[RFC] [PATCH 7/11] ovs/dp-cls ....
[RFC] [PATCH 8/11] ovs/dp-cls ....
[RFC] [PATCH 9/11] ovs/dp-cls ....
[RFC] [PATCH 10/11] ovs/dp-cls ....
[RFC] [PATCH 11/11] ovs/dp-cls ....
-Shachar Beiser
-----Original Message-----
From: Shachar Beiser [mailto:[email protected]]
Sent: Wednesday, July 5, 2017 3:27 PM
To: [email protected]
Cc: Shachar Beiser <[email protected]>; Mark Bloch <[email protected]>;
Olga Shern <[email protected]>
Subject: [PATCH 00/11] Data Path Classifier Offloading
Request for comments Subject: OVS data-path classifier offload
Versions:
OVS master.
DPDK 17.05.1.
Purpose & Scope
This RFC describes the flows’ hardware offloading over DPDK .
The motivation of hardware flows’ offloading is accelerating the OVS DPDK data
plane.The classification is done by the hardware.
A flow tag that represents the matched rule in the hardware, received by the
OVS and saves the lookup time.
OVS data-path classifier has to support additional functionality.
If the hardware supports flows table offloading and the user activates the
feature,the classifier rules are offloaded to the hardware classifier in
addition to the data-path classifier.
When flows are removed from the classifier the flows have to be removed from
the hardware flows table.
In the OVS classification data-path there are 4 stages:
Read packets from DPDK-PMD by OVS.
Find Matching flows.
Group packets by flows.
Execute action.
The suggested design intervene in the first two stages the 3rd and 4th stages
left untouched.
OVS additional code new file lib/hw-pipeline.c new file lib/hw-pipeline.h
The hw-pipeline.c implements three significant objects.
An offloading thread, a pipe and flow tag pool.
Inserting and removing classifiers rules to/from the hardware requires time
which is translated to latency. A new OVS thread takes care of the
insertion/deletion of rules and prevents blocking the PMD thread context.
A pipe ( mkpipe() ) transfers the classifier rules to the new thread context.
For each classifier rule, it is required to add a unique flow tag.
This same tag is attached later to the packet metadata by DPDK PMD if there is
a match in the hardware. OVS uses this tag as an index to find the relevant
software flow.
The lookup processing becomes very efficient .In order to generate a unique
tag, the design is introducing a new flow tag pool.
A flow tag pool is a list implemented in an array.
The flow tag is generated and returned efficiently.
OVS changes
lib/dpif-netdev.c changes:
1) ceate_dp_netdev() initializes the new flow tag pool data base, the
classifier offload thread and pipe.
2) dp_netdev_free() frees the new flow tag pool data base, the classifier
offload thread and pipe.
3) dp_netdev_pmd_remove_flow() if the feature exists and active, the function
sends the classifier rule through the mkpipe to new thread.
The new thread removes the rule from the hardware.
4) dp_netdev_flow_add() if the feature exists and active, the function sends
the classifier rule through the mkpipe to new thread.
The new thread inserts the rule to the hardware.
5) emc_processing() if the feature exists and active & OVS received a valid tag
from the hardware: OVS skips emc processing.
6) fast_path_processing() if the feature exists and active and the function
received a valid tag from the hardware:
a.OVS looks for the tag that is attached to the packet metadata and then OVS
finds the flow according to the tag.
b.The functions:
dp_netdev_pmd_lookup_flow(), dpcls_lookup() are not called by OVS.
c.Group packets by flow and executing the action are done
the same way as before.
In the file lib/netdev-dpdk.c:
struct netdev_class is enhanced .
The structure has additional pointer to a function called get_pipeline().
A new function is introduced called netdev_dpdk_get_pipeline.
The function netdev_dpdk_get_pipeline reads the received tag from the hardware
if there is a match between the packet and the inserted rules.
if (mbuf->ol_flags & PKT_RX_FDIR_ID) {
ppl_md->id = HW_OFFLOAD_PIPELINE;
ppl_md->flow_tag = mbuf->hash.fdir.hi;
}
The source code:
https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fopenvswitch%2Fovs%2Fcompare%2Fmaster...shacharbe%3Adp-cls-offload-no-tunnel-rfc%3Fexpand%3D1&data=02%7C01%7Cshacharbe%40mellanox.com%7C3c9dc6562d8c4ad8322708d4c3a1560a%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636348545082600173&sdata=UBVDp8NWPNwnUaZfm0SvyTutTmaR49QJy1lbqITA6q0%3D&reserved=0
References Intel work:
https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatchwork.ozlabs.org%2Fpatch%2F701623%2F&data=02%7C01%7Cshacharbe%40mellanox.com%7C3c9dc6562d8c4ad8322708d4c3a1560a%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636348545082600173&sdata=FzNGca72ySTJRcy1Dv5EHp9wbpRyyHslBWcBa0onuI4%3D&reserved=0
OVS presentation: fall 2016 summit:
https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fopenvswitch.org%2Fsupport%2Fovscon2016%2F7%2F1450-stringer.pdf&data=02%7C01%7Cshacharbe%40mellanox.com%7C3c9dc6562d8c4ad8322708d4c3a1560a%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C1%7C636348545082610175&sdata=wal%2Fhe3jwegxQ8Y2SFsiNay%2BfocTl8lq%2FcZ8YXWK72I%3D&reserved=0
Shachar Beiser (11):
ovs/dp-cls: fetching the mark id from hw
ovs/dp-cls: moving structures to dpif-netdev.h
ovs/dp-cls: saving rx queue identifier
ovs/dp-cls: initializing HW pipeline
ovs/dp-cls: free HW pipeline
ovs/dp-cls: remove data-path classifier rule
ovs/dp-cls: inserting data-path classifier rule
ovs/dp-cls: tag lookup and processing
ovs/dp-cls: flow tag read
ovs/dp-cls: removing flow from HW in the dp-cls offload thread
ovs/dp-cls: inserting rule to HW from offloading thread context
lib/automake.mk | 4 +-
lib/dp-packet.h | 1 +
lib/dpif-netdev.c | 300 +++++--------
lib/dpif-netdev.h | 298 ++++++++++++-
lib/hw-pipeline.c | 1146 +++++++++++++++++++++++++++++++++++++++++++++++++
lib/hw-pipeline.h | 48 +++
lib/netdev-bsd.c | 1 +
lib/netdev-dpdk.c | 49 +++
lib/netdev-dpdk.h | 22 +-
lib/netdev-dummy.c | 1 +
lib/netdev-linux.c | 1 +
lib/netdev-provider.h | 7 +-
lib/netdev-vport.c | 1 +
lib/netdev.c | 2 +
14 files changed, 1684 insertions(+), 197 deletions(-) create mode 100644
lib/hw-pipeline.c create mode 100644 lib/hw-pipeline.h
--
1.8.3.1
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
