We would like to introduce a HW offloading solution for scenario that one packet goes through DPDK OVS pipeline multiple times with recircle action. We call it merged-single-table HW offloading.
The standard use case is to support conntrack with HW offloading. Example, the packet matches flow 1 with action CT and recircle, then it matches flow 2 with action forward to certain port. Meaning the packet needs to go through multiple SW logic tables. When desiging such HW offloading, it would be straightforward to have 1to1 mapping for HW tables and SW logic tables. However there are some difficulty to do so: 1) Not all NIC support multiple HW tables. 2) When a lookup miss happens in non-1st table, it may be hard for CPU to continously handle this packet since the packet may already be changed in previous tables, like IP address change. 3) HW is hard to have Finite State Machine, eg to maintain conntrack state. 4) SW may introduce new actions that current HW can not support, like dp_hash. Our solution is to create only one single match/action flow for HW. The detail is 1) When packet goes through the SW logic tables (miniflow/SFC/Megaflows and so), it will record all mega flows it hits and some metadata during recircle. Some validation is needed to check the match/action for each flow. If validation failes, the record will be stopped. One special validation is that we only offload traffic with CT state EST. 2) When the packet hits one mega flow that has forward/drop action and no recircle action, all flows info and metadata will be sent to an offloading thread. 3) The offloading thread will merge the flows and actions into one single match/action flow, and offload this merged flow to HW. During the merging, the match may be expanded with extra items, like for CT case, we will expand 5 tuples into matchers. 4) Another aging thread is created to keep synchronization between HW and SW. It will periodically fetch stats from HW for offloaded flows, and contributes to megaflows' stats and Conntrack's stats. If some megaflow is dead, or the conntrack entry is aged out, the aging thread will also delete the merged rule from HW. Using this solution, we keep the HW simple (meaning less coupling/dependency on HW), and maintain the flexibility in SW. The solution is already massively deployed in our product environment, and works reliably. We would like to get some feedback before we submit all patches. Br, Zhike Wang JDCloud, Product Development, IaaS ------------------------------------------------------------------------------------------------ Mobile／+86 13466719566 E- mail／wangzh...@jd.com Address／5F Building A,North-Star Century Center,8 Beichen West Street,Chaoyang District Beijing Https://JDCloud.com ------------------------------------------------------------------------------------------------ _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
