On Wed, Mar 30, 2016 at 10:27 AM, Chandran, Sugesh <sugesh.chand...@intel.com> wrote: >> -----Original Message----- >> From: Jesse Gross [mailto:je...@kernel.org] >> Sent: Wednesday, March 30, 2016 1:44 AM >> To: Chandran, Sugesh <sugesh.chand...@intel.com> >> Cc: dev@openvswitch.org >> Subject: Re: [ovs-dev] [RFC PATCH] tunneling: Improving vxlan performance >> using DPDK flow director feature. >> >> On Tue, Mar 29, 2016 at 12:43 AM, Chandran, Sugesh >> <sugesh.chand...@intel.com> wrote: >> >> -----Original Message----- >> >> From: Jesse Gross [mailto:je...@kernel.org] >> >> Sent: Friday, March 25, 2016 12:38 AM >> >> To: Chandran, Sugesh <sugesh.chand...@intel.com> >> >> Cc: dev@openvswitch.org >> >> Subject: Re: [ovs-dev] [RFC PATCH] tunneling: Improving vxlan >> >> performance using DPDK flow director feature. >> >> * Chaining together the multiple lookups used by tunnels on the >> >> assumption that the outer VXLAN source port distinguishes the inner >> >> flow. This would allow avoiding netdev_flow_key_equal_mf() a second >> >> time. This is definitely not legal because the VXLAN source port is >> >> only capturing a small subset of the total data that OVS is using. >> > >> > [Sugesh] From our analysis we found that optimizing one lookup give no >> > significant performance boost when compared with the overhead. This is >> > due to the fact that the second netdev_flow_key_equal_mf() still need >> > the tunnel information to match on a flow. We found in our tests that >> > most CPU cycles spends on extracting header fields from the packets than >> lookup. >> > >> > The proposal is to avoid the header field extraction by using an >> > additional unique software flow ID to match on. The two flows for >> > tunnel are marked with this ID when installing on the EMC. The >> > hardware report this ID along with hash(to mitigate the hash collision >> > in EMC) for every incoming packets that match on a hardware rule. This >> > used in EMC along with hash to find the flow. Currently OVS compares >> > hash +key(from header fields) to match a flow. The inner flow uses the >> > same unique ID and hardware flow flag to match on than the source port. >> We have modified the code little bit more, so that it saves the hardware id >> in >> the matching flow, for every emc_insert. >> >> I think that the performance improvements look cool but unfortunately, I >> just don't see how this can work. >> >> There really isn't a way to avoid extracting the header fields in software - >> I >> don't think that any NIC short of an NPU or other programmable hardware >> has the capability to match on all of the fields that OVS supports. >> Certainly, >> the UDP source port used by VXLAN and other tunnel protocols does not >> contain all of the information and, worse, it's controlled by a remote >> system. >> We can't trust the information contained in it without further verification >> because OVS flow rules are often used for security checks. I realize that in >> many cases this will appear to work because for a flow represented by a 5- >> tuple many of the other fields will be the same. However, we can't just make >> this assumption. > > [Sugesh] Totally agree with you. How about let the OVS program the NIC only > if all the > flow fields that can be supported in NIC. If the flow has more fields than > the NIC can support, > the rules will not get programmed on the hardware. > I feel, If we make sure the rules in NIC can validate all the fields of that > corresponding software flow, > Its possible to avoid software header extraction on those packets. Any > comments??
I think that would work although I would definitely start with a pure software version that extracts only the relevant header fields first. Once that is done, it would be good to compare that with a NIC-based version and weigh it against the complexity of the implementation and whether there are common use cases than can take advantage of a more restricted set of flows that could be offloaded. However, I really think that the wildcarded lookups for EMC misses are the most interesting case for offloading to the NIC and that has the advantage of not requiring all fields to be supported by the NIC. _______________________________________________ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
