On Tue, Mar 29, 2016 at 12:43 AM, Chandran, Sugesh <sugesh.chand...@intel.com> wrote: >> -----Original Message----- >> From: Jesse Gross [mailto:je...@kernel.org] >> Sent: Friday, March 25, 2016 12:38 AM >> To: Chandran, Sugesh <sugesh.chand...@intel.com> >> Cc: dev@openvswitch.org >> Subject: Re: [ovs-dev] [RFC PATCH] tunneling: Improving vxlan performance >> using DPDK flow director feature. >> * Chaining together the multiple lookups used by tunnels on the assumption >> that the outer VXLAN source port distinguishes the inner flow. This would >> allow avoiding netdev_flow_key_equal_mf() a second time. This is definitely >> not legal because the VXLAN source port is only capturing a small subset of >> the total data that OVS is using. > > [Sugesh] From our analysis we found that optimizing one lookup give no > significant performance boost when compared with the overhead. This is due > to the > fact that the second netdev_flow_key_equal_mf() still need the tunnel > information > to match on a flow. We found in our tests that most CPU cycles spends on > extracting > header fields from the packets than lookup. > > The proposal is to avoid the header field extraction by using an additional > unique software > flow ID to match on. The two flows for tunnel are marked with this ID when > installing on the > EMC. The hardware report this ID along with hash(to mitigate the hash > collision in EMC) > for every incoming packets that match on a hardware rule. This used in EMC > along with hash to find the flow. Currently OVS compares hash +key(from > header fields) > to match a flow. The inner flow uses the same unique ID and hardware flow > flag to match > on than the source port. We have modified the code little bit more, so that > it saves the hardware > id in the matching flow, for every emc_insert.
I think that the performance improvements look cool but unfortunately, I just don't see how this can work. There really isn't a way to avoid extracting the header fields in software - I don't think that any NIC short of an NPU or other programmable hardware has the capability to match on all of the fields that OVS supports. Certainly, the UDP source port used by VXLAN and other tunnel protocols does not contain all of the information and, worse, it's controlled by a remote system. We can't trust the information contained in it without further verification because OVS flow rules are often used for security checks. I realize that in many cases this will appear to work because for a flow represented by a 5-tuple many of the other fields will be the same. However, we can't just make this assumption. One possible exception to this rule is if we did an analysis on the flows that are actually being used by OVS and only tried to extract those fields. This is a pure software optimization that might have similar effects to what you are observing here. This most likely makes the most sense in the context of a BPF based datapath where the flow extractor can be dynamically generated and compiled. >> I'm not sure that I really see any advantage in using a Flow Director perfect >> filter to return a software defined hash value compared to just using the RSS >> hash directly as we are doing today. I think the main case where it would be >> useful is if hardware wildcarding was used to skip the EMC altogether and its >> size constraints. If that was done then I think that this would no longer be >> specialized to VXLAN at all. > [Sugesh] This may give performance improvement when we have > large set of rules that overflows EMC. But for a typical use case where > 80-90% rules hits EMC > doesn’t get any performance benefit out of it. Please correct me if I am > wrong here. > The intention here is to optimize the tunneling performance in all the use > cases. To be honest, I think that last 10-20% may be more interesting. Up to this point in time, the DPDK implementation in OVS has placed a lot of emphasis on PPS throughput with a relatively small number of streams. However, while this looks great on benchmarks, it doesn't necessarily match real world use cases. Even worse, it tends to fall apart at the worst possible times - like a DoS attack. If the NIC were able to effectively enlarge the EMC to handle these cases then I think that would be a huge boost to the usability of OVS on DPDK. _______________________________________________ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
