On 2/27/24 07:51, Han Zhou wrote: > > > On Mon, Feb 26, 2024 at 9:44 AM Ilya Maximets <[email protected] > <mailto:[email protected]>> wrote: >> >> Currently, ovn-controller attempts to sync all the meters on each >> ofctrl_put() call. And the complexity of this logic is quadratic >> because for each desired meter we perform a full scan of all the >> rows in the Southbound Meter table in order to lookup a matching >> meter. This is very inefficient. In a setup with 25K meters this >> operation takes anywhere from 30 to 60 seconds to perform. All >> that time ovn-controller is blocked and doesn't process any updates. >> So, addition of new ports in such a setup becomes very slow. >> >> The meter lookup is performed by name and we have an index for it >> in the database schema. Might as well use it. >> >> Using the index for lookup reduces complexity to O(n * log n). >> And the time to process port addition on the same setup drops down >> to just 100 - 300 ms. >> >> We are still iterating over all the desired meters while they can >> probably be processed incrementally instead. But using an index >> is a simpler fix for now. >> >> Fixes: 885655e16e63 ("controller: reconfigure ovs meters for ovn meters") >> Fixes: 999e1adfb572 ("ovn: Support configuring meters through SB Meter >> table.") >> Reported-at: https://issues.redhat.com/browse/FDP-399 >> <https://issues.redhat.com/browse/FDP-399> >> Signed-off-by: Ilya Maximets <[email protected] <mailto:[email protected]>> >> --- >> >> This is a "performance bug", so the decision to backport this or not >> is on maintainers. But it is severe enough, IMO. >> > > Thanks Ilya. The fix looks good to me. And I think it is ok to backport, > since the change is simple enough. > > Acked-by: Han Zhou <[email protected] <mailto:[email protected]>> > > Just curious, how would the OVS perform with this large number of meters?
Hi. Thanks for review! I don't think 25K would cause any noticeable issues on OVS side, but I agree that if we scale further we may overload the classifier at some point. I don't have hard evidence to back this up, but based on my prior experience with flow explosion with negative matches, I think, we need about 200K rules in order for it to start becoming a problem. But also ACL rules are not normally conjunctive, i.e. should be easier for classifier to process, so we may probably handle more. Needs some testing. Best regards, Ilya Maximets. _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
