On Sat, Feb 5, 2022 at 12:47 AM Han Zhou <[email protected]> wrote: > > The cost of reprocessing a lflow referencing a big address set can be very > high. Today a single address change in an address set would cause the > related logical flows being reprocessed. When the change rate of an > address set is high, ovn-controller would be busy reprocessing lflows. > > This patch series optimizes this typical scenario for large scale environment > by incrementally processing the updates. > > For address deletions, it deletes the related flows for the deleted addresses > only, without deleting and recreating unrelated flows unnecessarily. > > For address additional, it generates a fake address set that only > contains the added addresses for flow generation, and then eliminates > the flows that are not related to the newly added addresses. > > There are limitations that these approaches can't apply. For example, when an > ACL is in the form: > > ... ip4.src == {$as1, $as2}, drop > > In this case during lflow parsing the expressions are combined to a single OR, > which loses the tracking information. There are other cases that can't be > handled are added in test cases. In these situation it just fall back to the > old approach that reprocesses the lflow. > > Scale test shows obvious performance gains because the time complexity > changed from O(n) to O(1). The bigger the size of address set, the more > CPU savings. With the AS size of 10k, the test shows ~40x speed up. > > Test setup: > CPU: Intel(R) Core(TM) i9-7920X CPU @ 2.90GHz. > 5 ACL all referencing an address set of 10,000 IPs. > > Measure the time spent by ovn-controller for handling one IP deletion > from the address set. > > Before: ~400ms > After: 11-12ms > > There is memory cost increase, due to the index built to track each > individual IP. The total memory cost for the OF flows in ovn-controller > increased ~20% in the 10k AS size test. > > Before: > ofctrl_desired_flow_usage-KB:22248 > ofctrl_installed_flow_usage-KB:14850 > ofctrl_sb_flow_ref_usage-KB:7208 > > After: > ofctrl_desired_flow_usage-KB:22248 > ofctrl_installed_flow_usage-KB:14850 > ofctrl_sb_flow_ref_usage-KB:15551 > > Han Zhou (10): > expr.c: Use expr_destroy and expr_clone instead of free and xmemdup. > ofctrl.c: Combine remove_flows_from_sb_to_flow and > ofctrl_flood_remove_flows. > ovn-controller: Track individual IP information of address set during > lflow parsing. > ovn-controller.c: Remove unnecessary asserts and useless variables. > ovn-controller.c: Refactor init_lflow_ctx. > ovn-controller: Tracking SB address set updates. > lflow.c: Set "changed" properly in lflow_handle_changed_ref(). > ovn-controller: Add tests for different ACL address set usage > patterns. > ovn-controller: Handle addresses deletion in address set > incrementally. > ovn-controller: Handle addresses addition in address set > incrementally. > > controller/lflow-conj-ids.c | 20 + > controller/lflow-conj-ids.h | 1 + > controller/lflow.c | 408 ++++++++++++++- > controller/lflow.h | 10 + > controller/ofctrl.c | 321 +++++++++--- > controller/ofctrl.h | 23 +- > controller/ovn-controller.c | 278 +++++------ > controller/physical.c | 2 +- > include/ovn/expr.h | 20 +- > lib/expr.c | 258 ++++++++-- > tests/ovn-controller.at | 965 ++++++++++++++++++++++++++++++++++++ > 11 files changed, 2021 insertions(+), 285 deletions(-) > > -- > 2.30.2 >
This RFC series is superseded by the formal patches: https://mail.openvswitch.org/pipermail/ovs-dev/2022-February/391395.html Please review that one instead. Thanks, Han _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
