The cost of reprocessing a lflow referencing a big address set can be very
high. Today a single address change in an address set would cause the
related logical flows being reprocessed. When the change rate of an
address set is high, ovn-controller would be busy reprocessing lflows.
This patch series optimizes this typical scenario for large scale environment
by incrementally processing the updates.
For address deletions, it deletes the related flows for the deleted addresses
only, without deleting and recreating unrelated flows unnecessarily.
For address additional, it generates a fake address set that only
contains the added addresses for flow generation, and then eliminates
the flows that are not related to the newly added addresses.
There are limitations that these approaches can't apply. For example, when an
ACL is in the form:
... ip4.src == {$as1, $as2}, drop
In this case during lflow parsing the expressions are combined to a single OR,
which loses the tracking information. There are other cases that can't be
handled are added in test cases. In these situation it just fall back to the
old approach that reprocesses the lflow.
Scale test shows obvious performance gains because the time complexity
changed from O(n) to O(1). The bigger the size of address set, the more
CPU savings. With the AS size of 10k, the test shows ~40x speed up.
Test setup:
CPU: Intel(R) Core(TM) i9-7920X CPU @ 2.90GHz.
5 ACL all referencing an address set of 10,000 IPs.
Measure the time spent by ovn-controller for handling one IP deletion
from the address set.
Before: ~400ms
After: 11-12ms
There is memory cost increase, due to the index built to track each
individual IP. The total memory cost for the OF flows in ovn-controller
increased ~20% in the 10k AS size test.
Before:
ofctrl_desired_flow_usage-KB:22248
ofctrl_installed_flow_usage-KB:14850
ofctrl_sb_flow_ref_usage-KB:7208
After:
ofctrl_desired_flow_usage-KB:22248
ofctrl_installed_flow_usage-KB:14850
ofctrl_sb_flow_ref_usage-KB:15551
Han Zhou (10):
expr.c: Use expr_destroy and expr_clone instead of free and xmemdup.
ofctrl.c: Combine remove_flows_from_sb_to_flow and
ofctrl_flood_remove_flows.
ovn-controller: Track individual IP information of address set during
lflow parsing.
ovn-controller.c: Remove unnecessary asserts and useless variables.
ovn-controller.c: Refactor init_lflow_ctx.
ovn-controller: Tracking SB address set updates.
lflow.c: Set "changed" properly in lflow_handle_changed_ref().
ovn-controller: Add tests for different ACL address set usage
patterns.
ovn-controller: Handle addresses deletion in address set
incrementally.
ovn-controller: Handle addresses addition in address set
incrementally.
controller/lflow-conj-ids.c | 20 +
controller/lflow-conj-ids.h | 1 +
controller/lflow.c | 408 ++++++++++++++-
controller/lflow.h | 10 +
controller/ofctrl.c | 321 +++++++++---
controller/ofctrl.h | 23 +-
controller/ovn-controller.c | 278 +++++------
controller/physical.c | 2 +-
include/ovn/expr.h | 20 +-
lib/expr.c | 258 ++++++++--
tests/ovn-controller.at | 965 ++++++++++++++++++++++++++++++++++++
11 files changed, 2021 insertions(+), 285 deletions(-)
--
2.30.2
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
