On Fri, Sep 30, 2022 at 7:01 AM Dumitru Ceara <[email protected]> wrote: > > Sometimes network components are compute node-specific. Sometimes such > components are replicated, almost identically, for multiple nodes > in the cluster. > > One such example is the case of Kubernetes NodePort services which > translate (in the ovn-kubernetes case) to Load_Balancer > objects being applied to each and every node's logical gateway router. > These load balancers are almost identical, the main difference being > the fact that they use different VIPs (the node's IP). > > With the current OVN load balancer design, this becomes a problem at > scale because the number of load balancers that must be configured is > N x M (N nodes times M services). > > This series proposes a new concept in OVN: virtual network component > templates. The goal of the templates is to help reduce resource > consumption in the OVN central components in specific cases like the one > described above. > > To achieve that, the CMS will instead configure a "templated" load > balancer for every service and apply that single template record to > the cluster-wide load balancer group. This template is then > instantiated differently on different compute nodes. This translation > is controlled through per-chassis "template variables" configured by > the CMS in the new NB.Template_Var table. > Thanks Dumitru for the great improvement!
> A syntetic benchmark simulating what an OpenShift router (using Node > Port services) scale test would do shows the following preliminary > results: > A. 120 node, 2K NodePort services: > - before: > - Southbound DB size on disk (compacted): ~385MB > - Southbound DB memory usage (RSS): ~3GB > - Southbound DB logical flows: 720K > > - after: > - Southbound DB size on disk (compacted): ~100MB > - Southbound DB memory usage (RSS): ~250MB > - Southbound DB logical flows: 6K > > B. 250 node, 2K NodePort services: > - after (didn't run the "before" test as it was taking way too long): > - Southbound DB size on disk (compacted): ~155MB > - Southbound DB memory usage (RSS): ~760MB > - Southbound DB logical flows: 6K A quick question to the test. How many LSPs per node? I am just wondering, how could the number of lflows be the same (6k) when number of nodes increased from 120 to 250? For some of my scale tests, the number of lflows are far more than this even if I don't create any LBs. (also consider that ovn-k8s deployment has at least an ext-LS and a GR per node) I have no doubt of the effectiveness of this improvement, but just need to understand the numbers better since I am also doing scale tests and measurements on top of this patch series. Thanks, Han > > A basic example of how to configure a templated load balancer follows: > $ ovn-nbctl create load_balancer name=lb-test \ > protocol=tcp options:template=true \ > vips:\"^vip:4200\"="^backends" > > $ ovn-nbctl ls-add ls > $ ovn-nbctl ls-lb-add ls lb-test > > # Instantiate the load balancer on chassis-1 > $ ovn-nbctl create template_var name=vip value=80.80.80.1 chassis_name=chassis-1 > $ ovn-nbctl create template_var name=backends value='"42.42.42.1:1000"' chassis_name=chassis-1 > > # Instantiate the load balancer on chassis-2 > $ ovn-nbctl create template_var name=vip value=80.80.80.2 chassis_name=chassis-2 > $ ovn-nbctl create template_var name=backends value='"42.42.42.2:1000"' chassis_name=chassis-2 > > Dumitru Ceara (8): > ofctrl: Use uuidset instead of ofctrl_flood_remove_node maps. > lflow: Use uuidset instead of lflow_processed_node maps. > lflow: Factor out the lflow reference handling code into a new module. > Add NB and SB Template_Var tables. > controller: Add support for templated actions and matches. > lb: Support using templates. > controller: Add Template_Var <- LB references. > system-ovn.at: Add system tests for template load balancers. > > > controller/lflow.c | 558 ++++++++++++++---------------------- > controller/lflow.h | 81 ++---- > controller/lport.c | 3 +- > controller/ofctrl.c | 9 +- > controller/ofctrl.h | 3 +- > controller/ovn-controller.c | 422 +++++++++++++++++++++++++-- > include/ovn/expr.h | 4 +- > include/ovn/lex.h | 14 +- > lib/actions.c | 9 +- > lib/automake.mk | 2 + > lib/expr.c | 18 +- > lib/lb.c | 464 +++++++++++++++++++++++++----- > lib/lb.h | 40 ++- > lib/lex.c | 55 ++++ > lib/objdep.c | 262 +++++++++++++++++ > lib/objdep.h | 122 ++++++++ > lib/ovn-util.c | 10 +- > lib/ovn-util.h | 3 + > northd/automake.mk | 4 +- > northd/en-northd.c | 4 + > northd/inc-proc-northd.c | 8 +- > northd/northd.c | 126 +++++--- > northd/northd.h | 2 + > northd/template-var.c | 77 +++++ > northd/template-var.h | 59 ++++ > ovn-nb.ovsschema | 16 +- > ovn-nb.xml | 46 +++ > ovn-sb.ovsschema | 11 +- > ovn-sb.xml | 20 ++ > tests/ovn-nbctl.at | 23 +- > tests/ovn-northd.at | 39 +++ > tests/ovn.at | 2 +- > tests/system-ovn.at | 195 +++++++++++++ > tests/test-ovn.c | 16 +- > utilities/ovn-nbctl.c | 122 ++++---- > utilities/ovn-trace.c | 26 +- > 36 files changed, 2217 insertions(+), 658 deletions(-) > create mode 100644 lib/objdep.c > create mode 100644 lib/objdep.h > create mode 100644 northd/template-var.c > create mode 100644 northd/template-var.h > _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
