Sometimes network components are compute node-specific. Sometimes such
components are replicated, almost identically, for multiple nodes
in the cluster.
One such example is the case of Kubernetes NodePort services which
translate (in the ovn-kubernetes case) to Load_Balancer
objects being applied to each and every node's logical gateway router.
These load balancers are almost identical, the main difference being
the fact that they use different VIPs (the node's IP).
With the current OVN load balancer design, this becomes a problem at
scale because the number of load balancers that must be configured is
N x M (N nodes times M services).
This series proposes a new concept in OVN: virtual network component
templates. The goal of the templates is to help reduce resource
consumption in the OVN central components in specific cases like the one
described above.
To achieve that, the CMS will instead configure a "templated" load
balancer for every service and apply that single template record to
the cluster-wide load balancer group. This template is then
instantiated differently on different compute nodes. This translation
is controlled through per-chassis "template variables" configured by
the CMS in the new NB.Template_Var table.
Patch 5/5 introduces a synthetic benchmark simulating what an OpenShift
router (using NodePort services) scale test would do. The benchmark
allows us to compare the "standard" (no-template) configuration
against the configuration that uses component templates:
+----------+-------+-------------------+--------------------+----------+---------+-----------+---------------------------+
| | | | NB | SB |
| northd | ovn-controller |
+----------+-------+-------------------+--------------------+----------+---------+-----------+---------------------------+
| Template | Nodes | NodePort Services | Size(MB) | RSS(MB) | Size(MB) |
RSS(MB) | loop(sec) | RSS (MB) | recompute(sec) |
| | | (5 backends) | | | |
| | | |
+----------+-------+-------------------+----------+---------+----------+---------+-----------+----------+----------------+
| No | 60 | 1000 | 25 | 116 | 118 | 589
| 2.70 | 463 | 0.20 |
| Yes | 60 | 1000 | 6 | 25 | 8 | 46
| 0.07 | 44 | 0.20 |
+----------+-------+-------------------+----------+---------+----------+---------+-----------+----------+----------------+
| No | 120 | 2000 | 67 | 865 | 471 | 9000
| 15.60 | 1016 | 0.40 |
| Yes | 120 | 2000 | 23 | 96 | 28 | 225
| 0.22 | 83 | 0.40 |
+----------+-------+-------------------+----------+---------+----------+---------+-----------+----------+----------------+
| Yes | 120 | 10000 | 118 | 440 | 136 | 668
| 0.72 | 311 | 1.77 |
+----------+-------+-------------------+----------+---------+----------+---------+-----------+----------+-----------------
| Yes | 250 | 10000 | 244 | 870 | 263 | 1502
| 1.26 | 318 | 1.87 |
+-------------------------------------------------------------------------------------------------------------------------
For the N=120 S=2000 case:
- NB size reduced by ~65%
- NB RSS reduced by ~90%
- SB size reduced by ~95%
- SB RSS reduced by ~98%
- ovn-northd loop time reduced by ~98%
- ovn-controller RSS reduced by ~92%
Dumitru Ceara (5):
lflow: Factor out the lflow reference handling code into a new module.
Add NB and SB Template_Var tables.
controller: Add support for templated actions and matches.
lb: Support using templates.
tutorial: Add scripts to simulate node-port ovn-k8s services.
controller/lflow.c | 244 ++++++++++----
controller/lflow.h | 8 +-
controller/ofctrl.c | 11 +-
controller/ofctrl.h | 3 +-
controller/ovn-controller.c | 350 +++++++++++++++++++-
include/ovn/expr.h | 4 +-
include/ovn/lex.h | 15 +-
lib/actions.c | 9 +-
lib/expr.c | 18 +-
lib/lb.c | 457 +++++++++++++++++++++++----
lib/lb.h | 40 ++-
lib/lex.c | 57 ++++
lib/objdep.c | 1 +
lib/objdep.h | 1 +
lib/ovn-util.c | 3 -
northd/automake.mk | 4 +-
northd/en-northd.c | 4 +
northd/inc-proc-northd.c | 8 +-
northd/northd.c | 130 +++++---
northd/northd.h | 4 +
northd/template-var.c | 74 +++++
northd/template-var.h | 58 ++++
ovn-nb.ovsschema | 17 +-
ovn-nb.xml | 82 ++++-
ovn-sb.ovsschema | 12 +-
ovn-sb.xml | 15 +
tests/ovn-controller.at | 50 +++
tests/ovn-nbctl.at | 23 +-
tests/ovn-northd.at | 42 +++
tests/ovn.at | 211 ++++++++++++-
tests/system-ovn.at | 183 +++++++++++
tests/test-ovn.c | 17 +-
tutorial/automake.mk | 4 +-
tutorial/ovn-gen-lb-template-vars.py | 116 +++++++
tutorial/ovn-lb-benchmark.sh | 110 +++++++
utilities/ovn-nbctl.c | 125 ++++----
utilities/ovn-sbctl.c | 3 +
utilities/ovn-trace.c | 33 +-
38 files changed, 2249 insertions(+), 297 deletions(-)
create mode 100644 northd/template-var.c
create mode 100644 northd/template-var.h
create mode 100755 tutorial/ovn-gen-lb-template-vars.py
create mode 100755 tutorial/ovn-lb-benchmark.sh
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
