Currently ovn-ic recomputes its entire transit-switch/gateway/route
state on every change to the IC databases (or the local NB/SB). On
large interconnected deployments this full recompute dominates the
ovn-ic runtime and adds significant latency between a configuration
change in one AZ and its propagation to the others.
This series introduces incremental processing (I+P) to ovn-ic, mirroring
the engine-node model already used by ovn-northd and ovn-controller. The
full recompute is split into per-subsystem engine nodes, and each node
gets an incremental change handler so that a change to a single object no
longer forces a global recompute.
Approach
--------
- The monolithic recompute is first refactored into per-subsystem
*_run functions and then into independent engine nodes (az, dp_enum,
gateway, ts, tr, port_binding, route, service_mon).
- Each node gains an incremental handler that reacts only to the
tracked changes on its inputs, falling back to recompute when a
change cannot be handled incrementally.
Testing
-------
- make check / make check-ovn-ic all pass.
- Scale test as above:
3: perf-ovn-ic.at:233 ovn-ic basic scale test -- 200 VPCs x 3 AZs x 4 TSs:
incremental vs recompute -- parallelization=yes
4: perf-ovn-ic.at:239 ovn-ic basic scale test -- 400 VPCs x 3 AZs x 4 TSs:
incremental vs recompute -- parallelization=yes
Results
-------
3: ovn-ic basic scale test -- 200 VPCs x 3 AZs x 4 TSs: incremental vs
recompute -- parallelization=yes
Results for 'Incremental: +1 VPC at 200-VPC scale (node handler times)'
---
en_ts incremental Maximum (msec): 2
en_ts incremental Average (msec): 0.999983
en_tr incremental Maximum (msec): 0
en_tr incremental Average (msec): 0
en_route incremental Maximum (msec): 2
en_route incremental Average (msec): 1.190430
en_port_binding incremental Maximum (msec): 0
en_port_binding incremental Average (msec): 0
en_tunnel_key incremental Maximum (msec): 0
en_tunnel_key incremental Average (msec): 0
3: ovn-ic basic scale test -- 200 VPCs x 3 AZs x 4 TSs: incremental vs
recompute -- parallelization=yes
Results for 'Full recompute at 201-VPC scale (node times)'
---
en_ts recompute Maximum (msec): 2
en_ts recompute Average (msec): 2.000000
en_tr recompute Maximum (msec): 0
en_tr recompute Average (msec): 0.000000
en_route recompute Maximum (msec): 55
en_route recompute Average (msec): 55.000000
en_port_binding recompute Maximum (msec): 27
en_port_binding recompute Average (msec): 27.000000
en_tunnel_key recompute Maximum (msec): 1
en_tunnel_key recompute Average (msec): 1.000000
4: ovn-ic basic scale test -- 400 VPCs x 3 AZs x 4 TSs: incremental vs
recompute -- parallelization=yes
Results for 'Incremental: +1 VPC at 400-VPC scale (node handler times)'
---
en_ts incremental Maximum (msec): 3
en_ts incremental Average (msec): 1.233665
en_tr incremental Maximum (msec): 0
en_tr incremental Average (msec): 0
en_route incremental Maximum (msec): 5
en_route incremental Average (msec): 2.799805
en_port_binding incremental Maximum (msec): 0
en_port_binding incremental Average (msec): 0
en_tunnel_key incremental Maximum (msec): 1
en_tunnel_key incremental Average (msec): 0
4: ovn-ic basic scale test -- 400 VPCs x 3 AZs x 4 TSs: incremental vs
recompute -- parallelization=yes
Results for 'Full recompute at 401-VPC scale (node times)'
---
en_ts recompute Maximum (msec): 4
en_ts recompute Average (msec): 4.000000
en_tr recompute Maximum (msec): 0
en_tr recompute Average (msec): 0.000000
en_route recompute Maximum (msec): 120
en_route recompute Average (msec): 120.000000
en_port_binding recompute Maximum (msec): 58
en_port_binding recompute Average (msec): 58.000000
en_tunnel_key recompute Maximum (msec): 2
en_tunnel_key recompute Average (msec): 2.000000
Paulo Guilherme Silva (12):
northd: Read ic-vxlan_mode as a boolean value, not key presence.
ic: Prepare *_run functions for engine nodes.
ic: Split full recompute into per-subsystem engine nodes.
ic: Add incremental handler for the en_az node.
ic: Add incremental handler for the en_dp_enum node.
ic: Add incremental handler for the en_gateway node.
ic: Add incremental handler for the en_ts node.
ic: Add incremental handler for the en_tr node.
ic: Add incremental handler for the en_port_binding node.
ic: Add incremental handler for the en_route node.
tests: Add ovn-ic incremental-processing basic scale test.
ic: Add incremental handler for the en_service_mon node.
Paulo Guilherme Silva (12):
northd: Read ic-vxlan_mode as a boolean value, not key presence.
ic: Prepare *_run functions for engine nodes.
ic: Split full recompute into per-subsystem engine nodes.
ic: Add incremental handler for the en_az node.
ic: Add the en_tunnel_key node.
ic: Add incremental handler for the en_gateway node.
ic: Add incremental handler for the en_ts node.
ic: Add incremental handler for the en_tr node.
ic: Add incremental handler for the en_port_binding node.
ic: Add incremental handler for the en_route node.
tests: Add ovn-ic incremental-processing basic scale test.
ic: Add incremental handler for the en_service_mon node.
ic/automake.mk | 18 +
ic/en-address-set.c | 56 +
ic/en-address-set.h | 13 +
ic/en-az.c | 127 ++
ic/en-az.h | 26 +
ic/en-gateway.c | 404 ++++
ic/en-gateway.h | 17 +
ic/en-ic.c | 9 +-
ic/en-port-binding.c | 1653 ++++++++++++++++
ic/en-port-binding.h | 41 +
ic/en-route.c | 2627 ++++++++++++++++++++++++++
ic/en-route.h | 41 +
ic/en-service-monitor.c | 555 ++++++
ic/en-service-monitor.h | 18 +
ic/en-tr.c | 206 ++
ic/en-tr.h | 17 +
ic/en-ts.c | 216 +++
ic/en-ts.h | 21 +
ic/en-tunnel-key.c | 426 +++++
ic/en-tunnel-key.h | 48 +
ic/inc-proc-ic.c | 273 ++-
ic/inc-proc-ic.h | 7 +
ic/ovn-ic.c | 3738 ++++---------------------------------
ic/ovn-ic.h | 59 +-
northd/en-global-config.c | 4 +-
ovn-ic-nb.xml | 23 +
tests/automake.mk | 3 +-
tests/ovn-ic.at | 659 +++++++
tests/perf-ovn-ic.at | 253 +++
tests/perf-testsuite.at | 1 +
30 files changed, 8161 insertions(+), 3398 deletions(-)
create mode 100644 ic/en-address-set.c
create mode 100644 ic/en-address-set.h
create mode 100644 ic/en-az.c
create mode 100644 ic/en-az.h
create mode 100644 ic/en-gateway.c
create mode 100644 ic/en-gateway.h
create mode 100644 ic/en-port-binding.c
create mode 100644 ic/en-port-binding.h
create mode 100644 ic/en-route.c
create mode 100644 ic/en-route.h
create mode 100644 ic/en-service-monitor.c
create mode 100644 ic/en-service-monitor.h
create mode 100644 ic/en-tr.c
create mode 100644 ic/en-tr.h
create mode 100644 ic/en-ts.c
create mode 100644 ic/en-ts.h
create mode 100644 ic/en-tunnel-key.c
create mode 100644 ic/en-tunnel-key.h
create mode 100644 tests/perf-ovn-ic.at
--
2.34.1
--
_'Esta mensagem é direcionada apenas para os endereços constantes no
cabeçalho inicial. Se você não está listado nos endereços constantes no
cabeçalho, pedimos-lhe que desconsidere completamente o conteúdo dessa
mensagem e cuja cópia, encaminhamento e/ou execução das ações citadas estão
imediatamente anuladas e proibidas'._
* **'Apesar do Magazine Luiza tomar
todas as precauções razoáveis para assegurar que nenhum vírus esteja
presente nesse e-mail, a empresa não poderá aceitar a responsabilidade por
quaisquer perdas ou danos causados por esse e-mail ou por seus anexos'.*
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
