Introduce the en_az engine node: move the availability-zone computation (previously done inline in ovn_db_run) into a dedicated node that resolves ctx->runned_az, which the downstream subsystem nodes gate on.
Each subsystem node NULL-checks the resolved AZ before running, so the AZ is handed to the subsystem entry points (gateway_run(), port_binding_run(), route_run(), sync_service_monitor() and address_set_run()) as an explicit parameter instead of being re-read from engine state deep inside them. inc_proc_ic_get_runned_az() remains for the main loop, which runs outside the engine. Also wire the ic-backoff-interval-ms option to throttle the engine loop (IC_NB_Global:options:ic-backoff-interval-ms), so a busy IC-SB does not spin ovn-ic at full speed. Assisted-by: Claude Opus 4.8, Claude Code Signed-off-by: Paulo Guilherme Silva <[email protected]> --- NEWS | 6 ++ ic/automake.mk | 2 + ic/en-address-set.c | 17 +++- ic/en-az.c | 127 ++++++++++++++++++++++++++ ic/en-az.h | 24 +++++ ic/en-gateway.c | 12 ++- ic/en-port-binding.c | 12 ++- ic/en-route.c | 12 ++- ic/en-service-monitor.c | 12 ++- ic/en-tr.c | 8 ++ ic/en-ts.c | 8 ++ ic/inc-proc-ic.c | 26 +++++- ic/inc-proc-ic.h | 7 ++ ic/ovn-ic.c | 194 +++++++++++++++++----------------------- ic/ovn-ic.h | 17 ++-- ovn-ic-nb.xml | 23 +++++ 16 files changed, 376 insertions(+), 131 deletions(-) create mode 100644 ic/en-az.c create mode 100644 ic/en-az.h diff --git a/NEWS b/NEWS index 44f117807..8b25bc27c 100644 --- a/NEWS +++ b/NEWS @@ -94,6 +94,12 @@ Post v26.03.0 - Added a new "ovn-debug lflow-pipeline-oftable-start-list" command that prints the starting OpenFlow table number of the logical ingress and egress pipelines. + - Added a new "options:ic-backoff-interval-ms" key to the IC_NB_Global + table. It caps how long ovn-ic postpones the next run of its + incremental-processing engine after the previous one finished, + coalescing bursts of change-driven runs into fewer, larger runs. It + defaults to 0, which keeps running as soon as there is work to do. A + forced full recompute is never delayed. OVN v26.03.0 - xxx xx xxxx -------------------------- diff --git a/ic/automake.mk b/ic/automake.mk index fd888d60c..b730fc578 100644 --- a/ic/automake.mk +++ b/ic/automake.mk @@ -4,6 +4,8 @@ ic_ovn_ic_SOURCES = ic/ovn-ic.c \ ic/ovn-ic.h \ ic/en-ic.c \ ic/en-ic.h \ + ic/en-az.c \ + ic/en-az.h \ ic/en-dp-enum.c \ ic/en-dp-enum.h \ ic/en-gateway.c \ diff --git a/ic/en-address-set.c b/ic/en-address-set.c index 764dd71d2..5a29f8201 100644 --- a/ic/en-address-set.c +++ b/ic/en-address-set.c @@ -15,6 +15,7 @@ #include <config.h> #include "en-address-set.h" +#include "en-az.h" #include "lib/inc-proc-eng.h" #include "openvswitch/vlog.h" #include "ovn-ic.h" @@ -22,12 +23,22 @@ VLOG_DEFINE_THIS_MODULE(en_ic_address_set); enum engine_node_state -en_address_set_run(struct engine_node *node OVS_UNUSED, void *data OVS_UNUSED) +en_address_set_run(struct engine_node *node, void *data OVS_UNUSED) { const struct engine_context *eng_ctx = engine_get_context(); struct ic_context *ctx = eng_ctx->client_ctx; - - address_set_run(ctx); + const struct ed_type_az *az = engine_get_input_data("az", node); + + /* runned_az is resolved by the upstream en_az node. Without an AZ there + * is nothing to sync; this also mirrors the previous main-loop gating and + * avoids running address_set_run() before the NB Global row exists (en_az + * leaves runned_az NULL until then, and address_set_run() asserts on a + * NULL NB Global). */ + if (!az->runned_az) { + return EN_UNCHANGED; + } + + address_set_run(ctx, az->runned_az); return EN_UPDATED; } diff --git a/ic/en-az.c b/ic/en-az.c new file mode 100644 index 000000000..4d3c9bc71 --- /dev/null +++ b/ic/en-az.c @@ -0,0 +1,127 @@ +/* + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include <config.h> + +#include "en-az.h" +#include "lib/inc-proc-eng.h" +#include "lib/ovn-ic-sb-idl.h" +#include "lib/ovn-nb-idl.h" +#include "openvswitch/vlog.h" +#include "ovn-ic.h" +#include "ovsdb-idl.h" +#include "util.h" + +VLOG_DEFINE_THIS_MODULE(en_ic_az); + +/* Resolves (and, if needed, registers) the availability zone for this ovn-ic + * instance. This node has no engine inputs: like the former az_run() helper + * it runs on every engine iteration and reads the NB Global and IC-SB + * Availability_Zone tables directly. The resolved AZ is stored in this node's + * own data (struct ed_type_az): dependent nodes declare en_az as an input and + * read it with engine_get_input_data("az", node), and the main loop reads it + * with inc_proc_ic_get_runned_az() after inc_proc_ic_run(). It reports + * EN_UPDATED only when the resolved AZ identity changes so dependent nodes do + * not recompute on every iteration. */ +enum engine_node_state +en_az_run(struct engine_node *node OVS_UNUSED, void *data_) +{ + const struct engine_context *eng_ctx = engine_get_context(); + struct ic_context *ctx = eng_ctx->client_ctx; + struct ed_type_az *data = data_; + const struct icsbrec_availability_zone *old_az = data->runned_az; + const struct icsbrec_availability_zone *az; + + data->runned_az = NULL; + + const struct nbrec_nb_global *nb_global = + nbrec_nb_global_first(ctx->ovnnb_idl); + if (!nb_global) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, "NB Global does not exist."); + goto out; + } + + /* Update old AZ if name changes. Note: if name changed when ovn-ic + * is not running, one has to manually delete/update the old AZ with: + * "ovn-ic-sbctl destroy avail <az>". */ + if (ctx->ovnisb_unlocked_txn && data->az_name + && strcmp(data->az_name, nb_global->name)) { + ICSBREC_AVAILABILITY_ZONE_FOR_EACH (az, ctx->ovnisb_unlocked_idl) { + /* AZ name update locally need to update az in ISB. */ + if (nb_global->name[0] && !strcmp(az->name, data->az_name)) { + icsbrec_availability_zone_set_name(az, nb_global->name); + break; + } else if (!nb_global->name[0] + && !strcmp(az->name, data->az_name)) { + icsbrec_availability_zone_delete(az); + break; + } + } + free(data->az_name); + data->az_name = NULL; + } + + if (!nb_global->name[0]) { + goto out; + } + + if (!data->az_name) { + data->az_name = xstrdup(nb_global->name); + } + + if (ctx->ovnisb_unlocked_txn) { + ovsdb_idl_txn_add_comment(ctx->ovnisb_unlocked_txn, "AZ %s", + data->az_name); + } + + ICSBREC_AVAILABILITY_ZONE_FOR_EACH (az, ctx->ovnisb_unlocked_idl) { + if (!strcmp(az->name, data->az_name)) { + data->runned_az = az; + goto out; + } + } + + /* Create AZ in ISB */ + if (ctx->ovnisb_unlocked_txn) { + VLOG_INFO("Register AZ %s to interconnection DB.", data->az_name); + az = icsbrec_availability_zone_insert(ctx->ovnisb_unlocked_txn); + icsbrec_availability_zone_set_name(az, data->az_name); + data->runned_az = az; + } + +out: { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_DBG_RL(&rl, "Availability zone: %s", + data->runned_az ? data->runned_az->name : "not created yet."); + + return data->runned_az == old_az ? EN_UNCHANGED : EN_UPDATED; +} +} + +void * +en_az_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + return xzalloc(sizeof(struct ed_type_az)); +} + +void +en_az_cleanup(void *data_) +{ + struct ed_type_az *data = data_; + + free(data->az_name); + data->az_name = NULL; +} diff --git a/ic/en-az.h b/ic/en-az.h new file mode 100644 index 000000000..875e9a03e --- /dev/null +++ b/ic/en-az.h @@ -0,0 +1,24 @@ +#ifndef EN_IC_AZ_H +#define EN_IC_AZ_H 1 + +#include "lib/inc-proc-eng.h" + +struct icsbrec_availability_zone; + +/* Data maintained by the "az" engine node: the availability zone this ovn-ic + * instance runs for. This replaces the former az_run() helper and its file + * scope 'az_name' static variable. */ +struct ed_type_az { + /* The availability zone record for this instance, or NULL if it is not + * known/registered yet. Owned by the IDL. */ + const struct icsbrec_availability_zone *runned_az; + /* The name this instance registered as, persisted across iterations so a + * later rename can update the existing IC-SB record. */ + char *az_name; +}; + +enum engine_node_state en_az_run(struct engine_node *node, void *data); +void *en_az_init(struct engine_node *node, struct engine_arg *arg); +void en_az_cleanup(void *data); + +#endif /* EN_IC_AZ_H */ diff --git a/ic/en-gateway.c b/ic/en-gateway.c index f41166018..3fb00ced9 100644 --- a/ic/en-gateway.c +++ b/ic/en-gateway.c @@ -15,6 +15,7 @@ #include <config.h> #include "en-gateway.h" +#include "en-az.h" #include "lib/inc-proc-eng.h" #include "openvswitch/vlog.h" #include "ovn-ic.h" @@ -22,12 +23,19 @@ VLOG_DEFINE_THIS_MODULE(en_ic_gateway); enum engine_node_state -en_gateway_run(struct engine_node *node OVS_UNUSED, void *data OVS_UNUSED) +en_gateway_run(struct engine_node *node, void *data OVS_UNUSED) { const struct engine_context *eng_ctx = engine_get_context(); struct ic_context *ctx = eng_ctx->client_ctx; + const struct ed_type_az *az = engine_get_input_data("az", node); - gateway_run(ctx); + /* runned_az is resolved by the upstream en_az node. Without an AZ there + * is nothing to sync (mirrors the previous main-loop gating). */ + if (!az->runned_az) { + return EN_UNCHANGED; + } + + gateway_run(ctx, az->runned_az); return EN_UPDATED; } diff --git a/ic/en-port-binding.c b/ic/en-port-binding.c index 8843d6cef..838b86f69 100644 --- a/ic/en-port-binding.c +++ b/ic/en-port-binding.c @@ -15,6 +15,7 @@ #include <config.h> #include "en-port-binding.h" +#include "en-az.h" #include "lib/inc-proc-eng.h" #include "openvswitch/vlog.h" #include "ovn-ic.h" @@ -22,12 +23,19 @@ VLOG_DEFINE_THIS_MODULE(en_ic_port_binding); enum engine_node_state -en_port_binding_run(struct engine_node *node OVS_UNUSED, void *data OVS_UNUSED) +en_port_binding_run(struct engine_node *node, void *data OVS_UNUSED) { const struct engine_context *eng_ctx = engine_get_context(); struct ic_context *ctx = eng_ctx->client_ctx; + const struct ed_type_az *az = engine_get_input_data("az", node); - port_binding_run(ctx); + /* runned_az is resolved by the upstream en_az node. Without an AZ there + * is nothing to sync (mirrors the previous main-loop gating). */ + if (!az->runned_az) { + return EN_UNCHANGED; + } + + port_binding_run(ctx, az->runned_az); return EN_UPDATED; } diff --git a/ic/en-route.c b/ic/en-route.c index ec39d89d9..a6d8ccf6e 100644 --- a/ic/en-route.c +++ b/ic/en-route.c @@ -15,6 +15,7 @@ #include <config.h> #include "en-route.h" +#include "en-az.h" #include "lib/inc-proc-eng.h" #include "openvswitch/vlog.h" #include "ovn-ic.h" @@ -22,12 +23,19 @@ VLOG_DEFINE_THIS_MODULE(en_ic_route); enum engine_node_state -en_route_run(struct engine_node *node OVS_UNUSED, void *data OVS_UNUSED) +en_route_run(struct engine_node *node, void *data OVS_UNUSED) { const struct engine_context *eng_ctx = engine_get_context(); struct ic_context *ctx = eng_ctx->client_ctx; + const struct ed_type_az *az = engine_get_input_data("az", node); - route_run(ctx); + /* runned_az is resolved by the upstream en_az node. Without an AZ there + * is nothing to sync (mirrors the previous main-loop gating). */ + if (!az->runned_az) { + return EN_UNCHANGED; + } + + route_run(ctx, az->runned_az); return EN_UPDATED; } diff --git a/ic/en-service-monitor.c b/ic/en-service-monitor.c index b85a36dc7..d0aab60bb 100644 --- a/ic/en-service-monitor.c +++ b/ic/en-service-monitor.c @@ -15,6 +15,7 @@ #include <config.h> #include "en-service-monitor.h" +#include "en-az.h" #include "lib/inc-proc-eng.h" #include "openvswitch/vlog.h" #include "ovn-ic.h" @@ -22,13 +23,20 @@ VLOG_DEFINE_THIS_MODULE(en_ic_service_monitor); enum engine_node_state -en_service_monitor_run(struct engine_node *node OVS_UNUSED, +en_service_monitor_run(struct engine_node *node, void *data OVS_UNUSED) { const struct engine_context *eng_ctx = engine_get_context(); struct ic_context *ctx = eng_ctx->client_ctx; + const struct ed_type_az *az = engine_get_input_data("az", node); - sync_service_monitor(ctx); + /* runned_az is resolved by the upstream en_az node. Without an AZ there + * is nothing to sync (mirrors the previous main-loop gating). */ + if (!az->runned_az) { + return EN_UNCHANGED; + } + + sync_service_monitor(ctx, az->runned_az); return EN_UPDATED; } diff --git a/ic/en-tr.c b/ic/en-tr.c index 4f05a4325..c3b1b60c6 100644 --- a/ic/en-tr.c +++ b/ic/en-tr.c @@ -16,6 +16,7 @@ #include "en-dp-enum.h" #include "en-tr.h" +#include "en-az.h" #include "lib/inc-proc-eng.h" #include "openvswitch/vlog.h" #include "ovn-ic.h" @@ -28,6 +29,13 @@ en_tr_run(struct engine_node *node, void *data OVS_UNUSED) const struct engine_context *eng_ctx = engine_get_context(); struct ic_context *ctx = eng_ctx->client_ctx; struct ed_type_dp_enum *dp = engine_get_input_data("dp_enum", node); + const struct ed_type_az *az = engine_get_input_data("az", node); + + /* runned_az is resolved by the upstream en_az node. Without an AZ there + * is nothing to sync (mirrors the previous main-loop gating). */ + if (!az->runned_az) { + return EN_UNCHANGED; + } tr_run(ctx, &dp->dp_tnlids, &dp->isb_tr_dps); diff --git a/ic/en-ts.c b/ic/en-ts.c index e8df1888a..e4ec2a0b7 100644 --- a/ic/en-ts.c +++ b/ic/en-ts.c @@ -16,6 +16,7 @@ #include "en-dp-enum.h" #include "en-ts.h" +#include "en-az.h" #include "lib/inc-proc-eng.h" #include "openvswitch/vlog.h" #include "ovn-ic.h" @@ -28,6 +29,13 @@ en_ts_run(struct engine_node *node, void *data OVS_UNUSED) const struct engine_context *eng_ctx = engine_get_context(); struct ic_context *ctx = eng_ctx->client_ctx; struct ed_type_dp_enum *dp = engine_get_input_data("dp_enum", node); + const struct ed_type_az *az = engine_get_input_data("az", node); + + /* runned_az is resolved by the upstream en_az node. Without an AZ there + * is nothing to sync (mirrors the previous main-loop gating). */ + if (!az->runned_az) { + return EN_UNCHANGED; + } ts_run(ctx, &dp->dp_tnlids, &dp->isb_ts_dps); diff --git a/ic/inc-proc-ic.c b/ic/inc-proc-ic.c index a6f029e37..66958dbf0 100644 --- a/ic/inc-proc-ic.c +++ b/ic/inc-proc-ic.c @@ -27,6 +27,7 @@ #include "openvswitch/vlog.h" #include "inc-proc-ic.h" #include "en-ic.h" +#include "en-az.h" #include "en-dp-enum.h" #include "en-gateway.h" #include "en-ts.h" @@ -173,6 +174,7 @@ VLOG_DEFINE_THIS_MODULE(inc_proc_ic); /* Define engine nodes for other nodes. They should be defined as static to * avoid sparse errors. */ +static ENGINE_NODE(az); static ENGINE_NODE(dp_enum); static ENGINE_NODE(gateway); static ENGINE_NODE(ts); @@ -215,6 +217,7 @@ void inc_proc_ic_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_dp_enum, &en_icnb_ic_nb_global, NULL); /* en_gateway: sync gateways/chassis between SB and IC-SB. */ + engine_add_input(&en_gateway, &en_az, NULL); engine_add_input(&en_gateway, &en_icsb_availability_zone, NULL); engine_add_input(&en_gateway, &en_icsb_gateway, NULL); engine_add_input(&en_gateway, &en_icsb_encap, NULL); @@ -222,6 +225,7 @@ void inc_proc_ic_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_gateway, &en_sb_encap, NULL); /* en_ts: sync transit switches to NB and IC-SB datapath bindings. */ + engine_add_input(&en_ts, &en_az, NULL); engine_add_input(&en_ts, &en_dp_enum, NULL); engine_add_input(&en_ts, &en_icnb_ic_nb_global, NULL); engine_add_input(&en_ts, &en_icnb_transit_switch, NULL); @@ -229,11 +233,13 @@ void inc_proc_ic_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_ts, &en_icsb_encap, NULL); /* en_tr: sync transit routers to NB and IC-SB datapath bindings. */ + engine_add_input(&en_tr, &en_az, NULL); engine_add_input(&en_tr, &en_dp_enum, NULL); engine_add_input(&en_tr, &en_icnb_transit_router, NULL); engine_add_input(&en_tr, &en_nb_logical_router, NULL); /* en_port_binding: sync cross-AZ port bindings. */ + engine_add_input(&en_port_binding, &en_az, NULL); engine_add_input(&en_port_binding, &en_icsb_availability_zone, NULL); engine_add_input(&en_port_binding, &en_icsb_port_binding, NULL); engine_add_input(&en_port_binding, &en_icnb_transit_switch, NULL); @@ -247,6 +253,7 @@ void inc_proc_ic_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_port_binding, &en_sb_chassis, NULL); /* en_route: advertise/learn cross-AZ routes. */ + engine_add_input(&en_route, &en_az, NULL); engine_add_input(&en_route, &en_icsb_availability_zone, NULL); engine_add_input(&en_route, &en_icsb_port_binding, NULL); engine_add_input(&en_route, &en_icsb_route, NULL); @@ -262,13 +269,21 @@ void inc_proc_ic_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_route, &en_sb_learned_route, NULL); /* en_service_monitor: sync load-balancer health checks across AZs. */ + engine_add_input(&en_service_monitor, &en_az, NULL); engine_add_input(&en_service_monitor, &en_icsb_availability_zone, NULL); engine_add_input(&en_service_monitor, &en_icsb_service_monitor, NULL); engine_add_input(&en_service_monitor, &en_sb_sb_global, NULL); engine_add_input(&en_service_monitor, &en_sb_service_monitor, NULL); engine_add_input(&en_service_monitor, &en_sb_port_binding, NULL); - /* en_address_set: advertise/learn address sets across AZs. */ + /* en_address_set: advertise/learn address sets across AZs. + * + * Like the other AZ-scoped nodes, address_set_run() partitions IC-SB + * address sets into local/remote by comparing their availability_zone + * against this instance's AZ, so it depends on en_az (which reports + * EN_UPDATED only when the AZ identity changes) to be re-run when the + * resolved AZ changes. */ + engine_add_input(&en_address_set, &en_az, NULL); engine_add_input(&en_address_set, &en_icsb_availability_zone, NULL); engine_add_input(&en_address_set, &en_nb_nb_global, NULL); engine_add_input(&en_address_set, &en_nb_address_set, NULL); @@ -331,7 +346,7 @@ inc_proc_ic_run(struct ic_context *ctx, int64_t now = time_msec(); /* Postpone the next run by length of current run with maximum capped - * by "northd-backoff-interval-ms" interval. */ + * by "ic-backoff-interval-ms" interval. */ ic_eng_ctx->next_run_ms = now + MIN(now - start, ic_eng_ctx->backoff_ms); return engine_has_updated(); @@ -344,6 +359,13 @@ inc_proc_ic_cleanup(void) engine_set_context(NULL); } +const struct icsbrec_availability_zone * +inc_proc_ic_get_runned_az(void) +{ + const struct ed_type_az *az = engine_get_data(&en_az); + return az ? az->runned_az : NULL; +} + bool inc_proc_ic_can_run(struct ic_engine_context *ctx) { diff --git a/ic/inc-proc-ic.h b/ic/inc-proc-ic.h index 30e3071ae..8c03a1e7b 100644 --- a/ic/inc-proc-ic.h +++ b/ic/inc-proc-ic.h @@ -26,6 +26,13 @@ bool inc_proc_ic_run(struct ic_context *ctx, void inc_proc_ic_cleanup(void); bool inc_proc_ic_can_run(struct ic_engine_context *ctx); +struct icsbrec_availability_zone; + +/* Returns the availability zone resolved by the en_az engine node during the + * last inc_proc_ic_run(), or NULL if none is resolved yet. Used by the main + * loop, which runs the AZ sequence-number bookkeeping outside the engine. */ +const struct icsbrec_availability_zone *inc_proc_ic_get_runned_az(void); + static inline void inc_proc_ic_force_recompute(void) { diff --git a/ic/ovn-ic.c b/ic/ovn-ic.c index af3107e35..9c21ef1de 100644 --- a/ic/ovn-ic.c +++ b/ic/ovn-ic.c @@ -99,68 +99,6 @@ Options:\n\ stream_usage("database", true, true, false); } -static const struct icsbrec_availability_zone * -az_run(struct ic_context *ctx) -{ - const struct nbrec_nb_global *nb_global = - nbrec_nb_global_first(ctx->ovnnb_idl); - - if (!nb_global) { - VLOG_INFO("NB Global not exist."); - return NULL; - } - - /* Update old AZ if name changes. Note: if name changed when ovn-ic - * is not running, one has to manually delete/update the old AZ with: - * "ovn-ic-sbctl destroy avail <az>". */ - static char *az_name; - const struct icsbrec_availability_zone *az; - if (ctx->ovnisb_unlocked_txn && az_name - && strcmp(az_name, nb_global->name)) { - ICSBREC_AVAILABILITY_ZONE_FOR_EACH (az, ctx->ovnisb_unlocked_idl) { - /* AZ name update locally need to update az in ISB. */ - if (nb_global->name[0] && !strcmp(az->name, az_name)) { - icsbrec_availability_zone_set_name(az, nb_global->name); - break; - } else if (!nb_global->name[0] && !strcmp(az->name, az_name)) { - icsbrec_availability_zone_delete(az); - break; - } - } - free(az_name); - az_name = NULL; - } - - if (!nb_global->name[0]) { - return NULL; - } - - if (!az_name) { - az_name = xstrdup(nb_global->name); - } - - if (ctx->ovnisb_unlocked_txn) { - ovsdb_idl_txn_add_comment(ctx->ovnisb_unlocked_txn, "AZ %s", az_name); - } - - ICSBREC_AVAILABILITY_ZONE_FOR_EACH (az, ctx->ovnisb_unlocked_idl) { - if (!strcmp(az->name, az_name)) { - ctx->runned_az = az; - return az; - } - } - - /* Create AZ in ISB */ - if (ctx->ovnisb_unlocked_txn) { - VLOG_INFO("Register AZ %s to interconnection DB.", az_name); - az = icsbrec_availability_zone_insert(ctx->ovnisb_unlocked_txn); - icsbrec_availability_zone_set_name(az, az_name); - ctx->runned_az = az; - return az; - } - return NULL; -} - static uint32_t allocate_dp_key(struct hmap *dp_tnlids, bool vxlan_mode, const char *name) { @@ -611,7 +549,8 @@ sync_addr_set_from_icsb(struct ovsdb_idl_txn *ovnnb_txn, } void -address_set_run(struct ic_context *ctx) +address_set_run(struct ic_context *ctx, + const struct icsbrec_availability_zone *runned_az) { if (!ctx->ovnisb_unlocked_txn || !ctx->ovnnb_txn || !ctx->ovnsb_txn) { return; @@ -621,7 +560,7 @@ address_set_run(struct ic_context *ctx) struct shash ic_remote_as = SHASH_INITIALIZER(&ic_remote_as); const struct icsbrec_address_set *ic_as; ICSBREC_ADDRESS_SET_FOR_EACH (ic_as, ctx->ovnisb_unlocked_idl) { - if (ic_as->availability_zone == ctx->runned_az) { + if (ic_as->availability_zone == runned_az) { shash_add(&ic_local_as, ic_as->name, ic_as); } else { /* Merge addresses from all remote AZs that share the same @@ -659,7 +598,7 @@ address_set_run(struct ic_context *ctx) const struct icsbrec_address_set *icsb_as; icsb_as = shash_find_and_delete(&ic_local_as, sb_as->name); sync_addr_set_to_icsb(ctx->ovnisb_unlocked_txn, sb_as, icsb_as, - ctx->runned_az); + runned_az); } } } @@ -711,7 +650,8 @@ address_set_run(struct ic_context *ctx) } void -gateway_run(struct ic_context *ctx) +gateway_run(struct ic_context *ctx, + const struct icsbrec_availability_zone *runned_az) { if (!ctx->ovnisb_unlocked_txn || !ctx->ovnsb_txn) { return; @@ -721,7 +661,7 @@ gateway_run(struct ic_context *ctx) struct shash remote_gws = SHASH_INITIALIZER(&remote_gws); const struct icsbrec_gateway *gw; ICSBREC_GATEWAY_FOR_EACH (gw, ctx->ovnisb_unlocked_idl) { - if (gw->availability_zone == ctx->runned_az) { + if (gw->availability_zone == runned_az) { shash_add(&local_gws, gw->name, gw); } else { shash_add(&remote_gws, gw->name, gw); @@ -734,7 +674,8 @@ gateway_run(struct ic_context *ctx) gw = shash_find_and_delete(&local_gws, chassis->name); if (!gw) { gw = icsbrec_gateway_insert(ctx->ovnisb_unlocked_txn); - icsbrec_gateway_set_availability_zone(gw, ctx->runned_az); + icsbrec_gateway_set_availability_zone(gw, + runned_az); icsbrec_gateway_set_name(gw, chassis->name); sync_sb_gw_to_isb(ctx, chassis, gw); } else if (is_gateway_data_changed(gw, chassis)) { @@ -1400,7 +1341,8 @@ find_lsp_in_sb(struct ic_context *ctx, } void -port_binding_run(struct ic_context *ctx) +port_binding_run(struct ic_context *ctx, + const struct icsbrec_availability_zone *runned_az) { if (!ctx->ovnisb_unlocked_txn || !ctx->ovnnb_txn || !ctx->ovnsb_txn) { return; @@ -1417,7 +1359,7 @@ port_binding_run(struct ic_context *ctx) const struct icsbrec_port_binding *isb_pb_key = icsbrec_port_binding_index_init_row(ctx->icsbrec_port_binding_by_az); icsbrec_port_binding_index_set_availability_zone(isb_pb_key, - ctx->runned_az); + runned_az); ICSBREC_PORT_BINDING_FOR_EACH_EQUAL (isb_pb, isb_pb_key, ctx->icsbrec_port_binding_by_az) { @@ -1456,7 +1398,7 @@ port_binding_run(struct ic_context *ctx) ICSBREC_PORT_BINDING_FOR_EACH_EQUAL (isb_pb, isb_pb_key, ctx->icsbrec_port_binding_by_ts) { - if (isb_pb->availability_zone == ctx->runned_az) { + if (isb_pb->availability_zone == runned_az) { shash_add(&local_pbs, isb_pb->logical_port, isb_pb); shash_find_and_delete(&switch_all_local_pbs, isb_pb->logical_port); @@ -1478,8 +1420,9 @@ port_binding_run(struct ic_context *ctx) if (ctx->ovnisb_txn && is_az_leader(ctx->ovnisb_txn)) { if (!isb_pb) { isb_pb = create_isb_pb( - ctx->ovnisb_txn, tsp->name, ctx->runned_az, - ts->name, &ts->header_.uuid, "transit-switch-port", + ctx->ovnisb_txn, tsp->name, + runned_az, ts->name, + &ts->header_.uuid, "transit-switch-port", &pb_tnlids); } sync_tsp_pb(tsp, isb_pb); @@ -1489,7 +1432,7 @@ port_binding_run(struct ic_context *ctx) isb_pb = shash_find_and_delete(&local_pbs, tsp->name); if (!isb_pb) { isb_pb = create_isb_pb(ctx->ovnisb_unlocked_txn, tsp->name, - ctx->runned_az, + runned_az, ts->name, &ts->header_.uuid, "transit-switch-port", &pb_tnlids); } @@ -1532,8 +1475,8 @@ port_binding_run(struct ic_context *ctx) if (!isb_pb) { isb_pb = create_isb_pb( ctx->ovnisb_unlocked_txn, sb_pb->logical_port, - ctx->runned_az, ts->name, &ts->header_.uuid, - "transit-switch-port", &pb_tnlids); + runned_az, ts->name, + &ts->header_.uuid, "transit-switch-port", &pb_tnlids); sync_ts_isb_pb(ctx, sb_pb, isb_pb); } else { sync_local_port(ctx, isb_pb, sb_pb, lsp); @@ -1617,7 +1560,7 @@ port_binding_run(struct ic_context *ctx) ICSBREC_PORT_BINDING_FOR_EACH_EQUAL (isb_pb, isb_pb_key, ctx->icsbrec_port_binding_by_ts) { - if (isb_pb->availability_zone == ctx->runned_az) { + if (isb_pb->availability_zone == runned_az) { shash_add(&local_pbs, isb_pb->logical_port, isb_pb); shash_find_and_delete(&router_all_local_pbs, isb_pb->logical_port); @@ -1636,7 +1579,7 @@ port_binding_run(struct ic_context *ctx) isb_pb = shash_find_and_delete(&local_pbs, trp->name); if (!isb_pb) { isb_pb = create_isb_pb(ctx->ovnisb_unlocked_txn, trp->name, - ctx->runned_az, + runned_az, tr->name, &tr->header_.uuid, "transit-router-port", &pb_tnlids); icsbrec_port_binding_set_address(isb_pb, trp->mac); @@ -2528,6 +2471,7 @@ lrp_is_ts_port(struct ic_context *ctx, struct ic_router_info *ic_lr, static void sync_learned_routes(struct ic_context *ctx, + const struct icsbrec_availability_zone *runned_az, struct ic_router_info *ic_lr) { ovs_assert(ctx->ovnnb_txn); @@ -2572,7 +2516,8 @@ sync_learned_routes(struct ic_context *ctx, || uuid_equals(&ic_lr->lr->header_.uuid, &lr_uuid)) { continue; } - } else if (isb_route->availability_zone == ctx->runned_az) { + } else if (isb_route->availability_zone == + runned_az) { continue; } @@ -3020,20 +2965,21 @@ delete_orphan_ic_routes(struct ic_context *ctx, } void -route_run(struct ic_context *ctx) +route_run(struct ic_context *ctx, + const struct icsbrec_availability_zone *runned_az) { if (!ctx->ovnisb_unlocked_txn || !ctx->ovnnb_txn || !ctx->ovnsb_txn) { return; } - delete_orphan_ic_routes(ctx, ctx->runned_az); + delete_orphan_ic_routes(ctx, runned_az); struct hmap ic_lrs = HMAP_INITIALIZER(&ic_lrs); const struct icsbrec_port_binding *isb_pb; const struct icsbrec_port_binding *isb_pb_key = icsbrec_port_binding_index_init_row(ctx->icsbrec_port_binding_by_az); icsbrec_port_binding_index_set_availability_zone(isb_pb_key, - ctx->runned_az); + runned_az); /* Each port on TS maps to a logical router, which is stored in the * external_ids:router-id of the IC SB port_binding record. @@ -3110,7 +3056,7 @@ route_run(struct ic_context *ctx) struct shash routes_ad_by_ts = SHASH_INITIALIZER(&routes_ad_by_ts); HMAP_FOR_EACH_SAFE (ic_lr, node, &ic_lrs) { collect_lr_routes(ctx, ic_lr, &routes_ad_by_ts); - sync_learned_routes(ctx, ic_lr); + sync_learned_routes(ctx, runned_az, ic_lr); vector_destroy(&ic_lr->isb_pbs); hmap_destroy(&ic_lr->routes_learned); hmap_remove(&ic_lrs, &ic_lr->node); @@ -3118,7 +3064,8 @@ route_run(struct ic_context *ctx) } struct shash_node *node; SHASH_FOR_EACH (node, &routes_ad_by_ts) { - advertise_routes(ctx, ctx->runned_az, node->name, node->data); + advertise_routes(ctx, runned_az, node->name, + node->data); hmap_destroy(node->data); } shash_destroy_free_data(&routes_ad_by_ts); @@ -3253,6 +3200,7 @@ remove_unused_sb_records(struct hmap *local_sb_svcs_map) static void create_pushed_svcs_mon(struct ic_context *ctx, + const struct icsbrec_availability_zone *runned_az, struct hmap *pushed_svcs_map) { struct sbrec_service_monitor *key = @@ -3271,8 +3219,8 @@ create_pushed_svcs_mon(struct ic_context *ctx, } create_service_monitor_info(pushed_svcs_map, sb_rec, &sb_rec->header_.uuid, - ctx->runned_az->name, target_az_name, - NULL, false); + runned_az->name, + target_az_name, NULL, false); } sbrec_service_monitor_index_destroy_row(key); @@ -3280,6 +3228,7 @@ create_pushed_svcs_mon(struct ic_context *ctx, static void create_synced_svcs_mon(struct ic_context *ctx, + const struct icsbrec_availability_zone *runned_az, struct hmap *synced_svcs_map) { struct icsbrec_service_monitor *key = @@ -3287,7 +3236,7 @@ create_synced_svcs_mon(struct ic_context *ctx, ctx->icsbrec_service_monitor_by_target_az); icsbrec_service_monitor_index_set_target_availability_zone( - key, ctx->runned_az->name); + key, runned_az->name); const struct icsbrec_service_monitor *ic_rec; ICSBREC_SERVICE_MONITOR_FOR_EACH_EQUAL (ic_rec, key, @@ -3304,7 +3253,7 @@ create_synced_svcs_mon(struct ic_context *ctx, const char *chassis_name = pb->chassis ? pb->chassis->name : NULL; create_service_monitor_info(synced_svcs_map, ic_rec, &ic_rec->header_.uuid, - ctx->runned_az->name, + runned_az->name, NULL, chassis_name, true); } @@ -3313,6 +3262,7 @@ create_synced_svcs_mon(struct ic_context *ctx, static void create_local_ic_svcs_map(struct ic_context *ctx, + const struct icsbrec_availability_zone *runned_az, struct hmap *owned_svc_map) { struct icsbrec_service_monitor *key = @@ -3320,14 +3270,14 @@ create_local_ic_svcs_map(struct ic_context *ctx, ctx->icsbrec_service_monitor_by_source_az); icsbrec_service_monitor_index_set_source_availability_zone( - key, ctx->runned_az->name); + key, runned_az->name); const struct icsbrec_service_monitor *ic_rec; ICSBREC_SERVICE_MONITOR_FOR_EACH_EQUAL (ic_rec, key, ctx->icsbrec_service_monitor_by_source_az) { create_service_monitor_info(owned_svc_map, ic_rec, &ic_rec->header_.uuid, - ctx->runned_az->name, NULL, + runned_az->name, NULL, NULL, true); } @@ -3336,6 +3286,7 @@ create_local_ic_svcs_map(struct ic_context *ctx, static void create_local_sb_svcs_map(struct ic_context *ctx, + const struct icsbrec_availability_zone *runned_az, struct hmap *owned_svc_map) { struct sbrec_service_monitor *key = @@ -3350,7 +3301,7 @@ create_local_sb_svcs_map(struct ic_context *ctx, ctx->sbrec_service_monitor_by_ic_learned) { create_service_monitor_info(owned_svc_map, sb_rec, &sb_rec->header_.uuid, - ctx->runned_az->name, NULL, + runned_az->name, NULL, NULL, false); } @@ -3430,6 +3381,7 @@ lookup_icsb_svc_rec(struct ic_context *ctx, static void create_service_monitor_data(struct ic_context *ctx, + const struct icsbrec_availability_zone *runned_az, struct sync_service_monitor_data *sync_data) { const struct sbrec_sb_global *ic_sb = sbrec_sb_global_first( @@ -3442,10 +3394,11 @@ create_service_monitor_data(struct ic_context *ctx, } sync_data->prpg_svc_monitor_mac = xstrdup(svc_monitor_mac); - create_pushed_svcs_mon(ctx, &sync_data->pushed_svcs_map); - create_synced_svcs_mon(ctx, &sync_data->synced_svcs_map); - create_local_ic_svcs_map(ctx, &sync_data->local_ic_svcs_map); - create_local_sb_svcs_map(ctx, &sync_data->local_sb_svcs_map); + create_pushed_svcs_mon(ctx, runned_az, &sync_data->pushed_svcs_map); + create_synced_svcs_mon(ctx, runned_az, &sync_data->synced_svcs_map); + create_local_ic_svcs_map(ctx, runned_az, &sync_data->local_ic_svcs_map); + create_local_sb_svcs_map(ctx, runned_az, + &sync_data->local_sb_svcs_map); } static void @@ -3466,7 +3419,8 @@ destroy_service_monitor_data(struct sync_service_monitor_data *sync_data) } void -sync_service_monitor(struct ic_context *ctx) +sync_service_monitor(struct ic_context *ctx, + const struct icsbrec_availability_zone *runned_az) { if (!ctx->ovnisb_unlocked_txn || !ctx->ovnsb_txn) { return; @@ -3479,7 +3433,7 @@ sync_service_monitor(struct ic_context *ctx) hmap_init(&sync_data.local_ic_svcs_map); hmap_init(&sync_data.local_sb_svcs_map); - create_service_monitor_data(ctx, &sync_data); + create_service_monitor_data(ctx, runned_az, &sync_data); struct service_monitor_info *svc_mon; HMAP_FOR_EACH_SAFE (svc_mon, hmap_node, &sync_data.pushed_svcs_map) { @@ -3593,6 +3547,7 @@ sync_service_monitor(struct ic_context *ctx) */ static void update_sequence_numbers(struct ic_context *ctx, + const struct icsbrec_availability_zone *runned_az, struct ovsdb_idl_loop *ic_sb_loop) { if (!ctx->ovnisb_unlocked_txn || !ctx->ovninb_txn) { @@ -3611,14 +3566,14 @@ update_sequence_numbers(struct ic_context *ctx, } if ((ic_nb->nb_ic_cfg != ic_sb->nb_ic_cfg) && - (ic_nb->nb_ic_cfg != ctx->runned_az->nb_ic_cfg)) { + (ic_nb->nb_ic_cfg != runned_az->nb_ic_cfg)) { /* Deal with potential overflows. */ - if (ctx->runned_az->nb_ic_cfg == INT64_MAX) { - icsbrec_availability_zone_set_nb_ic_cfg(ctx->runned_az, 0); + if (runned_az->nb_ic_cfg == INT64_MAX) { + icsbrec_availability_zone_set_nb_ic_cfg(runned_az, 0); } ic_sb_loop->next_cfg = ic_nb->nb_ic_cfg; ovsdb_idl_txn_increment(ctx->ovnisb_unlocked_txn, - &ctx->runned_az->header_, + &runned_az->header_, &icsbrec_availability_zone_col_nb_ic_cfg, true); return; } @@ -3626,22 +3581,22 @@ update_sequence_numbers(struct ic_context *ctx, /* handle cases where accidentally AZ:ic_nb_cfg exceeds * the INB:ic_nb_cfg. */ - if (ctx->runned_az->nb_ic_cfg != ic_sb_loop->cur_cfg) { - icsbrec_availability_zone_set_nb_ic_cfg(ctx->runned_az, + if (runned_az->nb_ic_cfg != ic_sb_loop->cur_cfg) { + icsbrec_availability_zone_set_nb_ic_cfg(runned_az, ic_sb_loop->cur_cfg); return; } const struct icsbrec_availability_zone *other_az; ICSBREC_AVAILABILITY_ZONE_FOR_EACH (other_az, ctx->ovnisb_unlocked_idl) { - if (other_az->nb_ic_cfg != ctx->runned_az->nb_ic_cfg) { + if (other_az->nb_ic_cfg != runned_az->nb_ic_cfg) { return; } } /* All the AZs are updated successfully, update SB/NB counter. */ if (ic_nb->nb_ic_cfg != ic_sb->nb_ic_cfg) { - icsbrec_ic_sb_global_set_nb_ic_cfg(ic_sb, ctx->runned_az->nb_ic_cfg); - icnbrec_ic_nb_global_set_sb_ic_cfg(ic_nb, ctx->runned_az->nb_ic_cfg); + icsbrec_ic_sb_global_set_nb_ic_cfg(ic_sb, runned_az->nb_ic_cfg); + icnbrec_ic_nb_global_set_sb_ic_cfg(ic_nb, runned_az->nb_ic_cfg); } } @@ -4218,6 +4173,17 @@ main(int argc, char *argv[]) ovnisb_idl_loop.idl, ovnisb_unlocked_idl_loop.idl, ovninb_idl_loop.idl); + + /* Postpone the next engine run by the length of the previous one, up + * to this interval, to coalesce bursts of change-driven runs. A + * forced recompute (set_force_recompute) still runs immediately, so + * this only throttles the natural, incremental path. */ + const struct icnbrec_ic_nb_global *ic_nb_global = + icnbrec_ic_nb_global_first(ovninb_idl_loop.idl); + eng_ctx.backoff_ms = ic_nb_global + ? smap_get_uint(&ic_nb_global->options, + "ic-backoff-interval-ms", 0) : 0; + memory_run(); if (memory_should_report()) { struct simap usage = SIMAP_INITIALIZER(&usage); @@ -4388,12 +4354,16 @@ main(int argc, char *argv[]) ovsdb_idl_has_ever_connected(ctx.ovnisb_unlocked_idl)) { if (ctx.ovnnb_txn && ctx.ovnsb_txn && ctx.ovninb_txn && ctx.ovnisb_unlocked_txn && inc_proc_ic_can_run(&eng_ctx)) { - ctx.runned_az = az_run(&ctx); - VLOG_DBG("Availability zone: %s", ctx.runned_az ? - ctx.runned_az->name : "not created yet."); - if (ctx.runned_az) { - (void) inc_proc_ic_run(&ctx, &eng_ctx); - update_sequence_numbers(&ctx, + /* The availability zone is now resolved by the en_az + * engine node. The subsystem nodes skip their work when + * there is no AZ; the sequence-number bookkeeping below + * runs outside the engine, so it reads the resolved AZ via + * inc_proc_ic_get_runned_az(). */ + (void) inc_proc_ic_run(&ctx, &eng_ctx); + const struct icsbrec_availability_zone *runned_az = + inc_proc_ic_get_runned_az(); + if (runned_az) { + update_sequence_numbers(&ctx, runned_az, &ovnisb_unlocked_idl_loop); } } else if (!inc_proc_ic_get_force_recompute()) { diff --git a/ic/ovn-ic.h b/ic/ovn-ic.h index b06725373..c6d5e0a6e 100644 --- a/ic/ovn-ic.h +++ b/ic/ovn-ic.h @@ -28,7 +28,6 @@ struct ic_context { struct ovsdb_idl_txn *ovninb_txn; struct ovsdb_idl_txn *ovnisb_txn; struct ovsdb_idl_txn *ovnisb_unlocked_txn; - const struct icsbrec_availability_zone *runned_az; struct ovsdb_idl_index *nbrec_ls_by_name; struct ovsdb_idl_index *nbrec_lr_by_name; struct ovsdb_idl_index *nbrec_lrp_by_name; @@ -63,20 +62,26 @@ enum ic_port_binding_type { IC_SWITCH_PORT, IC_ROUTER_PORT, IC_PORT_MAX }; struct hmap; struct shash; +struct icsbrec_availability_zone; /* Per-subsystem entry points, invoked by the incremental-processing engine * nodes (see ic/en-*.c). Each performs a full recompute of its subsystem and * may be invoked independently when its engine inputs change. */ void enumerate_datapaths(struct ic_context *ctx, struct hmap *dp_tnlids, struct shash *isb_ts_dps, struct shash *isb_tr_dps); -void gateway_run(struct ic_context *ctx); -void address_set_run(struct ic_context *ctx); +void gateway_run(struct ic_context *ctx, + const struct icsbrec_availability_zone *runned_az); +void address_set_run(struct ic_context *ctx, + const struct icsbrec_availability_zone *runned_az); void ts_run(struct ic_context *ctx, struct hmap *dp_tnlids, struct shash *isb_ts_dps); void tr_run(struct ic_context *ctx, struct hmap *dp_tnlids, struct shash *isb_tr_dps); -void port_binding_run(struct ic_context *ctx); -void route_run(struct ic_context *ctx); -void sync_service_monitor(struct ic_context *ctx); +void port_binding_run(struct ic_context *ctx, + const struct icsbrec_availability_zone *runned_az); +void route_run(struct ic_context *ctx, + const struct icsbrec_availability_zone *runned_az); +void sync_service_monitor(struct ic_context *ctx, + const struct icsbrec_availability_zone *runned_az); #endif /* OVN_IC_H */ diff --git a/ovn-ic-nb.xml b/ovn-ic-nb.xml index f6110324c..408b44558 100644 --- a/ovn-ic-nb.xml +++ b/ovn-ic-nb.xml @@ -65,6 +65,29 @@ options are described individually below. </column> + <column name="options" key="ic-backoff-interval-ms"> + <p> + Upper bound, in milliseconds, on how long <code>ovn-ic</code> + postpones the next run of its incremental-processing engine. + After each engine run, the next run is delayed by the duration of + the run that just finished, capped by this value. This coalesces + bursts of change-driven runs into fewer, larger runs, trading a + small amount of latency for reduced CPU usage when the databases + are changing rapidly. + </p> + + <p> + This throttling only applies to the natural, change-driven + incremental path. A forced full recompute is never delayed and + always runs immediately. + </p> + + <p> + The default value is <code>0</code>, which disables the backoff so + each run is scheduled as soon as there is work to do. + </p> + </column> + <column name="options" key="ic_probe_interval"> <p> The inactivity probe interval of the connection to the OVN IC -- 2.34.1 -- _'Esta mensagem é direcionada apenas para os endereços constantes no cabeçalho inicial. Se você não está listado nos endereços constantes no cabeçalho, pedimos-lhe que desconsidere completamente o conteúdo dessa mensagem e cuja cópia, encaminhamento e/ou execução das ações citadas estão imediatamente anuladas e proibidas'._ * **'Apesar do Magazine Luiza tomar todas as precauções razoáveis para assegurar que nenhum vírus esteja presente nesse e-mail, a empresa não poderá aceitar a responsabilidade por quaisquer perdas ou danos causados por esse e-mail ou por seus anexos'.* _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
