From: Han Zhou <[email protected]> This patch introduces interconnection controller, ovn-ic, and implements the basic AZ registration feature: taking the AZ name from NB DB and create an Availability_Zone entry in IC-SB DB.
Signed-off-by: Han Zhou <[email protected]> --- Makefile.am | 1 + ic/.gitignore | 2 + ic/automake.mk | 10 ++ ic/ovn-ic.8.xml | 111 +++++++++++++++ ic/ovn-ic.c | 393 +++++++++++++++++++++++++++++++++++++++++++++++++++ ovn-nb.ovsschema | 5 +- ovn-nb.xml | 7 + tutorial/ovs-sandbox | 2 +- 8 files changed, 528 insertions(+), 3 deletions(-) create mode 100644 ic/.gitignore create mode 100644 ic/automake.mk create mode 100644 ic/ovn-ic.8.xml create mode 100644 ic/ovn-ic.c diff --git a/Makefile.am b/Makefile.am index 33c18c5..d22a220 100644 --- a/Makefile.am +++ b/Makefile.am @@ -500,4 +500,5 @@ include selinux/automake.mk include controller/automake.mk include controller-vtep/automake.mk include northd/automake.mk +include ic/automake.mk include build-aux/automake.mk diff --git a/ic/.gitignore b/ic/.gitignore new file mode 100644 index 0000000..1b73eb4 --- /dev/null +++ b/ic/.gitignore @@ -0,0 +1,2 @@ +/ovn-ic +/ovn-ic.8 diff --git a/ic/automake.mk b/ic/automake.mk new file mode 100644 index 0000000..8e71bc3 --- /dev/null +++ b/ic/automake.mk @@ -0,0 +1,10 @@ +# ovn-ic +bin_PROGRAMS += ic/ovn-ic +ic_ovn_ic_SOURCES = ic/ovn-ic.c +ic_ovn_ic_LDADD = \ + lib/libovn.la \ + $(OVSDB_LIBDIR)/libovsdb.la \ + $(OVS_LIBDIR)/libopenvswitch.la +man_MANS += ic/ovn-ic.8 +EXTRA_DIST += ic/ovn-ic.8.xml +CLEANFILES += ic/ovn-ic.8 diff --git a/ic/ovn-ic.8.xml b/ic/ovn-ic.8.xml new file mode 100644 index 0000000..7b2a333 --- /dev/null +++ b/ic/ovn-ic.8.xml @@ -0,0 +1,111 @@ +<?xml version="1.0" encoding="utf-8"?> +<manpage program="ovn-ic" section="8" title="ovn-ic"> + <h1>Name</h1> + <p>ovn-ic -- Open Virtual Network interconnection controller</p> + + <h1>Synopsis</h1> + <p><code>ovn-ic</code> [<var>options</var>]</p> + + <h1>Description</h1> + <p> + <code>ovn-ic</code>, OVN interconnection controller, is a centralized + daemon which communicates with global interconnection databases INB/ISB + to configure and exchange data with local NB/SB for interconnecting + with other OVN deployments. + </p> + + <h1>Options</h1> + <dl> + <dt><code>--ovnnb-db=<var>database</var></code></dt> + <dd> + The OVSDB database containing the OVN Northbound Database. If the + <env>OVN_NB_DB</env> environment variable is set, its value is used + as the default. Otherwise, the default is + <code>unix:@RUNDIR@/ovnnb_db.sock</code>. + </dd> + <dt><code>--ovnsb-db=<var>database</var></code></dt> + <dd> + The OVSDB database containing the OVN Southbound Database. If the + <env>OVN_SB_DB</env> environment variable is set, its value is used + as the default. Otherwise, the default is + <code>unix:@RUNDIR@/ovnsb_db.sock</code>. + </dd> + <dt><code>--ovninb-db=<var>database</var></code></dt> + <dd> + The OVSDB database containing the OVN Interconnection Northbound + Database. If the <env>OVN_INB_DB</env> environment variable is set, + its value is used as the default. Otherwise, the default is + <code>unix:@RUNDIR@/ovninb_db.sock</code>. + </dd> + <dt><code>--ovnisb-db=<var>database</var></code></dt> + <dd> + The OVSDB database containing the OVN Interconnection Southbound + Database. If the <env>OVN_ISB_DB</env> environment variable is set, + its value is used as the default. Otherwise, the default is + <code>unix:@RUNDIR@/ovnisb_db.sock</code>. + </dd> + </dl> + <p> + <var>database</var> in the above options must be an OVSDB active or + passive connection method, as described in <code>ovsdb</code>(7). + </p> + + <h2>Daemon Options</h2> + <xi:include href="lib/daemon.xml" xmlns:xi="http://www.w3.org/2003/XInclude"/> + + <h2>Logging Options</h2> + <xi:include href="lib/vlog.xml" xmlns:xi="http://www.w3.org/2003/XInclude"/> + + <h2>PKI Options</h2> + <p> + PKI configuration is required in order to use SSL for the connections to + the Northbound and Southbound databases. + </p> + <xi:include href="lib/ssl.xml" xmlns:xi="http://www.w3.org/2003/XInclude"/> + + <h2>Other Options</h2> + <xi:include href="lib/unixctl.xml" + xmlns:xi="http://www.w3.org/2003/XInclude"/> + <h3></h3> + <xi:include href="lib/common.xml" + xmlns:xi="http://www.w3.org/2003/XInclude"/> + + <h1>Runtime Management Commands</h1> + <p> + <code>ovs-appctl</code> can send commands to a running + <code>ovn-ic</code> process. The currently supported commands + are described below. + <dl> + <dt><code>exit</code></dt> + <dd> + Causes <code>ovn-ic</code> to gracefully terminate. + </dd> + + <dt><code>pause</code></dt> + <dd> + Pauses the ovn-ic operation from processing any Northbound and + Southbound database changes. + </dd> + + <dt><code>resume</code></dt> + <dd> + Resumes the ovn-ic operation to process Northbound and + Southbound database contents and generate logical flows. + </dd> + + <dt><code>is-paused</code></dt> + <dd> + Returns "true" if ovn-ic is currently paused, "false" otherwise. + </dd> + </dl> + </p> + + <h1>Active-Standby for High Availability</h1> + <p> + You may run <code>ovn-ic</code> more than once in an OVN deployment. + OVN will automatically ensure that only one of them is active at a time. + If multiple instances of <code>ovn-ic</code> are running and the + active <code>ovn-ic</code> fails, one of the hot standby instances + of <code>ovn-ic</code> will automatically take over. + </p> +</manpage> diff --git a/ic/ovn-ic.c b/ic/ovn-ic.c new file mode 100644 index 0000000..59f798d --- /dev/null +++ b/ic/ovn-ic.c @@ -0,0 +1,393 @@ +/* + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include <config.h> + +#include <getopt.h> +#include <stdlib.h> +#include <stdio.h> + +#include "bitmap.h" +#include "command-line.h" +#include "daemon.h" +#include "dirs.h" +#include "openvswitch/dynamic-string.h" +#include "fatal-signal.h" +#include "hash.h" +#include "openvswitch/hmap.h" +#include "openvswitch/json.h" +#include "ovn/lex.h" +#include "lib/ovn-inb-idl.h" +#include "lib/ovn-isb-idl.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "lib/ovn-util.h" +#include "packets.h" +#include "openvswitch/poll-loop.h" +#include "smap.h" +#include "sset.h" +#include "svec.h" +#include "stream.h" +#include "stream-ssl.h" +#include "unixctl.h" +#include "util.h" +#include "uuid.h" +#include "openvswitch/vlog.h" + +VLOG_DEFINE_THIS_MODULE(ovn_ic); + +static unixctl_cb_func ovn_ic_exit; +static unixctl_cb_func ovn_ic_pause; +static unixctl_cb_func ovn_ic_resume; +static unixctl_cb_func ovn_ic_is_paused; + +struct ic_context { + struct ovsdb_idl *ovnnb_idl; + struct ovsdb_idl *ovnsb_idl; + struct ovsdb_idl *ovninb_idl; + struct ovsdb_idl *ovnisb_idl; + struct ovsdb_idl_txn *ovnnb_txn; + struct ovsdb_idl_txn *ovnsb_txn; + struct ovsdb_idl_txn *ovninb_txn; + struct ovsdb_idl_txn *ovnisb_txn; +}; + +static const char *ovnnb_db; +static const char *ovnsb_db; +static const char *ovninb_db; +static const char *ovnisb_db; +static const char *unixctl_path; + + +static void +usage(void) +{ + printf("\ +%s: OVN interconnection management daemon\n\ +usage: %s [OPTIONS]\n\ +\n\ +Options:\n\ + --ovnnb-db=DATABASE connect to ovn-nb database at DATABASE\n\ + (default: %s)\n\ + --ovnsb-db=DATABASE connect to ovn-sb database at DATABASE\n\ + (default: %s)\n\ + --unixctl=SOCKET override default control socket name\n\ + -h, --help display this help message\n\ + -o, --options list available options\n\ + -V, --version display version information\n\ +", program_name, program_name, default_nb_db(), default_sb_db()); + daemon_usage(); + vlog_usage(); + stream_usage("database", true, true, false); +} + +static const struct isbrec_availability_zone * +az_run(struct ic_context *ctx) +{ + /* XXX: handle AZ deletion. */ + const struct nbrec_nb_global *nb_global = + nbrec_nb_global_first(ctx->ovnnb_idl); + + if (!nb_global) { + VLOG_INFO("NB Global not exist."); + return NULL; + } + + if (!nb_global->name[0]) { + return NULL; + } + + const char *az_name = nb_global->name; + const struct isbrec_availability_zone *az; + ISBREC_AVAILABILITY_ZONE_FOR_EACH (az, ctx->ovnisb_idl) { + if (!strcmp(az->name, az_name)) { + return az; + } + } + + /* Create AZ in ISB */ + if (ctx->ovnisb_txn) { + VLOG_INFO("Register AZ %s to interconnection DB.", az_name); + az = isbrec_availability_zone_insert(ctx->ovnisb_txn); + isbrec_availability_zone_set_name(az, az_name); + return az; + } + return NULL; +} + +static void +ovn_db_run(struct ic_context *ctx) +{ + const struct isbrec_availability_zone *az = az_run(ctx); + VLOG_DBG("Availability zone: %s", az ? az->name : "not created yet."); +} + +static void +parse_options(int argc OVS_UNUSED, char *argv[] OVS_UNUSED) +{ + enum { + DAEMON_OPTION_ENUMS, + VLOG_OPTION_ENUMS, + SSL_OPTION_ENUMS, + }; + static const struct option long_options[] = { + {"ovnsb-db", required_argument, NULL, 'd'}, + {"ovnnb-db", required_argument, NULL, 'D'}, + {"ovnisb-db", required_argument, NULL, 'i'}, + {"ovninb-db", required_argument, NULL, 'I'}, + {"unixctl", required_argument, NULL, 'u'}, + {"help", no_argument, NULL, 'h'}, + {"options", no_argument, NULL, 'o'}, + {"version", no_argument, NULL, 'V'}, + DAEMON_LONG_OPTIONS, + VLOG_LONG_OPTIONS, + STREAM_SSL_LONG_OPTIONS, + {NULL, 0, NULL, 0}, + }; + char *short_options = ovs_cmdl_long_options_to_short_options(long_options); + + for (;;) { + int c; + + c = getopt_long(argc, argv, short_options, long_options, NULL); + if (c == -1) { + break; + } + + switch (c) { + DAEMON_OPTION_HANDLERS; + VLOG_OPTION_HANDLERS; + STREAM_SSL_OPTION_HANDLERS; + + case 'd': + ovnsb_db = optarg; + break; + + case 'D': + ovnnb_db = optarg; + break; + + case 'u': + unixctl_path = optarg; + break; + + case 'h': + usage(); + exit(EXIT_SUCCESS); + + case 'o': + ovs_cmdl_print_options(long_options); + exit(EXIT_SUCCESS); + + case 'V': + ovs_print_version(0, 0); + exit(EXIT_SUCCESS); + + default: + break; + } + } + + if (!ovnsb_db) { + ovnsb_db = default_sb_db(); + } + + if (!ovnnb_db) { + ovnnb_db = default_nb_db(); + } + + if (!ovnisb_db) { + ovnisb_db = default_isb_db(); + } + + if (!ovninb_db) { + ovninb_db = default_inb_db(); + } + + free(short_options); +} + +static void OVS_UNUSED +add_column_noalert(struct ovsdb_idl *idl, + const struct ovsdb_idl_column *column) +{ + ovsdb_idl_add_column(idl, column); + ovsdb_idl_omit_alert(idl, column); +} + +int +main(int argc, char *argv[]) +{ + int res = EXIT_SUCCESS; + struct unixctl_server *unixctl; + int retval; + bool exiting; + bool paused; + + fatal_ignore_sigpipe(); + ovs_cmdl_proctitle_init(argc, argv); + set_program_name(argv[0]); + service_start(&argc, &argv); + parse_options(argc, argv); + + daemonize_start(false); + + retval = unixctl_server_create(unixctl_path, &unixctl); + if (retval) { + exit(EXIT_FAILURE); + } + unixctl_command_register("exit", "", 0, 0, ovn_ic_exit, &exiting); + unixctl_command_register("pause", "", 0, 0, ovn_ic_pause, &paused); + unixctl_command_register("resume", "", 0, 0, ovn_ic_resume, &paused); + unixctl_command_register("is-paused", "", 0, 0, ovn_ic_is_paused, + &paused); + + daemonize_complete(); + + /* ovn-inb db. */ + struct ovsdb_idl_loop ovninb_idl_loop = OVSDB_IDL_LOOP_INITIALIZER( + ovsdb_idl_create(ovninb_db, &inbrec_idl_class, true, true)); + + /* ovn-isb db. */ + struct ovsdb_idl_loop ovnisb_idl_loop = OVSDB_IDL_LOOP_INITIALIZER( + ovsdb_idl_create(ovnisb_db, &isbrec_idl_class, true, true)); + + /* ovn-nb db. XXX: add only needed tables and columns */ + struct ovsdb_idl_loop ovnnb_idl_loop = OVSDB_IDL_LOOP_INITIALIZER( + ovsdb_idl_create(ovnnb_db, &nbrec_idl_class, true, true)); + + /* ovn-sb db. XXX: add only needed tables and columns */ + struct ovsdb_idl_loop ovnsb_idl_loop = OVSDB_IDL_LOOP_INITIALIZER( + ovsdb_idl_create(ovnsb_db, &sbrec_idl_class, true, true)); + + /* Ensure that only a single ovn-ic is active in the deployment by + * acquiring a lock called "ovn_ic" on the southbound database + * and then only performing DB transactions if the lock is held. */ + ovsdb_idl_set_lock(ovnsb_idl_loop.idl, "ovn_ic"); + bool had_lock = false; + + /* Main loop. */ + exiting = false; + paused = false; + while (!exiting) { + if (!paused) { + struct ic_context ctx = { + .ovnnb_idl = ovnnb_idl_loop.idl, + .ovnnb_txn = ovsdb_idl_loop_run(&ovnnb_idl_loop), + .ovnsb_idl = ovnsb_idl_loop.idl, + .ovnsb_txn = ovsdb_idl_loop_run(&ovnsb_idl_loop), + .ovninb_idl = ovninb_idl_loop.idl, + .ovninb_txn = ovsdb_idl_loop_run(&ovninb_idl_loop), + .ovnisb_idl = ovnisb_idl_loop.idl, + .ovnisb_txn = ovsdb_idl_loop_run(&ovnisb_idl_loop), + }; + + if (!had_lock && ovsdb_idl_has_lock(ovnsb_idl_loop.idl)) { + VLOG_INFO("ovn-ic lock acquired. " + "This ovn-ic instance is now active."); + had_lock = true; + } else if (had_lock && !ovsdb_idl_has_lock(ovnsb_idl_loop.idl)) { + VLOG_INFO("ovn-ic lock lost. " + "This ovn-ic instance is now on standby."); + had_lock = false; + } + + if (ovsdb_idl_has_lock(ovnsb_idl_loop.idl)) { + ovn_db_run(&ctx); + } + + ovsdb_idl_loop_commit_and_wait(&ovnnb_idl_loop); + ovsdb_idl_loop_commit_and_wait(&ovnsb_idl_loop); + ovsdb_idl_loop_commit_and_wait(&ovninb_idl_loop); + ovsdb_idl_loop_commit_and_wait(&ovnisb_idl_loop); + } else { + /* ovn-ic is paused + * - we still want to handle any db updates and update the + * local IDL. Otherwise, when it is resumed, the local IDL + * copy will be out of sync. + * - but we don't want to create any txns. + * */ + ovsdb_idl_run(ovnnb_idl_loop.idl); + ovsdb_idl_run(ovnsb_idl_loop.idl); + ovsdb_idl_run(ovninb_idl_loop.idl); + ovsdb_idl_run(ovnisb_idl_loop.idl); + ovsdb_idl_wait(ovnnb_idl_loop.idl); + ovsdb_idl_wait(ovnsb_idl_loop.idl); + ovsdb_idl_wait(ovninb_idl_loop.idl); + ovsdb_idl_wait(ovnisb_idl_loop.idl); + } + + unixctl_server_run(unixctl); + unixctl_server_wait(unixctl); + if (exiting) { + poll_immediate_wake(); + } + + poll_block(); + if (should_service_stop()) { + exiting = true; + } + } + + unixctl_server_destroy(unixctl); + ovsdb_idl_loop_destroy(&ovnnb_idl_loop); + ovsdb_idl_loop_destroy(&ovnsb_idl_loop); + ovsdb_idl_loop_destroy(&ovninb_idl_loop); + ovsdb_idl_loop_destroy(&ovnisb_idl_loop); + service_stop(); + + exit(res); +} + +static void +ovn_ic_exit(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[] OVS_UNUSED, void *exiting_) +{ + bool *exiting = exiting_; + *exiting = true; + + unixctl_command_reply(conn, NULL); +} + +static void +ovn_ic_pause(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[] OVS_UNUSED, void *pause_) +{ + bool *pause = pause_; + *pause = true; + + unixctl_command_reply(conn, NULL); +} + +static void +ovn_ic_resume(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[] OVS_UNUSED, void *pause_) +{ + bool *pause = pause_; + *pause = false; + + unixctl_command_reply(conn, NULL); +} + +static void +ovn_ic_is_paused(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[] OVS_UNUSED, void *paused_) +{ + bool *paused = paused_; + if (*paused) { + unixctl_command_reply(conn, "true"); + } else { + unixctl_command_reply(conn, "false"); + } +} diff --git a/ovn-nb.ovsschema b/ovn-nb.ovsschema index 2c87cbb..1a9acdb 100644 --- a/ovn-nb.ovsschema +++ b/ovn-nb.ovsschema @@ -1,10 +1,11 @@ { "name": "OVN_Northbound", - "version": "5.16.0", - "cksum": "923459061 23095", + "version": "5.16.1", + "cksum": "2818115555 23139", "tables": { "NB_Global": { "columns": { + "name": {"type": "string"}, "nb_cfg": {"type": {"key": "integer"}}, "sb_cfg": {"type": {"key": "integer"}}, "hv_cfg": {"type": {"key": "integer"}}, diff --git a/ovn-nb.xml b/ovn-nb.xml index b41b579..07f60d3 100644 --- a/ovn-nb.xml +++ b/ovn-nb.xml @@ -36,6 +36,13 @@ one row. </p> + <group title="Identity"> + <column name="name"> + The name of the OVN cluster, which uniquely identifies the OVN cluster + throughout all OVN clusters supposed to interconnect with each other. + </column> + </group> + <group title="Status"> These columns allow a client to track the overall configuration state of the system. diff --git a/tutorial/ovs-sandbox b/tutorial/ovs-sandbox index a19dea2..35b9435 100755 --- a/tutorial/ovs-sandbox +++ b/tutorial/ovs-sandbox @@ -340,7 +340,7 @@ if $built; then exit 1 fi PATH=$ovsbuilddir/ovsdb:$ovsbuilddir/vswitchd:$ovsbuilddir/utilities:$ovsbuilddir/vtep:$PATH - PATH=$builddir/controller:$builddir/controller-vtep:$builddir/northd:$builddir/utilities:$PATH + PATH=$builddir/controller:$builddir/controller-vtep:$builddir/northd:$builddir/ic:$builddir/utilities:$PATH export PATH else case $schema in -- 2.1.0 _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
