On Tue, Feb 04, 2025 at 12:39:11AM +0100, Martin Kalcok wrote:
> This change builds on top of the new "dynamic routing" OVN feature
> that allows advertising routes to the fabric network. When LR option
> "dynamic-routing" is set on the router, following two new LRP options
> become available:
>
> * dynamic-routing-nat - When set to "true", ovn-controller will advertise
> routes for external NAT IPs valid for the LRP.
> * dynamic-routing-lb-vips - When set to "true", ovn-controller will advertise
> host routes to LB VIPs via the LRP.
>
> Co-authored-by: Frode Nordahl <[email protected]>
> Signed-off-by: Frode Nordahl <[email protected]>
> Signed-off-by: Martin Kalcok <[email protected]>
Hi Martin,
thanks for the patch, i'll just comment on what i found.
> ---
> NEWS | 17 ++
> northd/en-advertised-route-sync.c | 170 ++++++++++++++
> northd/en-northd.c | 5 +-
> northd/inc-proc-northd.c | 5 +
> northd/northd.c | 138 ++++++++++-
> northd/northd.h | 8 +-
> ovn-nb.xml | 32 +++
> ovs | 2 +-
> tests/ovn-northd.at | 139 +++++++++++
> tests/system-ovn.at | 379 ++++++++++++++++++++++++++++++
> 10 files changed, 890 insertions(+), 5 deletions(-)
>
> diff --git a/NEWS b/NEWS
> index eca17b728..137ff48fd 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -61,6 +61,23 @@ Post v24.09.0
> * Add the option "dynamic-routing-ifname" to LRPs. If set only routes
> learned from a linux iterfaces with that name are treated as relevant
> routes for this LRP.
> + - Add the option "dynamic-routing" to Logical Routers. If set to true all
> + static and connected routes attached to the router are shared to the
> + southbound "Route" table for sharing outside of OVN.
> + The routes can furthe be filtered by setting `dynamic-routing-connected`
> + and `dynamic-routing-static` on the LR or LRP. The LRP settings
> overwrite
> + the LR settings for all routes using this interface as an exit.
> + - Allow Logical Routers to dynamically learn routes from outside the
> fabric.
> + Routes entered into the "Route" table in the southbound database will be
> + learned by the respective LR. They are included in the route table with
> + a lower priority than static routes.
> + - Add the option "dynamic-routing-connected-as-host-routes" to LRPs. If
> set
> + to true then connected routes are announced as individual host routes.
> + - Add 'dynamic-routing-lb-vips' LRP option. If set to true, the LRP can be
> + used to advertise host paths to the Load Balancer VIPs associated with
> the
> + LR.
> + - Add 'dynamic-routing-nat' LRP option. If set to true, the LRP can be
> used
> + to advertise external NAT IPs associated with it.
>
> OVN v24.09.0 - 13 Sep 2024
> --------------------------
> diff --git a/northd/en-advertised-route-sync.c
> b/northd/en-advertised-route-sync.c
> index 02b3aba7a..8a1da02fd 100644
> --- a/northd/en-advertised-route-sync.c
> +++ b/northd/en-advertised-route-sync.c
> @@ -16,6 +16,7 @@
>
> #include <config.h>
>
> +#include "openvswitch/vlog.h"
> #include "smap.h"
> #include "stopwatch.h"
> #include "northd.h"
> @@ -26,6 +27,8 @@
> #include "openvswitch/hmap.h"
> #include "ovn-util.h"
>
> +VLOG_DEFINE_THIS_MODULE(en_advertised_route_sync);
> +
> static void
> advertised_route_table_sync(
> struct ovsdb_idl_txn *ovnsb_txn,
> @@ -312,6 +315,157 @@ publish_host_routes(struct hmap *sync_routes,
> }
> }
>
> +/* This function searches for an ovn_port with specific "op_ip"
> + * IP address in all LS datapaths directly connected to the
> + * LR datapath "od".
> + * If no match is found, this function returns NULL.*/
I see multiple points to optimize the function. Not sure if they are all
relevant or worth it, but i just want to share the options i see.
> +static struct ovn_port *
> +find_port_in_connected_ls(struct ovn_datapath *od, char *op_ip)
> +{
AFAIK there can not be a port found if op_ip is not a host ip (so /32 or
/128). So this could directly return NULL. This would directly cover
e.g. large SNAT pools.
> + for (int i = 0; i < od->n_ls_peers; i++) {
> + struct ovn_datapath *peer_od = od->ls_peers[i];
> + struct ovn_port *op;
We would only need to search on the op if the network of the op matches
to the op_ip. So there is no need to search missmatched subnets. If such
a validation is efficient, i don't know.
> + HMAP_FOR_EACH (op, dp_node, &peer_od->ports) {
> + for (int j = 0; j < op->n_lsp_addrs; j++) {
> + struct lport_addresses *addrs = &op->lsp_addrs[j];
We could check if op_ip is a v4 or v6 address so we only need one loop
based on that.
> + for (int k = 0; k < addrs->n_ipv4_addrs; k++) {
> + if (!strcmp(op_ip, addrs->ipv4_addrs[k].addr_s)) {
> + return op;
> + }
> + }
> + for (int k = 0; k < addrs->n_ipv6_addrs; k++) {
> + if (!strcmp(op_ip, addrs->ipv6_addrs[k].addr_s)) {
> + return op;
> + }
> + }
> + }
> + }
> + }
> + return NULL;
> +}
> +
> +/* Publish parsed_route as a Advertised_Route record. The main purpose
> + * of this function is to find LSP with IP address that matches the
> + * logical_ip of a NAT rule on the route's out_port datapath. This port is
> + * then set as a tracked_port for the route.
> + * Search is performed in all LS datapaths directly connected to the
> + * route's out_port datapath.
> + * If no suitable tracked_port is found, route is still published, but
> + * without the tracked_port set.*/
> +static void
> +publish_nat_route(struct hmap *route_map,
> + const struct parsed_route *route)
> +{
> + const struct ovn_datapath *advertised_dp = route->od;
> + const struct ovn_port *ext_nat_port = route->out_port;
> +
> + if (!advertised_dp->nbr || !ext_nat_port->od) {
> + return;
> + }
> +
> + char *ip_prefix = normalize_v46_prefix(&route->prefix,
> + route->plen);
> + char *logical_ip = NULL;
> +
> + /* Find NAT rule with external IP that matches the route's
> + * ip_prefix.*/
> + for (int i = 0; i < ext_nat_port->od->nbr->n_nat; i++) {
> + struct nbrec_nat *nat = ext_nat_port->od->nbr->nat[i];
> + if (!strcmp(nat->external_ip, ip_prefix)) {
> + logical_ip = nat->logical_ip;
> + break;
> + }
> + }
> +
> + if (!logical_ip) {
> + return;
> + }
> +
> + const struct sbrec_port_binding *tracked_pb = NULL;
> + struct ovn_port *tracked_port =
> find_port_in_connected_ls(ext_nat_port->od,
> + logical_ip);
> + if (tracked_port) {
> + tracked_pb = tracked_port->sb;
> + }
> +
> + char *ip_with_mask = xasprintf("%s/%d", ip_prefix, route->plen);
> + ar_alloc_entry(route_map,
> + route->od->sb,
> + route->out_port->sb,
> + ip_with_mask,
> + tracked_pb);
> + free(ip_prefix);
> +}
> +
> +/* Publish parsed_route as a Advertised_Route record. The main purpose
> + * of this function is to find LSP with IP address that matches the
> + * endpoint IP of a LB on the route's out_port datapath. This port is then
> + * set as a tracked_port for the route.
> + * Search is performed in all LS datapaths directly connected to the
> + * route's out_port datapath.
> + * If no suitable tracked_port is found, route is still published, but
> + * without the tracked_port set.*/
> +static void
> +publish_lb_route(struct hmap *route_map,
> + const struct parsed_route *route)
> +{
> + const struct ovn_datapath *advertised_dp = route->od;
> + const struct ovn_port *ext_lb_port = route->out_port;
> +
> + if (!advertised_dp->nbr || !ext_lb_port->od) {
> + return;
> + }
> +
> + char *ip_prefix = normalize_v46_prefix(&route->prefix,
> + route->plen);
> +
> + for (int i = 0; i < ext_lb_port->od->nbr->n_load_balancer; i++) {
> + struct nbrec_load_balancer *lb =
> + ext_lb_port->od->nbr->load_balancer[i];
> + struct smap lb_vips = SMAP_INITIALIZER(&lb_vips);
> + struct smap_node *node;
> + SMAP_FOR_EACH (node, &lb->vips) {
> + struct ovn_lb_vip *lb_vip = xmalloc(sizeof(*lb_vip));
> + char *error = ovn_lb_vip_init(lb_vip, node->key,
> + node->value, false, AF_INET6);
> + if (error) {
> + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5,
> 1);
> + VLOG_WARN_RL(&rl, "Failed to parse LB VIP: %s", error);
> + ovn_lb_vip_destroy(lb_vip);
> + free(error);
> + continue;
> + }
> +
> + /* Continue only for LBs whose VIP matches route's ip_prefix.*/
> + if (strcmp(lb_vip->vip_str, ip_prefix)) {
> + ovn_lb_vip_destroy(lb_vip);
> + continue;
> + }
> +
> + /* Find LSP that matches at least one endpoint IP of the LB.*/
> + const struct sbrec_port_binding *tracked_pb = NULL;
> + for (int j = 0; j < lb_vip->n_backends; j++) {
> + char *backend = lb_vip->backends[j].ip_str;
> + struct ovn_port *tracked_port =
> + find_port_in_connected_ls(ext_lb_port->od, backend);
> + if (tracked_port) {
> + tracked_pb = tracked_port->sb;
> + break;
> + }
> + }
> +
> + char *ip_with_mask = xasprintf("%s/%d", ip_prefix, route->plen);
> + ar_alloc_entry(route_map,
> + route->od->sb,
> + route->out_port->sb,
> + ip_with_mask,
> + tracked_pb);
> + ovn_lb_vip_destroy(lb_vip);
> + free(ip_prefix);
> + }
> + }
> +}
> +
> static void
> advertised_route_table_sync(
> struct ovsdb_idl_txn *ovnsb_txn,
> @@ -359,6 +513,22 @@ advertised_route_table_sync(
> !route->out_port->dynamic_routing_static) {
> continue;
> }
> + if (route->source == ROUTE_SOURCE_NAT) {
> + if (!smap_get_bool(&route->out_port->nbrp->options,
> + "dynamic-routing-nat", false)) {
> + continue;
> + }
> + publish_nat_route(&sync_routes, route);
> + continue;
> + }
> + if (route->source == ROUTE_SOURCE_LB) {
> + if (!smap_get_bool(&route->out_port->nbrp->options,
> + "dynamic-routing-lb-vips", false)) {
> + continue;
> + }
> + publish_lb_route(&sync_routes, route);
> + continue;
> + }
>
> char *ip_prefix = normalize_v46_prefix(&route->prefix,
> route->plen);
> diff --git a/northd/en-northd.c b/northd/en-northd.c
> index c7d1ebcb3..60e295628 100644
> --- a/northd/en-northd.c
> +++ b/northd/en-northd.c
> @@ -316,6 +316,8 @@ en_routes_run(struct engine_node *node, void *data)
> {
> struct northd_data *northd_data = engine_get_input_data("northd", node);
> struct bfd_data *bfd_data = engine_get_input_data("bfd", node);
> + struct ed_type_lr_stateful *lr_stateful_data =
> + engine_get_input_data("lr_stateful", node);
> struct routes_data *routes_data = data;
>
> routes_destroy(data);
> @@ -330,7 +332,8 @@ en_routes_run(struct engine_node *node, void *data)
> route_table_name);
> }
>
> - build_parsed_routes(od, &northd_data->lr_ports,
> + build_parsed_routes(od, &lr_stateful_data->table,
> + &northd_data->lr_ports,
> &bfd_data->bfd_connections,
> &routes_data->parsed_routes,
> &routes_data->route_tables,
> diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c
> index 86b7b999e..d74c2e3dc 100644
> --- a/northd/inc-proc-northd.c
> +++ b/northd/inc-proc-northd.c
> @@ -267,6 +267,11 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb,
> engine_add_input(&en_routes, &en_bfd, NULL);
> engine_add_input(&en_routes, &en_northd,
> routes_northd_change_handler);
> + engine_add_input(&en_routes, &en_lr_nat,
> + NULL);
> + engine_add_input(&en_routes, &en_lb_data,
> + NULL);
> + engine_add_input(&en_routes, &en_lr_stateful, engine_noop_handler);
>
> engine_add_input(&en_bfd_sync, &en_bfd, NULL);
> engine_add_input(&en_bfd_sync, &en_nb_bfd, NULL);
> diff --git a/northd/northd.c b/northd/northd.c
> index 71adb88d7..a41ad7642 100644
> --- a/northd/northd.c
> +++ b/northd/northd.c
> @@ -11075,8 +11075,111 @@ parsed_routes_add_connected(const struct
> ovn_datapath *od,
> }
> }
>
> +/* Add "parsed_route"s to the "routes" for each IP address in laddrs.*/
> +static void
> +lport_addrs_to_parsed_routes(const struct ovn_datapath *od,
> + const struct ovn_port *op,
> + const struct lport_addresses *laddrs,
> + enum route_source route_type,
> + struct hmap *routes,
> + bool install_lflow)
> +{
> + for (int i = 0; i < laddrs->n_ipv4_addrs; i++) {
> + struct ipv4_netaddr *addr = &laddrs->ipv4_addrs[i];
> + struct in6_addr prefix;
> + ip46_parse(addr->network_s, &prefix);
> + parsed_route_add(od, NULL, &prefix, addr->plen,
> + false, addr->addr_s, op,
> + 0, false,
> + false, NULL, route_type,
> + &op->nbrp->header_, routes, install_lflow);
> + }
> + for (int i = 0; i < laddrs->n_ipv6_addrs; i++) {
> + struct ipv6_netaddr *addr = &laddrs->ipv6_addrs[i];
> + parsed_route_add(od, NULL, &addr->addr, addr->plen,
> + false, addr->addr_s, op,
> + 0, false,
> + false, NULL, route_type,
> + &op->nbrp->header_, routes, install_lflow);
> + }
> +}
> +
> +/* Add routes of all NAT external_addresses on the ovn_port "op", to be
> + * advertised by the datapath "od".
> + * If "routeable_only" is true, only NATs with option "add_route=true" will
> + * be considered.
> + * Note that the "op" does not necessarily need to be in the "od" datapath.
> + * Such use-case is for advertising external NAT IPs of LRPs in neighboring
> + * routers and should be paired with routable_only=true.
> + */
> +static void
> +parsed_routes_add_nat(const struct ovn_datapath *od,
> + const struct ovn_port *op,
> + struct hmap *routes,
> + bool routable_only)
> +{
> + if (!op->nbrp || !smap_get_bool(&op->nbrp->options,
> + "dynamic-routing-nat", false)) {
> + return;
> + }
> + size_t n_nats = 0;
> + char **nats = NULL;
> + nats = get_nat_addresses(op, &n_nats, routable_only, false, NULL, true);
> +
> + for (size_t i = 0; i < n_nats; i++) {
> + struct lport_addresses *laddrs = xzalloc(sizeof *laddrs);
> + int ofs = 0;
> + extract_addresses(nats[i], laddrs, &ofs);
> + lport_addrs_to_parsed_routes(od, op, laddrs, ROUTE_SOURCE_NAT,
> + routes, false);
> + destroy_lport_addresses(laddrs);
> + free(nats[i]);
> + }
> + free(nats);
> +}
> +
> +/* Add routes of all LB VIPs on the ovn_port "op", to be advertised by the
> + * datapath "od".
> + * If "routeable_only" is true, only LBs with option "add_route=true" will
> + * be considered.
> + * Note that the "op" does not necessarily need to be in the "od" datapath.
> + * Such use-case is for advertising LB VIPs of LRPs in neighboring routers
> + * and should be paired with routable_only=true.
> + */
> +static void
> +parsed_routes_add_lb(const struct ovn_datapath *od,
> + const struct ovn_port *op,
> + struct hmap *routes,
> + const struct lr_stateful_record *lr_stateful_rec,
> + bool routable_only)
> +{
> + if (!op->nbrp || !smap_get_bool(&op->nbrp->options,
> + "dynamic-routing-lb-vips", false)) {
> + return;
> + }
> +
> + if (!lr_stateful_rec) {
> + return;
> + }
> +
> + struct ds addresses = DS_EMPTY_INITIALIZER;
> + get_lb_addresses(lr_stateful_rec, &addresses, routable_only);
> +
> + struct lport_addresses *laddrs = xzalloc(sizeof *laddrs);
> + extract_ip_addresses(ds_cstr(&addresses), laddrs);
> +
> + lport_addrs_to_parsed_routes(od, op, laddrs, ROUTE_SOURCE_LB,
> + routes, false);
> +
> + destroy_lport_addresses(laddrs);
> + ds_destroy(&addresses);
> +
> +}
> +
> void
> -build_parsed_routes(const struct ovn_datapath *od, const struct hmap
> *lr_ports,
> +build_parsed_routes(const struct ovn_datapath *od,
> + const struct lr_stateful_table *lr_stateful_table,
> + const struct hmap *lr_ports,
> const struct hmap *bfd_connections, struct hmap *routes,
> struct simap *route_tables,
> struct hmap *bfd_active_connections)
> @@ -11096,7 +11199,34 @@ build_parsed_routes(const struct ovn_datapath *od,
> const struct hmap *lr_ports,
>
> const struct ovn_port *op;
> HMAP_FOR_EACH (op, dp_node, &od->ports) {
> + const struct lr_stateful_record *lr_stateful_rec = NULL;
> + lr_stateful_rec = lr_stateful_table_find_by_index(
> + lr_stateful_table, op->od->index);
> +
> parsed_routes_add_connected(od, op, routes);
> + parsed_routes_add_nat(od, op, routes, false);
> + parsed_routes_add_lb(od, op, routes, lr_stateful_rec, false);
> + }
> +
> + /* For GW routers we traverse also neighboring LR in search for
> + * LBs and NATs with "add_route" option set to "true". Such NATs/LBs
> + * install logical flows for their external IP addresses to their
> + * neighboring routers, and as such, we can advertise them as routes.
> + * The LRP on the neighboring router needs to have appropriate
> + * "dynamic-routing-{nat,lb-vips}" option set for this to take effect.*/
> + for (size_t i = 0; od->is_gw_router && i < od->n_ls_peers; i++) {
> + for (size_t j = 0; j < od->ls_peers[i]->n_router_ports; j++) {
> + struct ovn_port *router_port;
> + router_port = od->ls_peers[i]->router_ports[j]->peer;
> +
> + const struct lr_stateful_record *lr_stateful_rec = NULL;
> + lr_stateful_rec = lr_stateful_table_find_by_index(
> + lr_stateful_table, router_port->od->index);
> +
> + parsed_routes_add_nat(od, router_port, routes, true);
> + parsed_routes_add_lb(od, router_port, routes,
> + lr_stateful_rec, true);
> + }
I am not sure if I understood this correctly.
But this seems for me to similar to what publish_host_routes in
https://patchwork.ozlabs.org/project/ovn/patch/8b1bbafc0ebaaf1e3182a182161894203af5b8d7.1738676766.git.felix.huettner@stackit.cloud/
implements.
There we also iterate through all LSPs of one LRP. And if one of these
LSPs is a LR we collect addresses from them. This just covers NAT and no
LB, but maybe we can combine the logic?
Thanks a lot,
Felix
> }
>
> HMAP_FOR_EACH_SAFE (pr, key_node, routes) {
> @@ -11278,6 +11408,8 @@ route_source_to_offset(enum route_source source)
> {
> switch (source) {
> case ROUTE_SOURCE_CONNECTED:
> + case ROUTE_SOURCE_NAT:
> + case ROUTE_SOURCE_LB:
> return ROUTE_PRIO_OFFSET_CONNECTED;
> case ROUTE_SOURCE_STATIC:
> return ROUTE_PRIO_OFFSET_STATIC;
> @@ -13560,7 +13692,9 @@ build_route_flows_for_lrouter(
> struct parsed_route *route;
> HMAP_FOR_EACH_WITH_HASH (route, key_node, uuid_hash(&od->key),
> parsed_routes) {
> - if (route->source == ROUTE_SOURCE_CONNECTED) {
> + if (route->source == ROUTE_SOURCE_CONNECTED ||
> + route->source == ROUTE_SOURCE_NAT ||
> + route->source == ROUTE_SOURCE_LB) {
> unique_routes_add(&unique_routes, route);
> continue;
> }
> diff --git a/northd/northd.h b/northd/northd.h
> index fffaab3a0..d55909f76 100644
> --- a/northd/northd.h
> +++ b/northd/northd.h
> @@ -708,6 +708,10 @@ enum route_source {
> ROUTE_SOURCE_CONNECTED,
> /* The route is derived from a northbound static route entry. */
> ROUTE_SOURCE_STATIC,
> + /* Host route generated from NAT's external IP. */
> + ROUTE_SOURCE_NAT,
> + /* Host route generated from LB's external IP. */
> + ROUTE_SOURCE_LB,
> /* The route is dynamically learned by an ovn-controller. */
> ROUTE_SOURCE_LEARNED,
> };
> @@ -780,7 +784,9 @@ void northd_indices_create(struct northd_data *data,
>
> void route_policies_init(struct route_policies_data *);
> void route_policies_destroy(struct route_policies_data *);
> -void build_parsed_routes(const struct ovn_datapath *, const struct hmap *,
> +void build_parsed_routes(const struct ovn_datapath *,
> + const struct lr_stateful_table *lr_stateful_table,
> + const struct hmap *,
> const struct hmap *, struct hmap *, struct simap *,
> struct hmap *);
> uint32_t get_route_table_id(struct simap *, const char *);
> diff --git a/ovn-nb.xml b/ovn-nb.xml
> index 734fcf2bd..e8d9d5491 100644
> --- a/ovn-nb.xml
> +++ b/ovn-nb.xml
> @@ -2971,6 +2971,10 @@ or
> table="Logical_Router_Port"/></li>
> <li><ref column="options" key="dynamic-routing-static"
> table="Logical_Router_Port"/></li>
> + <li><ref column="options" key="dynamic-routing-lb-vips"
> + table="Logical_Router_Port"/></li>
> + <li><ref column="options" key="dynamic-routing-nat"
> + table="Logical_Router_Port"/></li>
> </ul>
> </column>
>
> @@ -3828,6 +3832,34 @@ or
> This option allows to have a 1:1 mapping between a single LRP and an
> individual link.
> </column>
> +
> + <column name="options" key="dynamic-routing-lb-vips"
> + type='{"type": "boolean"}'>
> + <p>
> + Only relevant if <ref column="options" key="dynamic-routing"
> + table="Logical_Router"/> on the respective Logical_Router is set
> + to <code>true</code>.
> +
> + If this option is <code>true</code>, northd will create host route
> + entries in the southbound <ref table="Advertised_Route"
> + db="OVN_Southbound"/> table, associated with this LRP, for each LB
> + VIP.
> + </p>
> + </column>
> +
> + <column name="options" key="dynamic-routing-nat"
> + type='{"type": "boolean"}'>
> + <p>
> + Only relevant if <ref column="options" key="dynamic-routing"
> + table="Logical_Router"/> on the respective Logical_Router is set
> + to <code>true</code>.
> +
> + If this option is <code>true</code>, northd will create host route
> + entries in the southbound <ref table="Advertised_Route"
> + db="OVN_Southbound"/> table, for external IP addresses of NAT rules
> + associated with this LRP.
> + </p>
> + </column>
> </group>
>
> <group title="Attachment">
> diff --git a/ovs b/ovs
> index a3c06c309..c648ee626 160000
> --- a/ovs
> +++ b/ovs
> @@ -1 +1 @@
> -Subproject commit a3c06c309ab9f21fd9ce4e8dca542119d46be9c5
> +Subproject commit c648ee626efdc959ed0cff37c2b9128816450f15
> diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
> index 44dc92869..9baa0ea15 100644
> --- a/tests/ovn-northd.at
> +++ b/tests/ovn-northd.at
> @@ -15134,3 +15134,142 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE
> AT_CLEANUP
> ])
>
> +OVN_FOR_EACH_NORTHD_NO_HV([
> +AT_SETUP([dynamic-routing - nat sync to sb])
> +AT_KEYWORDS([dynamic-routing])
> +ovn_start
> +
> +# Start with GW router and a single LRP
> +check ovn-nbctl lr-add lr0
> +check ovn-nbctl \
> + -- \
> + set Logical_Router lr0 options:dynamic-routing=true \
> + options:chassis=hv1
> +check ovn-nbctl --wait=sb \
> + -- \
> + lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
> +
> +check_row_count Advertised_Route 0
> +
> +datapath=$(ovn-sbctl --bare --columns _uuid list datapath_binding lr0)
> +pb=$(ovn-sbctl --bare --columns _uuid list port_binding lr0-sw0)
> +
> +# Adding LRP dynamic-routing-nat option and NAT rule advertises a route entry
> +check ovn-nbctl --wait=sb \
> + -- \
> + lrp-set-options lr0-sw0 dynamic-routing-nat=true \
> + -- \
> + lr-nat-add lr0 dnat_and_snat 172.16.1.10 192.168.1.10
> +
> +ovn-nbctl list NAT
> +ovn-sbctl list Advertised_Route
> +ovn-sbctl lflow-list
> +
> +check_row_count Advertised_Route 1
> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
> +datapath=$datapath logical_port=$pb], [0], [dnl
> +172.16.1.10/32
> +])
> +
> +# Add LR with distributed LRP connected to GW router through join LS
> +check ovn-nbctl \
> + -- \
> + lrp-add lr0 lr0-join 00:00:00:00:ff:02 10.42.0.1/24 \
> + -- \
> + ls-add ls-join \
> + -- \
> + lsp-add ls-join lsp-join-to-lr0 \
> + -- \
> + lsp-set-type lsp-join-to-lr0 router \
> + -- \
> + lsp-set-options lsp-join-to-lr0 router-port=lr0-join \
> + -- \
> + lsp-set-addresses lsp-join-to-lr0 router \
> + -- \
> + lr-add lr-guest0 \
> + -- \
> + lrp-add lr-guest0 lrp-guest0-sw0 00:00:00:00:fe:01 10.51.0.1/24 \
> + -- \
> + lrp-add lr-guest0 lrp-guest0-join 00:00:00:00:fe:02 10.42.0.2/24 \
> + -- \
> + lrp-set-options lrp-guest0-join dynamic-routing-nat=true \
> + -- \
> + lsp-add ls-join lsp-join-to-guest0 \
> + -- \
> + lsp-set-type lsp-join-to-guest0 router \
> + -- \
> + lsp-set-options lsp-join-to-guest0 router-port=lrp-guest0-join \
> + -- \
> + lrp-set-gateway-chassis lrp-guest0-join hv1
> +
> +pb2=$(ovn-sbctl --bare --columns _uuid list port_binding lrp-guest0-join)
> +
> +check ovn-nbctl --wait=sb \
> + --add-route lr-nat-add lr-guest0 dnat_and_snat 172.16.2.10 192.168.2.10
> +
> +check_row_count Advertised_Route 2
> +ovn-sbctl list advertised_route
> +echo $datapath
> +echo $pb
> +echo $pb2
> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
> +datapath=$datapath logical_port="$pb"], [0], [dnl
> +172.16.1.10/32
> +])
> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
> +datapath=$datapath logical_port=$pb2], [0], [dnl
> +172.16.2.10/32
> +])
> +
> +# Add nonlocal LR with distributed LRP connected to GW router through join LS
> +check ovn-nbctl \
> + -- \
> + lr-add lr-guest1 \
> + -- \
> + lrp-add lr-guest1 lrp-guest1-sw0 00:00:00:00:fd:01 10.51.1.1/24 \
> + -- \
> + lrp-add lr-guest1 lrp-guest1-join 00:00:00:00:fd:02 10.42.0.3/24 \
> + -- \
> + lrp-set-options lrp-guest1-join dynamic-routing-nat=true \
> + -- \
> + lsp-add ls-join lsp-join-to-guest1 \
> + -- \
> + lsp-set-type lsp-join-to-guest1 router \
> + -- \
> + lsp-set-options lsp-join-to-guest1 router-port=lrp-guest1-join \
> + -- \
> + lrp-set-gateway-chassis lrp-guest1-join nonlocalhv
> +
> +pb3=$(ovn-sbctl --bare --columns _uuid list port_binding lrp-guest1-join)
> +
> +check ovn-nbctl --wait=sb \
> + --add-route lr-nat-add lr-guest1 dnat_and_snat 172.16.3.10 192.168.3.10
> +check_row_count Advertised_Route 3
> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
> +datapath=$datapath logical_port=$pb], [0], [dnl
> +172.16.1.10/32
> +])
> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
> +datapath=$datapath logical_port=$pb2], [0], [dnl
> +172.16.2.10/32
> +])
> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
> +datapath=$datapath logical_port=$pb3], [0], [dnl
> +172.16.3.10/32
> +])
> +
> +# removing the option:dynamic-routing removes all routes
> +check ovn-nbctl --wait=sb remove Logical_Router lr0 option dynamic-routing
> +check_row_count Advertised_Route 0
> +
> +# and setting it again adds them again
> +check ovn-nbctl --wait=sb set Logical_Router lr0 option:dynamic-routing=true
> +check_row_count Advertised_Route 3
> +
> +# removing the lr will remove all routes
> +check ovn-nbctl --wait=sb lr-del lr0
> +check_row_count Advertised_Route 0
> +
> +AT_CLEANUP
> +])
> +
> diff --git a/tests/system-ovn.at b/tests/system-ovn.at
> index 5c4364663..918266ad2 100644
> --- a/tests/system-ovn.at
> +++ b/tests/system-ovn.at
> @@ -16033,3 +16033,382 @@ OVS_TRAFFIC_VSWITCHD_STOP(["/.*error receiving.*/d
> AT_CLEANUP
> ])
>
> +OVN_FOR_EACH_NORTHD([
> +AT_SETUP([route-exchange for LB VIPs with gateway router IPv4])
> +AT_KEYWORDS([route-exchange])
> +
> +CHECK_VRF()
> +CHECK_CONNTRACK()
> +CHECK_CONNTRACK_NAT()
> +ovn_start
> +OVS_TRAFFIC_VSWITCHD_START()
> +ADD_BR([br-int])
> +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone])
> +
> +# Set external-ids in br-int needed for ovn-controller
> +ovs-vsctl \
> + -- set Open_vSwitch . external-ids:system-id=hv1 \
> + -- set Open_vSwitch .
> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
> + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
> + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \
> + -- set bridge br-int fail-mode=secure
> other-config:disable-in-band=true
> +
> +# Start ovn-controller
> +start_daemon ovn-controller
> +
> +ovn-appctl vlog/set route_exchange
> +check ovn-nbctl -- lr-add R1 \
> + -- set Logical_Router R1 options:requested-tnl-key=1000
> options:dynamic-routing=true
> +
> +check ovn-nbctl ls-add sw0
> +check ovn-nbctl ls-add public
> +
> +check ovn-nbctl --wait=hv sync
> +
> +AT_CHECK([ip link | grep -q ovnvrf1000:.*UP], [1])
> +
> +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 192.168.1.1/24
> +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03 172.16.1.1/24 \
> + -- lrp-set-options rp-public \
> + dynamic-routing-maintain-vrf=true \
> + dynamic-routing-lb-vips=true
> +
> +check ovn-nbctl set logical_router R1 options:chassis=hv1
> +
> +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-rp \
> + type=router options:router-port=rp-sw0 \
> + -- lsp-set-addresses sw0-rp router
> +
> +check ovn-nbctl lsp-add public public-rp -- set Logical_Switch_Port
> public-rp \
> + type=router options:router-port=rp-public \
> + -- lsp-set-addresses public-rp router
> +
> +check ovs-vsctl set Open_vSwitch .
> external-ids:ovn-bridge-mappings=phynet:br-ext
> +
> +check ovn-nbctl lsp-add public public1 \
> + -- lsp-set-addresses public1 unknown \
> + -- lsp-set-type public1 localnet \
> + -- lsp-set-options public1 network_name=phynet
> +
> +check ovn-nbctl --wait=hv sync
> +
> +AT_CHECK([test `ip route show table 1000 | wc -l` -eq 1], [1])
> +
> +
> +# Create a load balancer and associate to R1
> +check ovn-nbctl lb-add lb1 172.16.1.150:80 172.16.1.100:80
> +check ovn-nbctl lr-lb-add R1 lb1
> +
> +check ovn-nbctl --wait=hv sync
> +
> +AT_CHECK([ip link | grep -q ovnvrf1000:.*UP])
> +AT_CHECK([test `ip route show table 1000 | wc -l` -eq 1])
> +AT_CHECK([ip route show table 1000 | grep -q 172.16.1.150])
> +
> +OVS_APP_EXIT_AND_WAIT([ovn-controller])
> +
> +# Ensure system resources are cleaned up
> +AT_CHECK([ip link | grep -q ovnvrf1000:.*UP], [1])
> +AT_CHECK([test `ip route show table 1000 | wc -l` -eq 1], [1])
> +
> +as ovn-sb
> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> +
> +as ovn-nb
> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> +
> +as northd
> +OVS_APP_EXIT_AND_WAIT([ovn-northd])
> +
> +as
> +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d
> +/Failed to acquire.*/d
> +/connection dropped.*/d"])
> +AT_CLEANUP
> +])
> +
> +OVN_FOR_EACH_NORTHD([
> +AT_SETUP([route-exchange for LB VIPs with gateway router IPv6])
> +AT_KEYWORDS([route-exchange])
> +
> +CHECK_VRF()
> +CHECK_CONNTRACK()
> +CHECK_CONNTRACK_NAT()
> +ovn_start
> +OVS_TRAFFIC_VSWITCHD_START()
> +ADD_BR([br-int])
> +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone])
> +
> +# Set external-ids in br-int needed for ovn-controller
> +ovs-vsctl \
> + -- set Open_vSwitch . external-ids:system-id=hv1 \
> + -- set Open_vSwitch .
> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
> + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
> + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \
> + -- set bridge br-int fail-mode=secure
> other-config:disable-in-band=true
> +
> +# Start ovn-controller
> +start_daemon ovn-controller
> +
> +ovn-appctl vlog/set route_exchange
> +check ovn-nbctl -- lr-add R1 \
> + -- set Logical_Router R1 options:requested-tnl-key=1001
> options:dynamic-routing=true
> +
> +check ovn-nbctl ls-add sw0
> +check ovn-nbctl ls-add public
> +
> +check ovn-nbctl --wait=hv sync
> +
> +AT_CHECK([ip link | grep -q ovnvrf1001:.*UP], [1])
> +
> +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 2001:db8:100::1/64
> +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03
> 2001:db8:1001::1/64 \
> + -- lrp-set-options rp-public \
> + dynamic-routing-maintain-vrf=true \
> + dynamic-routing-lb-vips=true
> +
> +check ovn-nbctl set logical_router R1 options:chassis=hv1
> +
> +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-rp \
> + type=router options:router-port=rp-sw0 \
> + -- lsp-set-addresses sw0-rp router
> +
> +check ovn-nbctl lsp-add public public-rp -- set Logical_Switch_Port
> public-rp \
> + type=router options:router-port=rp-public \
> + -- lsp-set-addresses public-rp router
> +
> +check ovs-vsctl set Open_vSwitch .
> external-ids:ovn-bridge-mappings=phynet:br-ext
> +
> +check ovn-nbctl lsp-add public public1 \
> + -- lsp-set-addresses public1 unknown \
> + -- lsp-set-type public1 localnet \
> + -- lsp-set-options public1 network_name=phynet
> +
> +check ovn-nbctl --wait=hv sync
> +
> +AT_CHECK([test `ip -6 route show table 1001 | wc -l` -eq 1], [1])
> +
> +# Create a load balancer and associate to R1
> +check ovn-nbctl lb-add lb1 [[2001:db8:1001::150]]:80
> [[2001:db8:1001::100]]:80
> +check ovn-nbctl lr-lb-add R1 lb1
> +
> +check ovn-nbctl --wait=hv sync
> +
> +AT_CHECK([ip link | grep -q ovnvrf1001:.*UP])
> +AT_CHECK([test `ip -6 route show table 1001 | wc -l` -eq 1])
> +AT_CHECK([ip -6 route show table 1001 | grep -q 2001:db8:1001::150])
> +
> +
> +OVS_APP_EXIT_AND_WAIT([ovn-controller])
> +
> +# Ensure system resources are cleaned up
> +AT_CHECK([ip link | grep -q ovnvrf1001:.*UP], [1])
> +AT_CHECK([test `ip -6 route show table 1001 | wc -l` -eq 1], [1])
> +
> +as ovn-sb
> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> +
> +as ovn-nb
> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> +
> +as northd
> +OVS_APP_EXIT_AND_WAIT([ovn-northd])
> +
> +as
> +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d
> +/Failed to acquire.*/d
> +/connection dropped.*/d"])
> +AT_CLEANUP
> +])
> +
> +OVN_FOR_EACH_NORTHD([
> +AT_SETUP([route-exchange for DNAT and DNAT_AND_SNAT with gateway router
> IPv4])
> +AT_KEYWORDS([route-exchange])
> +
> +CHECK_VRF()
> +CHECK_CONNTRACK()
> +CHECK_CONNTRACK_NAT()
> +ovn_start
> +OVS_TRAFFIC_VSWITCHD_START()
> +ADD_BR([br-int])
> +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone])
> +
> +# Set external-ids in br-int needed for ovn-controller
> +ovs-vsctl \
> + -- set Open_vSwitch . external-ids:system-id=hv1 \
> + -- set Open_vSwitch .
> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
> + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
> + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \
> + -- set bridge br-int fail-mode=secure
> other-config:disable-in-band=true
> +
> +# Start ovn-controller
> +start_daemon ovn-controller
> +
> +ovn-appctl vlog/set route_exchange
> +check ovn-nbctl -- lr-add R1 \
> + -- set Logical_Router R1 options:requested-tnl-key=1002
> options:dynamic-routing=true
> +
> +check ovn-nbctl ls-add sw0
> +check ovn-nbctl ls-add public
> +
> +check ovn-nbctl --wait=hv sync
> +
> +AT_CHECK([ip link | grep -q ovnvrf1002:.*UP], [1])
> +
> +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 192.168.1.1/24
> +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03 172.16.1.1/24 \
> + -- lrp-set-options rp-public \
> + dynamic-routing-maintain-vrf=true \
> + dynamic-routing-nat=true
> +
> +check ovn-nbctl set logical_router R1 options:chassis=hv1
> +
> +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-rp \
> + type=router options:router-port=rp-sw0 \
> + -- lsp-set-addresses sw0-rp router
> +
> +check ovn-nbctl lsp-add public public-rp -- set Logical_Switch_Port
> public-rp \
> + type=router options:router-port=rp-public \
> + -- lsp-set-addresses public-rp router
> +
> +check ovs-vsctl set Open_vSwitch .
> external-ids:ovn-bridge-mappings=phynet:br-ext
> +
> +check ovn-nbctl lsp-add public public1 \
> + -- lsp-set-addresses public1 unknown \
> + -- lsp-set-type public1 localnet \
> + -- lsp-set-options public1 network_name=phynet
> +
> +check ovn-nbctl --wait=hv sync
> +
> +AT_CHECK([test `ip route show table 1002 | wc -l` -eq 2], [1])
> +
> +# Create dnat_and_snat, dnat rules in R1
> +check ovn-nbctl lr-nat-add R1 dnat_and_snat 172.16.1.10 192.168.1.10
> +check ovn-nbctl lr-nat-add R1 dnat 172.16.1.11 192.168.1.11
> +
> +check ovn-nbctl --wait=hv sync
> +
> +AT_CHECK([ip link | grep -q ovnvrf1002:.*UP])
> +AT_CHECK([test `ip route show table 1002 | wc -l` -eq 2])
> +AT_CHECK([ip route show table 1002 | grep -q 172.16.1.10])
> +AT_CHECK([ip route show table 1002 | grep -q 172.16.1.11])
> +
> +
> +OVS_APP_EXIT_AND_WAIT([ovn-controller])
> +
> +# Ensure system resources are cleaned up
> +AT_CHECK([ip link | grep -q ovnvrf1002:.*UP], [1])
> +AT_CHECK([test `ip route show table 1002 | wc -l` -eq 1], [1])
> +
> +as ovn-sb
> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> +
> +as ovn-nb
> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> +
> +as northd
> +OVS_APP_EXIT_AND_WAIT([ovn-northd])
> +
> +as
> +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d
> +/Failed to acquire.*/d
> +/connection dropped.*/d"])
> +AT_CLEANUP
> +])
> +
> +OVN_FOR_EACH_NORTHD([
> +AT_SETUP([route-exchange for DNAT and DNAT_AND_SNAT with gateway router
> IPv6])
> +AT_KEYWORDS([route-exchange])
> +
> +CHECK_VRF()
> +CHECK_CONNTRACK()
> +CHECK_CONNTRACK_NAT()
> +ovn_start
> +OVS_TRAFFIC_VSWITCHD_START()
> +ADD_BR([br-int])
> +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone])
> +
> +# Set external-ids in br-int needed for ovn-controller
> +ovs-vsctl \
> + -- set Open_vSwitch . external-ids:system-id=hv1 \
> + -- set Open_vSwitch .
> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
> + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
> + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \
> + -- set bridge br-int fail-mode=secure
> other-config:disable-in-band=true
> +
> +# Start ovn-controller
> +start_daemon ovn-controller
> +
> +ovn-appctl vlog/set route_exchange
> +check ovn-nbctl -- lr-add R1 \
> + -- set Logical_Router R1 options:requested-tnl-key=1003
> options:dynamic-routing=true
> +
> +check ovn-nbctl ls-add sw0
> +check ovn-nbctl ls-add public
> +
> +check ovn-nbctl --wait=hv sync
> +
> +AT_CHECK([ip link | grep -q ovnvrf1003:.*UP], [1])
> +
> +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 2001:db8:100::1/64
> +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03
> 2001:db8:1003::1/64 \
> + -- lrp-set-options rp-public \
> + dynamic-routing-maintain-vrf=true \
> + dynamic-routing-nat=true
> +
> +check ovn-nbctl set logical_router R1 options:chassis=hv1
> +
> +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-rp \
> + type=router options:router-port=rp-sw0 \
> + -- lsp-set-addresses sw0-rp router
> +
> +check ovn-nbctl lsp-add public public-rp -- set Logical_Switch_Port
> public-rp \
> + type=router options:router-port=rp-public \
> + -- lsp-set-addresses public-rp router
> +
> +check ovs-vsctl set Open_vSwitch .
> external-ids:ovn-bridge-mappings=phynet:br-ext
> +
> +check ovn-nbctl lsp-add public public1 \
> + -- lsp-set-addresses public1 unknown \
> + -- lsp-set-type public1 localnet \
> + -- lsp-set-options public1 network_name=phynet
> +
> +check ovn-nbctl --wait=hv sync
> +
> +AT_CHECK([test `ip -6 route show table 1003 | wc -l` -eq 2], [1])
> +
> +# Create dnat_and_snat, dnat rules in R1
> +check ovn-nbctl lr-nat-add R1 \
> + dnat_and_snat 2001:db8:1003::150 2001:db8:100::100
> +check ovn-nbctl lr-nat-add R1 \
> + dnat 2001:db8:1003::151 2001:db8:100::100
> +
> +check ovn-nbctl --wait=hv sync
> +
> +AT_CHECK([ip link | grep -q ovnvrf1003:.*UP])
> +AT_CHECK([test `ip -6 route show table 1003 | wc -l` -eq 2])
> +AT_CHECK([ip -6 route show table 1003 | grep -q 2001:db8:1003::150])
> +AT_CHECK([ip -6 route show table 1003 | grep -q 2001:db8:1003::151])
> +
> +OVS_APP_EXIT_AND_WAIT([ovn-controller])
> +
> +# Ensure system resources are cleaned up
> +AT_CHECK([ip link | grep -q ovnvrf1003:.*UP], [1])
> +AT_CHECK([test `ip -6 route show table 1003 | wc -l` -eq 2], [1])
> +
> +as ovn-sb
> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> +
> +as ovn-nb
> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> +
> +as northd
> +OVS_APP_EXIT_AND_WAIT([ovn-northd])
> +
> +as
> +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d
> +/Failed to acquire.*/d
> +/connection dropped.*/d"])
> +AT_CLEANUP
> +])
> +
> --
> 2.43.0
>
> _______________________________________________
> dev mailing list
> [email protected]
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
