There are essentially three problems with the current combination of DGP + SNAT + LB:
1) The first packet is being SNATed in common zone due to a problem with pinctrl not preserving ct_mark/ct_label. The commit would create a SNAT entry within the same with DNAT entry. 2) The unSNAT for reply always happened in common zone because of the loopback check which would be triggered only when we loop the packet through the LR. Now there are two possibilities how the reply packet would be handled: a) If the entry for SNAT in common zone did not time out yet, the packet would be passed through unSNAT in common zone which would be fine and continue on. However, the unDNAT wouldn't work due to the limitation of CT not capable of doing unSNAT/unDNAT on the same packet twice in the same zone. So the reply would arrive to the original interface, but with wrong source address. b) If the entry for SNAT timed out it would loop and do unSNAT correctly in separate zone and then also unDNAT. This is not possible anymore with a recent change 8c341b9d (northd: Drop packets destined to router owned NAT IP for DGP). The reply would be dropped before looping after that change co the traffic would never arrive to the original interface. 3) The unDNAT was happening only if the DGP was outport meaning the reply traffic was routed out, but in the opposite case the unDNAT was happening only because of the looping which made outport=inport. That's why it worked before introduction of explicit drop. In order to fix all those issues do two changes: 1) Include inport in the unDNAT match, so that we have both routing directions covered e.g. (inport == "dgp_port" || outport == "dpg_port"). 2) Always use the separate zone for SNAT and DNAT. As the common zone was needed for HWOL make the common zone optional with configuration option called "use_common_zone". This option is by default "false" and can be specified per LR. Use of separate zones also eliminates the need for the flag propagation in "lr_out_chk_dnat_local" stage, removing the match on ct_mark/ct_label. The "SNAT in separate zone from DNAT" system test is moved to run only with kernel datapath. The reason is that this test doesn't work with userspace datapath due to recirculation limit, currently set to 6 [0]. [0]https://github.com/openvswitch/ovs/blob/9d840923d32124fe427de76e8234c49d64e4bb77/lib/dpif-netdev.c#L102 Reported-at: https://bugzilla.redhat.com/2161281 Signed-off-by: Ales Musil <amu...@redhat.com> --- v2: Fix flaky system test. v3: Rebase on top of current main. v4: Rebase on top of current main. Move the system test to system-ovn-kmod to unblock the failures with userspace. v5: Rebase on top of current main. v6: Rebase on top of current main. Change the config to a global option instead. v7: Address comments from Dumitru: Rename the stateless helper to better reflect that it applies only to snat_and_dnat type. Call the common zone and separate zone functions separately based on condition. --- northd/northd.c | 536 +++++++++++++++++++++------------------ northd/ovn-northd.8.xml | 90 +------ ovn-nb.xml | 9 + tests/ovn-northd.at | 213 +++++++++++----- tests/ovn.at | 3 + tests/system-ovn-kmod.at | 166 ++++++++++++ tests/system-ovn.at | 117 --------- 7 files changed, 634 insertions(+), 500 deletions(-) diff --git a/northd/northd.c b/northd/northd.c index 7a3886de0..1d13c3d8e 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -66,6 +66,9 @@ static bool check_lsp_is_up; static bool install_ls_lb_from_router; +/* Use common zone for SNAT and DNAT if this option is set to "true". */ +static bool use_common_zone = false; + /* MAC allocated for service monitor usage. Just one mac is allocated * for this purpose and ovn-controller's on each chassis will make use * of this mac when sending out the packets to monitor the services @@ -10662,6 +10665,8 @@ build_distr_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, enum lrouter_nat_lb_flow_type type, struct ovn_datapath *od) { + struct ovn_port *dgp = od->l3dgw_ports[0]; + const char *undnat_action; switch (type) { @@ -10673,9 +10678,12 @@ build_distr_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, break; case LROUTER_NAT_LB_FLOW_NORMAL: case LROUTER_NAT_LB_FLOW_MAX: - undnat_action = od->is_gw_router ? "ct_dnat;" : "ct_dnat_in_czone;"; + undnat_action = !od->is_gw_router && use_common_zone + ? "ct_dnat_in_czone;" + : "ct_dnat;"; break; } + /* Store the match lengths, so we can reuse the ds buffer. */ size_t new_match_len = ctx->new_match->length; size_t undnat_match_len = ctx->undnat_match->length; @@ -10702,10 +10710,9 @@ build_distr_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, return; } - ds_put_format(ctx->undnat_match, - ") && outport == %s && is_chassis_resident(%s)", - od->l3dgw_ports[0]->json_key, - od->l3dgw_ports[0]->cr_port->json_key); + ds_put_format(ctx->undnat_match, ") && (inport == %s || outport == %s)" + " && is_chassis_resident(%s)", dgp->json_key, dgp->json_key, + dgp->cr_port->json_key); ovn_lflow_add_with_hint(ctx->lflows, od, S_ROUTER_OUT_UNDNAT, 120, ds_cstr(ctx->undnat_match), undnat_action, &ctx->lb->nlb->header_); @@ -11157,6 +11164,14 @@ lrouter_dnat_and_snat_is_stateless(const struct nbrec_nat *nat) !strcmp(nat->type, "dnat_and_snat"); } +static inline bool +lrouter_use_common_zone_in_nat(const struct ovn_datapath *od, + const struct nbrec_nat *nat) +{ + return !od->is_gw_router && use_common_zone && + !lrouter_dnat_and_snat_is_stateless(nat); +} + /* Handles the match criteria and actions in logical flow * based on external ip based NAT rule filter. * @@ -13648,85 +13663,90 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, } } +static void +build_lrouter_in_unsnat_in_czone_flow(struct hmap *lflows, + struct ovn_datapath *od, + const struct nbrec_nat *nat, + struct ds *match, bool distributed, + bool is_v6, struct ovn_port *l3dgw_port) +{ + if (strcmp(nat->type, "snat") && strcmp(nat->type, "dnat_and_snat")) { + return; + } + + ds_clear(match); + + struct ds zone_match = DS_EMPTY_INITIALIZER; + + ds_put_format(match, "ip && ip%c.dst == %s && inport == %s", + is_v6 ? '6' : '4', nat->external_ip, l3dgw_port->json_key); + ds_clone(&zone_match, match); + + ds_put_cstr(match, " && flags.loopback == 0"); + + /* Update common zone match for the hairpin traffic. */ + ds_put_cstr(&zone_match, " && flags.loopback == 1" + " && flags.use_snat_zone == 1"); + + + if (!distributed && od->n_l3dgw_ports) { + /* Flows for NAT rules that are centralized are only + * programmed on the gateway chassis. */ + ds_put_format(match, " && is_chassis_resident(%s)", + l3dgw_port->cr_port->json_key); + ds_put_format(&zone_match, " && is_chassis_resident(%s)", + l3dgw_port->cr_port->json_key); + } + + ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_UNSNAT, + 100, ds_cstr(match), "ct_snat_in_czone;", + &nat->header_); + + ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_UNSNAT, + 100, ds_cstr(&zone_match), "ct_snat;", + &nat->header_); + + ds_destroy(&zone_match); +} + static void build_lrouter_in_unsnat_flow(struct hmap *lflows, struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, - struct ds *actions, bool distributed, bool is_v6, + bool distributed, bool is_v6, struct ovn_port *l3dgw_port) { - /* Ingress UNSNAT table: It is for already established connections' - * reverse traffic. i.e., SNAT has already been done in egress - * pipeline and now the packet has entered the ingress pipeline as - * part of a reply. We undo the SNAT here. - * - * Undoing SNAT has to happen before DNAT processing. This is - * because when the packet was DNATed in ingress pipeline, it did - * not know about the possibility of eventual additional SNAT in - * egress pipeline. */ + if (strcmp(nat->type, "snat") && strcmp(nat->type, "dnat_and_snat")) { return; } - bool stateless = lrouter_dnat_and_snat_is_stateless(nat); - if (od->is_gw_router) { - ds_clear(match); - ds_clear(actions); - ds_put_format(match, "ip && ip%s.dst == %s", - is_v6 ? "6" : "4", nat->external_ip); - if (stateless) { - ds_put_format(actions, "next;"); - } else { - ds_put_cstr(actions, "ct_snat;"); - } + ds_clear(match); - ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_UNSNAT, - 90, ds_cstr(match), ds_cstr(actions), - &nat->header_); - } else { + uint16_t priority = od->is_gw_router ? 90 : 100; + const char *action = lrouter_dnat_and_snat_is_stateless(nat) + ? "next;" + : "ct_snat;"; + + ds_put_format(match, "ip && ip%c.dst == %s", + is_v6 ? '6' : '4', nat->external_ip); + + if (!od->is_gw_router) { /* Distributed router. */ /* Traffic received on l3dgw_port is subject to NAT. */ - ds_clear(match); - ds_clear(actions); - ds_put_format(match, "ip && ip%s.dst == %s && inport == %s && " - "flags.loopback == 0", is_v6 ? "6" : "4", - nat->external_ip, l3dgw_port->json_key); + ds_put_format(match, " && inport == %s", l3dgw_port->json_key); + if (!distributed && od->n_l3dgw_ports) { /* Flows for NAT rules that are centralized are only - * programmed on the gateway chassis. */ + * programmed on the gateway chassis. */ ds_put_format(match, " && is_chassis_resident(%s)", l3dgw_port->cr_port->json_key); } - - if (stateless) { - ds_put_format(actions, "next;"); - } else { - ds_put_cstr(actions, "ct_snat_in_czone;"); - } - - ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_UNSNAT, - 100, ds_cstr(match), ds_cstr(actions), - &nat->header_); - - if (!stateless) { - ds_clear(match); - ds_clear(actions); - ds_put_format(match, "ip && ip%s.dst == %s && inport == %s && " - "flags.loopback == 1 && flags.use_snat_zone == 1", - is_v6 ? "6" : "4", nat->external_ip, - l3dgw_port->json_key); - if (!distributed && od->n_l3dgw_ports) { - /* Flows for NAT rules that are centralized are only - * programmed on the gateway chassis. */ - ds_put_format(match, " && is_chassis_resident(%s)", - l3dgw_port->cr_port->json_key); - } - ds_put_cstr(actions, "ct_snat;"); - ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_UNSNAT, - 100, ds_cstr(match), ds_cstr(actions), - &nat->header_); - } } + + ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_UNSNAT, + priority, ds_cstr(match), action, + &nat->header_); } static void @@ -13739,82 +13759,64 @@ build_lrouter_in_dnat_flow(struct hmap *lflows, struct ovn_datapath *od, /* Ingress DNAT table: Packets enter the pipeline with destination * IP address that needs to be DNATted from a external IP address * to a logical IP address. */ - if (!strcmp(nat->type, "dnat") || !strcmp(nat->type, "dnat_and_snat")) { - bool stateless = lrouter_dnat_and_snat_is_stateless(nat); + if (strcmp(nat->type, "dnat") && strcmp(nat->type, "dnat_and_snat")) { + return; + } - if (od->is_gw_router) { - /* Packet when it goes from the initiator to destination. - * We need to set flags.loopback because the router can - * send the packet back through the same interface. */ - ds_clear(match); - ds_put_format(match, "ip && ip%s.dst == %s", - is_v6 ? "6" : "4", nat->external_ip); - ds_clear(actions); - if (nat->allowed_ext_ips || nat->exempted_ext_ips) { - lrouter_nat_add_ext_ip_match(od, lflows, match, nat, - is_v6, true, cidr_bits); - } + ds_clear(match); + ds_clear(actions); - if (!lport_addresses_is_empty(&od->dnat_force_snat_addrs)) { - /* Indicate to the future tables that a DNAT has taken - * place and a force SNAT needs to be done in the - * Egress SNAT table. */ - ds_put_format(actions, "flags.force_snat_for_dnat = 1; "); - } + const char *nat_action = !od->is_gw_router && use_common_zone + ? "ct_dnat_in_czone" + : "ct_dnat"; - if (stateless) { - ds_put_format(actions, "flags.loopback = 1; " - "ip%s.dst=%s; next;", - is_v6 ? "6" : "4", nat->logical_ip); - } else { - ds_put_format(actions, "flags.loopback = 1; ct_dnat(%s", - nat->logical_ip); + ds_put_format(match, "ip && ip%c.dst == %s", is_v6 ? '6' : '4', + nat->external_ip); - if (nat->external_port_range[0]) { - ds_put_format(actions, ",%s", nat->external_port_range); - } - ds_put_format(actions, ");"); - } + if (od->is_gw_router) { + if (!lport_addresses_is_empty(&od->dnat_force_snat_addrs)) { + /* Indicate to the future tables that a DNAT has taken + * place and a force SNAT needs to be done in the + * Egress SNAT table. */ + ds_put_cstr(actions, "flags.force_snat_for_dnat = 1; "); + } - ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_DNAT, 100, - ds_cstr(match), ds_cstr(actions), - &nat->header_); - } else { - /* Distributed router. */ + /* Packet when it goes from the initiator to destination. + * We need to set flags.loopback because the router can + * send the packet back through the same interface. */ + ds_put_cstr(actions, "flags.loopback = 1; "); + } else { + /* Distributed router. */ - /* Traffic received on l3dgw_port is subject to NAT. */ - ds_clear(match); - ds_put_format(match, "ip && ip%s.dst == %s && inport == %s", - is_v6 ? "6" : "4", nat->external_ip, - l3dgw_port->json_key); - if (!distributed && od->n_l3dgw_ports) { - /* Flows for NAT rules that are centralized are only - * programmed on the gateway chassis. */ - ds_put_format(match, " && is_chassis_resident(%s)", - l3dgw_port->cr_port->json_key); - } - ds_clear(actions); - if (nat->allowed_ext_ips || nat->exempted_ext_ips) { - lrouter_nat_add_ext_ip_match(od, lflows, match, nat, - is_v6, true, cidr_bits); - } + /* Traffic received on l3dgw_port is subject to NAT. */ + ds_put_format(match, " && inport == %s", l3dgw_port->json_key); + if (!distributed && od->n_l3dgw_ports) { + /* Flows for NAT rules that are centralized are only + * programmed on the gateway chassis. */ + ds_put_format(match, " && is_chassis_resident(%s)", + l3dgw_port->cr_port->json_key); + } + } - if (!strcmp(nat->type, "dnat_and_snat") && stateless) { - ds_put_format(actions, "ip%s.dst=%s; next;", - is_v6 ? "6" : "4", nat->logical_ip); - } else { - ds_put_format(actions, "ct_dnat_in_czone(%s", nat->logical_ip); - if (nat->external_port_range[0]) { - ds_put_format(actions, ",%s", nat->external_port_range); - } - ds_put_format(actions, ");"); - } + if (nat->allowed_ext_ips || nat->exempted_ext_ips) { + lrouter_nat_add_ext_ip_match(od, lflows, match, nat, + is_v6, true, cidr_bits); + } - ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_DNAT, 100, - ds_cstr(match), ds_cstr(actions), - &nat->header_); + if (lrouter_dnat_and_snat_is_stateless(nat)) { + ds_put_format(actions, "ip%c.dst=%s; next;", + is_v6 ? '6' : '4', nat->logical_ip); + } else { + ds_put_format(actions, "%s(%s", nat_action, nat->logical_ip); + if (nat->external_port_range[0]) { + ds_put_format(actions, ",%s", nat->external_port_range); } + ds_put_format(actions, ");"); } + + ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_DNAT, 100, + ds_cstr(match), ds_cstr(actions), + &nat->header_); } static void @@ -13837,8 +13839,10 @@ build_lrouter_out_undnat_flow(struct hmap *lflows, struct ovn_datapath *od, } ds_clear(match); - ds_put_format(match, "ip && ip%s.src == %s && outport == %s", - is_v6 ? "6" : "4", nat->logical_ip, + ds_clear(actions); + + ds_put_format(match, "ip && ip%c.src == %s && outport == %s", + is_v6 ? '6' : '4', nat->logical_ip, l3dgw_port->json_key); if (!distributed && od->n_l3dgw_ports) { /* Flows for NAT rules that are centralized are only @@ -13846,7 +13850,7 @@ build_lrouter_out_undnat_flow(struct hmap *lflows, struct ovn_datapath *od, ds_put_format(match, " && is_chassis_resident(%s)", l3dgw_port->cr_port->json_key); } - ds_clear(actions); + if (distributed) { ds_put_format(actions, "eth.src = "ETH_ADDR_FMT"; ", ETH_ADDR_ARGS(mac)); @@ -13855,8 +13859,8 @@ build_lrouter_out_undnat_flow(struct hmap *lflows, struct ovn_datapath *od, if (lrouter_dnat_and_snat_is_stateless(nat)) { ds_put_format(actions, "next;"); } else { - ds_put_format(actions, - od->is_gw_router ? "ct_dnat;" : "ct_dnat_in_czone;"); + ds_put_cstr(actions, + use_common_zone ? "ct_dnat_in_czone;" : "ct_dnat;"); } ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_UNDNAT, 100, @@ -13918,6 +13922,77 @@ build_lrouter_drop_ct_inv_flow(struct ovn_datapath *od, struct hmap *lflows) "ip && ct.trk && ct.inv", debug_drop_action()); } +static void +build_lrouter_out_snat_in_czone_flow(struct hmap *lflows, + struct ovn_datapath *od, + const struct nbrec_nat *nat, + struct ds *match, + struct ds *actions, bool distributed, + struct eth_addr mac, int cidr_bits, + bool is_v6, struct ovn_port *l3dgw_port) +{ + if (strcmp(nat->type, "snat") && strcmp(nat->type, "dnat_and_snat")) { + return; + } + + ds_clear(match); + ds_clear(actions); + + /* The priority here is calculated such that the + * nat->logical_ip with the longest mask gets a higher + * priority. */ + uint16_t priority = cidr_bits + 1; + struct ds zone_actions = DS_EMPTY_INITIALIZER; + + ds_put_format(match, "ip && ip%c.src == %s && outport == %s", + is_v6 ? '6' : '4', nat->logical_ip, l3dgw_port->json_key); + + if (od->n_l3dgw_ports) { + priority += 128; + ds_put_format(match, " && is_chassis_resident(\"%s\")", + distributed + ? nat->logical_port + : l3dgw_port->cr_port->key); + } + + if (distributed) { + ds_put_format(actions, "eth.src = "ETH_ADDR_FMT"; ", + ETH_ADDR_ARGS(mac)); + ds_put_format(&zone_actions, "eth.src = "ETH_ADDR_FMT"; ", + ETH_ADDR_ARGS(mac)); + } + + if (nat->allowed_ext_ips || nat->exempted_ext_ips) { + lrouter_nat_add_ext_ip_match(od, lflows, match, nat, + is_v6, false, cidr_bits); + } + + ds_put_cstr(&zone_actions, REGBIT_DST_NAT_IP_LOCAL" = 0; "); + + ds_put_format(actions, "ct_snat_in_czone(%s", nat->external_ip); + ds_put_format(&zone_actions, "ct_snat(%s", nat->external_ip); + + if (nat->external_port_range[0]) { + ds_put_format(actions, ",%s", nat->external_port_range); + ds_put_format(&zone_actions, ",%s", nat->external_port_range); + } + + ds_put_cstr(actions, ");"); + ds_put_cstr(&zone_actions, ");"); + + ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_SNAT, + priority, ds_cstr(match), + ds_cstr(actions), &nat->header_); + + ds_put_cstr(match, " && "REGBIT_DST_NAT_IP_LOCAL" == 1"); + + ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_SNAT, + priority + 1, ds_cstr(match), + ds_cstr(&zone_actions), &nat->header_); + + ds_destroy(&zone_actions); +} + static void build_lrouter_out_snat_flow(struct hmap *lflows, struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, @@ -13925,116 +14000,64 @@ build_lrouter_out_snat_flow(struct hmap *lflows, struct ovn_datapath *od, struct eth_addr mac, int cidr_bits, bool is_v6, struct ovn_port *l3dgw_port) { - /* Egress SNAT table: Packets enter the egress pipeline with - * source ip address that needs to be SNATted to a external ip - * address. */ if (strcmp(nat->type, "snat") && strcmp(nat->type, "dnat_and_snat")) { return; } - bool stateless = lrouter_dnat_and_snat_is_stateless(nat); - if (od->is_gw_router) { - ds_clear(match); - ds_put_format(match, "ip && ip%s.src == %s", - is_v6 ? "6" : "4", nat->logical_ip); - ds_clear(actions); - - if (nat->allowed_ext_ips || nat->exempted_ext_ips) { - lrouter_nat_add_ext_ip_match(od, lflows, match, nat, - is_v6, false, cidr_bits); - } + ds_clear(match); + ds_clear(actions); - if (!strcmp(nat->type, "dnat_and_snat") && stateless) { - ds_put_format(actions, "ip%s.src=%s; next;", - is_v6 ? "6" : "4", nat->external_ip); - } else { - ds_put_format(match, " && (!ct.trk || !ct.rpl)"); - ds_put_format(actions, "ct_snat(%s", nat->external_ip); + bool stateless = lrouter_dnat_and_snat_is_stateless(nat); - if (nat->external_port_range[0]) { - ds_put_format(actions, ",%s", - nat->external_port_range); - } - ds_put_format(actions, ");"); - } + /* The priority here is calculated such that the + * nat->logical_ip with the longest mask gets a higher + * priority. */ + uint16_t priority = cidr_bits + 1; - /* The priority here is calculated such that the - * nat->logical_ip with the longest mask gets a higher - * priority. */ - ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_SNAT, - cidr_bits + 1, ds_cstr(match), - ds_cstr(actions), &nat->header_); - } else { - uint16_t priority = cidr_bits + 1; + ds_put_format(match, "ip && ip%c.src == %s", + is_v6 ? '6' : '4', nat->logical_ip); + if (!od->is_gw_router) { /* Distributed router. */ - ds_clear(match); - ds_put_format(match, "ip && ip%s.src == %s && outport == %s", - is_v6 ? "6" : "4", nat->logical_ip, - l3dgw_port->json_key); + ds_put_format(match, " && outport == %s", l3dgw_port->json_key); if (od->n_l3dgw_ports) { - if (distributed) { - ovs_assert(nat->logical_port); - priority += 128; - ds_put_format(match, " && is_chassis_resident(\"%s\")", - nat->logical_port); - } else { - /* Flows for NAT rules that are centralized are only - * programmed on the gateway chassis. */ - priority += 128; - ds_put_format(match, " && is_chassis_resident(%s)", - l3dgw_port->cr_port->json_key); - } - } - ds_clear(actions); - - if (nat->allowed_ext_ips || nat->exempted_ext_ips) { - lrouter_nat_add_ext_ip_match(od, lflows, match, nat, - is_v6, false, cidr_bits); + priority += 128; + ds_put_format(match, " && is_chassis_resident(\"%s\")", + distributed + ? nat->logical_port + : l3dgw_port->cr_port->key); } if (distributed) { ds_put_format(actions, "eth.src = "ETH_ADDR_FMT"; ", ETH_ADDR_ARGS(mac)); } + } - if (stateless) { - ds_put_format(actions, "ip%s.src=%s; next;", - is_v6 ? "6" : "4", nat->external_ip); - } else { - ds_put_format(actions, "ct_snat_in_czone(%s", - nat->external_ip); - if (nat->external_port_range[0]) { - ds_put_format(actions, ",%s", nat->external_port_range); - } - ds_put_format(actions, ");"); - } + if (nat->allowed_ext_ips || nat->exempted_ext_ips) { + lrouter_nat_add_ext_ip_match(od, lflows, match, nat, + is_v6, false, cidr_bits); + } - /* The priority here is calculated such that the - * nat->logical_ip with the longest mask gets a higher - * priority. */ - ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_SNAT, - priority, ds_cstr(match), - ds_cstr(actions), &nat->header_); + if (od->is_gw_router && !stateless) { + /* Gateway router. */ + ds_put_cstr(match, " && (!ct.trk || !ct.rpl)"); + } - if (!stateless) { - ds_put_cstr(match, " && "REGBIT_DST_NAT_IP_LOCAL" == 1"); - ds_clear(actions); - if (distributed) { - ds_put_format(actions, "eth.src = "ETH_ADDR_FMT"; ", - ETH_ADDR_ARGS(mac)); - } - ds_put_format(actions, REGBIT_DST_NAT_IP_LOCAL" = 0; ct_snat(%s", - nat->external_ip); - if (nat->external_port_range[0]) { - ds_put_format(actions, ",%s", nat->external_port_range); - } - ds_put_format(actions, ");"); - ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_SNAT, - priority + 1, ds_cstr(match), - ds_cstr(actions), &nat->header_); + if (stateless) { + ds_put_format(actions, "ip%c.src=%s; next;", + is_v6 ? '6' : '4', nat->external_ip); + } else { + ds_put_format(actions, "ct_snat(%s", nat->external_ip); + if (nat->external_port_range[0]) { + ds_put_format(actions, ",%s", nat->external_port_range); } + ds_put_format(actions, ");"); } + + ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_SNAT, + priority, ds_cstr(match), + ds_cstr(actions), &nat->header_); } static void @@ -14420,12 +14443,27 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, continue; } - /* S_ROUTER_IN_UNSNAT */ - build_lrouter_in_unsnat_flow(lflows, od, nat, match, actions, distributed, - is_v6, l3dgw_port); + /* S_ROUTER_IN_UNSNAT + * Ingress UNSNAT table: It is for already established connections' + * reverse traffic. i.e., SNAT has already been done in egress + * pipeline and now the packet has entered the ingress pipeline as + * part of a reply. We undo the SNAT here. + * + * Undoing SNAT has to happen before DNAT processing. This is + * because when the packet was DNATed in ingress pipeline, it did + * not know about the possibility of eventual additional SNAT in + * egress pipeline. */ + if (lrouter_use_common_zone_in_nat(od, nat)) { + build_lrouter_in_unsnat_in_czone_flow(lflows, od, nat, match, + distributed, is_v6, + l3dgw_port); + } else { + build_lrouter_in_unsnat_flow(lflows, od, nat, match, distributed, + is_v6, l3dgw_port); + } /* S_ROUTER_IN_DNAT */ - build_lrouter_in_dnat_flow(lflows, od, nat, match, actions, distributed, - cidr_bits, is_v6, l3dgw_port); + build_lrouter_in_dnat_flow(lflows, od, nat, match, actions, + distributed, cidr_bits, is_v6, l3dgw_port); /* ARP resolve for NAT IPs. */ if (od->is_gw_router) { @@ -14494,16 +14532,28 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, } } - /* S_ROUTER_OUT_DNAT_LOCAL */ - build_lrouter_out_is_dnat_local(lflows, od, nat, match, actions, - distributed, is_v6, l3dgw_port); + if (use_common_zone) { + /* S_ROUTER_OUT_DNAT_LOCAL */ + build_lrouter_out_is_dnat_local(lflows, od, nat, match, actions, + distributed, is_v6, l3dgw_port); + } /* S_ROUTER_OUT_UNDNAT */ - build_lrouter_out_undnat_flow(lflows, od, nat, match, actions, distributed, - mac, is_v6, l3dgw_port); - /* S_ROUTER_OUT_SNAT */ - build_lrouter_out_snat_flow(lflows, od, nat, match, actions, distributed, - mac, cidr_bits, is_v6, l3dgw_port); + build_lrouter_out_undnat_flow(lflows, od, nat, match, actions, + distributed, mac, is_v6, l3dgw_port); + /* S_ROUTER_OUT_SNAT + * Egress SNAT table: Packets enter the egress pipeline with + * source ip address that needs to be SNATted to a external ip + * address. */ + if (lrouter_use_common_zone_in_nat(od, nat)) { + build_lrouter_out_snat_in_czone_flow(lflows, od, nat, match, + actions, distributed, mac, + cidr_bits, is_v6, l3dgw_port); + } else { + build_lrouter_out_snat_flow(lflows, od, nat, match, actions, + distributed, mac, cidr_bits, is_v6, + l3dgw_port); + } /* S_ROUTER_IN_ADMISSION - S_ROUTER_IN_IP_INPUT */ build_lrouter_ingress_flow(lflows, od, nat, match, actions, mac, @@ -14573,8 +14623,11 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, "clone { ct_clear; " "inport = outport; outport = \"\"; " "eth.dst <-> eth.src; " - "flags = 0; flags.loopback = 1; " - "flags.use_snat_zone = "REGBIT_DST_NAT_IP_LOCAL"; "); + "flags = 0; flags.loopback = 1; "); + if (use_common_zone) { + ds_put_cstr(actions, "flags.use_snat_zone = " + REGBIT_DST_NAT_IP_LOCAL"; "); + } for (int j = 0; j < MFF_N_LOG_REGS; j++) { ds_put_format(actions, "reg%d = 0; ", j); } @@ -14587,7 +14640,7 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, } } - if (od->nbr->n_nat) { + if (use_common_zone && od->nbr->n_nat) { ds_clear(match); const char *ct_natted = features->ct_no_masked_label ? "ct_mark.natted" : @@ -16501,6 +16554,7 @@ ovnnb_db_run(struct northd_input *input_data, install_ls_lb_from_router = smap_get_bool(&nb->options, "install_ls_lb_from_router", false); + use_common_zone = smap_get_bool(&nb->options, "use_common_zone", false); build_chassis_features(input_data->sbrec_chassis_table, &data->features); diff --git a/northd/ovn-northd.8.xml b/northd/ovn-northd.8.xml index 4100cae0d..f11cdc468 100644 --- a/northd/ovn-northd.8.xml +++ b/northd/ovn-northd.8.xml @@ -3244,13 +3244,11 @@ icmp6 { <p> The first flow matches <code>ip && ip4.dst == <var>B</var> && inport == <var>GW</var> - && flags.loopback == 0</code> or - <code>ip && - ip6.dst == <var>B</var> && inport == <var>GW</var> - && flags.loopback == 0</code> + </code> or <code>ip && ip6.dst == <var>B</var> && + inport == <var>GW</var></code> where <var>GW</var> is the distributed gateway port corresponding to the NAT rule (specified or inferred), with an - action <code>ct_snat_in_czone;</code> to unSNAT in the common + action <code>ct_snat;</code> to unSNAT in the common zone. If the NAT rule is of type dnat_and_snat and has <code>stateless=true</code> in the options, then the action would be <code>next;</code>. @@ -3263,32 +3261,6 @@ icmp6 { <var>GW</var>. </p> </li> - - <li> - <p> - The second flow matches <code>ip && - ip4.dst == <var>B</var> && inport == <var>GW</var> - && flags.loopback == 1 && - flags.use_snat_zone == 1</code> or - <code>ip && - ip6.dst == <var>B</var> && inport == <var>GW</var> - && flags.loopback == 0 && - flags.use_snat_zone == 1</code> - where <var>GW</var> is the distributed gateway port - corresponding to the NAT rule (specified or inferred), with an - action <code>ct_snat;</code> to unSNAT in the snat zone. If the - NAT rule is of type dnat_and_snat and has - <code>stateless=true</code> in the options, then the action - would be <code>ip4/6.dst=(<var>B</var>)</code>. - </p> - - <p> - If the NAT entry is of type <code>snat</code>, then there is an - additional match <code>is_chassis_resident(<var>cr-GW</var>) - </code> where <var>cr-GW</var> is the chassis resident port of - <var>GW</var>. - </p> - </li> </ul> <p> @@ -4613,46 +4585,12 @@ nd_ns { </p> <ul> - <li> - <p> - For each NAT rule in the OVN Northbound database on a - distributed router, a priority-50 logical flow with match - <code>ip4.dst == <var>E</var> && - is_chassis_resident(<var>P</var>)</code>, where <var>E</var> is the - external IP address specified in the NAT rule, <var>GW</var> - is the logical router distributed gateway port. For dnat_and_snat - NAT rule, <var>P</var> is the logical port specified in the NAT rule. - If <ref column="logical_port" - table="NAT" db="OVN_Northbound"/> column of - <ref table="NAT" db="OVN_Northbound"/> table is NOT set, then - <var>P</var> is the <code>chassisredirect port</code> of - <var>GW</var> with the actions: - <code>REGBIT_DST_NAT_IP_LOCAL = 1; next; </code> - </p> - </li> - <li> A priority-0 logical flow with match <code>1</code> has actions <code>REGBIT_DST_NAT_IP_LOCAL = 0; next;</code>. </li> </ul> - <p> - This table also installs a priority-50 logical flow for each logical - router that has NATs configured on it. The flow has match - <code>ip && ct_label.natted == 1</code> and action - <code>REGBIT_DST_NAT_IP_LOCAL = 1; next;</code>. This is intended - to ensure that traffic that was DNATted locally will use a separate - conntrack zone for SNAT if SNAT is required later in the egress - pipeline. Note that this flow checks the value of - <code>ct_label.natted</code>, which is set in the ingress pipeline. - This means that ovn-northd assumes that this value is carried over - from the ingress pipeline to the egress pipeline and is not altered - or cleared. If conntrack label values are ever changed to be cleared - between the ingress and egress pipelines, then the match conditions - of this flow will be updated accordingly. - </p> - <h3>Egress Table 1: UNDNAT</h3> <p> @@ -4694,7 +4632,7 @@ nd_ns { gateway chassis that matches <code>ip && ip4.src == <var>B</var> && outport == <var>GW</var></code>, where <var>GW</var> is the logical - router gateway port with an action <code>ct_dnat_in_czone;</code>. + router gateway port with an action <code>ct_dnat;</code>. If the backend IPv4 address <var>B</var> is also configured with L4 port <var>PORT</var> of protocol <var>P</var>, then the match also includes <code>P.src</code> == <var>PORT</var>. These @@ -4716,7 +4654,7 @@ nd_ns { matches <code>ip && ip4.src == <var>B</var> && outport == <var>GW</var></code>, where <var>GW</var> is the logical router gateway port, with an action - <code>ct_dnat_in_czone;</code>. If the NAT rule is of type + <code>ct_dnat;</code>. If the NAT rule is of type dnat_and_snat and has <code>stateless=true</code> in the options, then the action would be <code>next;</code>. </p> @@ -4724,7 +4662,7 @@ nd_ns { <p> If the NAT rule cannot be handled in a distributed manner, then the priority-100 flow above is only programmed on the - gateway chassis with the action <code>ct_dnat_in_czone</code>. + gateway chassis with the action <code>ct_dnat</code>. </p> <p> @@ -4900,24 +4838,11 @@ nd_ns { and match <code>ip && ip4.src == <var>A</var> && outport == <var>GW</var></code>, where <var>GW</var> is the logical router gateway port, with an action - <code>ct_snat_in_czone(<var>B</var>);</code> to SNATed in the + <code>ct_snat(<var>B</var>);</code> to SNATed in the common zone. If the NAT rule is of type dnat_and_snat and has <code>stateless=true</code> in the options, then the action would be <code>ip4/6.src=(<var>B</var>)</code>. </li> - - <li> - The second flow is added with the calculated priority - <code><var>P</var> + 1 </code> and match - <code>ip && ip4.src == <var>A</var> && - outport == <var>GW</var> && - REGBIT_DST_NAT_IP_LOCAL == 0</code>, where <var>GW</var> is the - logical router gateway port, with an action - <code>ct_snat(<var>B</var>);</code> to SNAT in the snat zone. - If the NAT rule is of type dnat_and_snat and has - <code>stateless=true</code> in the options, then the action would - be <code>ip4/6.src=(<var>B</var>)</code>. - </li> </ul> <p> @@ -5024,7 +4949,6 @@ clone { outport = ""; flags = 0; flags.loopback = 1; - flags.use_snat_zone = REGBIT_DST_NAT_IP_LOCAL; reg0 = 0; reg1 = 0; ... diff --git a/ovn-nb.xml b/ovn-nb.xml index d6694778f..0552eff19 100644 --- a/ovn-nb.xml +++ b/ovn-nb.xml @@ -314,6 +314,15 @@ </p> </column> + <column name="options" key="use_common_zone" type='{"type": "boolean"}'> + Default value is <code>false</code>. If set to <code>true</code> + the SNAT and DNAT happens in common zone, instead of happening in + separate zones, depending on the configuration. However, this option + breaks traffic when there is configuration of DGP + LB + SNAT on + this LR. The value <code>true</code> should be used only in case + of HWOL compatibility with GDP. + </column> + <group title="Options for configuring interconnection route advertisement"> <p> These options control how routes are advertised between OVN diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index a3db189ac..6f5650416 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -892,7 +892,7 @@ check_flow_match_sets() { echo echo "IPv4: stateful" ovn-nbctl --wait=sb lr-nat-add R1 dnat_and_snat 172.16.1.1 50.0.0.11 -check_flow_match_sets 3 4 2 0 0 0 0 +check_flow_match_sets 2 2 2 0 0 0 0 ovn-nbctl lr-nat-del R1 dnat_and_snat 172.16.1.1 echo @@ -904,7 +904,7 @@ ovn-nbctl lr-nat-del R1 dnat_and_snat 172.16.1.1 echo echo "IPv6: stateful" ovn-nbctl --wait=sb lr-nat-add R1 dnat_and_snat fd01::1 fd11::2 -check_flow_match_sets 3 4 2 0 0 0 0 +check_flow_match_sets 2 2 2 0 0 0 0 ovn-nbctl lr-nat-del R1 dnat_and_snat fd01::1 echo @@ -939,9 +939,9 @@ echo "CR-LRP UUID is: " $uuid ovn-nbctl --portrange lr-nat-add R1 dnat_and_snat 172.16.1.1 50.0.0.11 1-3000 AT_CAPTURE_FILE([sbflows]) -OVS_WAIT_UNTIL([ovn-sbctl dump-flows R1 > sbflows && test 3 = `grep -c lr_in_unsnat sbflows`]) +OVS_WAIT_UNTIL([ovn-sbctl dump-flows R1 > sbflows && test 2 = `grep -c lr_in_unsnat sbflows`]) AT_CHECK([grep -c 'ct_snat.*3000' sbflows && grep -c 'ct_dnat.*3000' sbflows], - [0], [2 + [0], [1 1 ]) @@ -949,9 +949,9 @@ ovn-nbctl lr-nat-del R1 dnat_and_snat 172.16.1.1 ovn-nbctl --wait=sb --portrange lr-nat-add R1 snat 172.16.1.1 50.0.0.11 1-3000 AT_CAPTURE_FILE([sbflows2]) -OVS_WAIT_UNTIL([ovn-sbctl dump-flows R1 > sbflows2 && test 3 = `grep -c lr_in_unsnat sbflows`]) +OVS_WAIT_UNTIL([ovn-sbctl dump-flows R1 > sbflows2 && test 2 = `grep -c lr_in_unsnat sbflows`]) AT_CHECK([grep -c 'ct_snat.*3000' sbflows2 && grep -c 'ct_dnat.*3000' sbflows2], - [1], [2 + [1], [1 0 ]) @@ -959,7 +959,7 @@ ovn-nbctl lr-nat-del R1 snat 172.16.1.1 ovn-nbctl --wait=sb --portrange --stateless lr-nat-add R1 dnat_and_snat 172.16.1.2 50.0.0.12 1-3000 AT_CAPTURE_FILE([sbflows3]) -OVS_WAIT_UNTIL([ovn-sbctl dump-flows R1 > sbflows3 && test 4 = `grep -c lr_in_unsnat sbflows3`]) +OVS_WAIT_UNTIL([ovn-sbctl dump-flows R1 > sbflows3 && test 3 = `grep -c lr_in_unsnat sbflows3`]) AT_CHECK([grep 'ct_[s]dnat.*172\.16\.1\.2.*3000' sbflows3], [1]) ovn-nbctl lr-nat-del R1 dnat_and_snat 172.16.1.1 @@ -1026,8 +1026,7 @@ AT_CAPTURE_FILE([crflows]) AT_CHECK([grep -e "lr_out_snat" drflows | sed 's/table=../table=??/' | sort], [0], [dnl table=??(lr_out_snat ), priority=0 , match=(1), action=(next;) table=??(lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) - table=??(lr_out_snat ), priority=161 , match=(ip && ip4.src == 50.0.0.11 && outport == "DR-S1" && is_chassis_resident("cr-DR-S1") && ip4.dst == $allowed_range), action=(ct_snat_in_czone(172.16.1.1);) - table=??(lr_out_snat ), priority=162 , match=(ip && ip4.src == 50.0.0.11 && outport == "DR-S1" && is_chassis_resident("cr-DR-S1") && ip4.dst == $allowed_range && reg9[[4]] == 1), action=(reg9[[4]] = 0; ct_snat(172.16.1.1);) + table=??(lr_out_snat ), priority=161 , match=(ip && ip4.src == 50.0.0.11 && outport == "DR-S1" && is_chassis_resident("cr-DR-S1") && ip4.dst == $allowed_range), action=(ct_snat(172.16.1.1);) ]) AT_CHECK([grep -e "lr_out_snat" crflows | sed 's/table=../table=??/' | sort], [0], [dnl @@ -1057,8 +1056,7 @@ AT_CAPTURE_FILE([crflows2]) AT_CHECK([grep -e "lr_out_snat" drflows2 | sed 's/table=../table=??/' | sort], [0], [dnl table=??(lr_out_snat ), priority=0 , match=(1), action=(next;) table=??(lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) - table=??(lr_out_snat ), priority=161 , match=(ip && ip4.src == 50.0.0.11 && outport == "DR-S1" && is_chassis_resident("cr-DR-S1")), action=(ct_snat_in_czone(172.16.1.1);) - table=??(lr_out_snat ), priority=162 , match=(ip && ip4.src == 50.0.0.11 && outport == "DR-S1" && is_chassis_resident("cr-DR-S1") && reg9[[4]] == 1), action=(reg9[[4]] = 0; ct_snat(172.16.1.1);) + table=??(lr_out_snat ), priority=161 , match=(ip && ip4.src == 50.0.0.11 && outport == "DR-S1" && is_chassis_resident("cr-DR-S1")), action=(ct_snat(172.16.1.1);) table=??(lr_out_snat ), priority=163 , match=(ip && ip4.src == 50.0.0.11 && outport == "DR-S1" && is_chassis_resident("cr-DR-S1") && ip4.dst == $disallowed_range), action=(next;) ]) @@ -1087,8 +1085,7 @@ AT_CAPTURE_FILE([crflows2]) AT_CHECK([grep -e "lr_out_snat" drflows3 | sed 's/table=../table=??/' | sort], [0], [dnl table=??(lr_out_snat ), priority=0 , match=(1), action=(next;) table=??(lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) - table=??(lr_out_snat ), priority=161 , match=(ip && ip4.src == 50.0.0.11 && outport == "DR-S1" && is_chassis_resident("cr-DR-S1") && ip4.dst == $allowed_range), action=(ct_snat_in_czone(172.16.1.2);) - table=??(lr_out_snat ), priority=162 , match=(ip && ip4.src == 50.0.0.11 && outport == "DR-S1" && is_chassis_resident("cr-DR-S1") && ip4.dst == $allowed_range && reg9[[4]] == 1), action=(reg9[[4]] = 0; ct_snat(172.16.1.2);) + table=??(lr_out_snat ), priority=161 , match=(ip && ip4.src == 50.0.0.11 && outport == "DR-S1" && is_chassis_resident("cr-DR-S1") && ip4.dst == $allowed_range), action=(ct_snat(172.16.1.2);) ]) AT_CHECK([grep -e "lr_out_snat" crflows3 | sed 's/table=../table=??/' | sort], [0], [dnl @@ -1115,8 +1112,7 @@ AT_CAPTURE_FILE([crflows2]) AT_CHECK([grep -e "lr_out_snat" drflows4 | sed 's/table=../table=??/' | sort], [0], [dnl table=??(lr_out_snat ), priority=0 , match=(1), action=(next;) table=??(lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) - table=??(lr_out_snat ), priority=161 , match=(ip && ip4.src == 50.0.0.11 && outport == "DR-S1" && is_chassis_resident("cr-DR-S1")), action=(ct_snat_in_czone(172.16.1.2);) - table=??(lr_out_snat ), priority=162 , match=(ip && ip4.src == 50.0.0.11 && outport == "DR-S1" && is_chassis_resident("cr-DR-S1") && reg9[[4]] == 1), action=(reg9[[4]] = 0; ct_snat(172.16.1.2);) + table=??(lr_out_snat ), priority=161 , match=(ip && ip4.src == 50.0.0.11 && outport == "DR-S1" && is_chassis_resident("cr-DR-S1")), action=(ct_snat(172.16.1.2);) table=??(lr_out_snat ), priority=163 , match=(ip && ip4.src == 50.0.0.11 && outport == "DR-S1" && is_chassis_resident("cr-DR-S1") && ip4.dst == $disallowed_range), action=(next;) ]) @@ -5144,6 +5140,9 @@ ovn-nbctl lsp-add public public-lr0 -- set Logical_Switch_Port public-lr0 \ type=router options:router-port=lr0-public \ -- lsp-set-addresses public-lr0 router +# Common zone for DGP + +check ovn-nbctl set nb_global . options:use_common_zone="true" check ovn-nbctl --wait=sb sync ovn-sbctl dump-flows lr0 > lr0flows @@ -5196,6 +5195,51 @@ AT_CHECK([grep "lr_out_snat" lr0flows | sed 's/table=./table=?/' | sort], [0], [ table=? (lr_out_snat ), priority=162 , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && reg9[[4]] == 1), action=(reg9[[4]] = 0; ct_snat(172.168.0.20);) ]) +# Separate zones for DGP + +check ovn-nbctl remove nb_global . options use_common_zone +check ovn-nbctl --wait=sb sync + +ovn-sbctl dump-flows lr0 > lr0flows +AT_CAPTURE_FILE([lr0flows]) + +AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) + table=4 (lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.10 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.30 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;) +]) + +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) +]) + +AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl + table=7 (lr_in_dnat ), priority=0 , match=(1), action=(next;) + table=7 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);) +]) + +AT_CHECK([grep "lr_out_chk_dnat_local" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl + table=? (lr_out_chk_dnat_local), priority=0 , match=(1), action=(reg9[[4]] = 0; next;) +]) + +AT_CHECK([grep "lr_out_undnat" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl + table=? (lr_out_undnat ), priority=0 , match=(1), action=(next;) + table=? (lr_out_undnat ), priority=100 , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;) +]) + +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl + table=? (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) +]) + +AT_CHECK([grep "lr_out_snat" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl + table=? (lr_out_snat ), priority=0 , match=(1), action=(next;) + table=? (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) + table=? (lr_out_snat ), priority=153 , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat(172.168.0.10);) + table=? (lr_out_snat ), priority=161 , match=(ip && ip4.src == 10.0.0.10 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat(172.168.0.30);) + table=? (lr_out_snat ), priority=161 , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat(172.168.0.20);) +]) + # Associate load balancer to lr0 check ovn-nbctl lb-add lb0 172.168.0.100:8082 "10.0.0.50:82,10.0.0.60:82" @@ -5207,6 +5251,10 @@ check ovn-nbctl lb-add lb2 172.168.0.210:60 "10.0.0.50:6062,10.0.0.60:6062" udp check ovn-nbctl lr-lb-add lr0 lb0 check ovn-nbctl lr-lb-add lr0 lb1 check ovn-nbctl lr-lb-add lr0 lb2 + +# Common zone for DGP + +check ovn-nbctl set nb_global . options:use_common_zone="true" check ovn-nbctl --wait=sb sync ovn-sbctl dump-flows lr0 > lr0flows @@ -5256,10 +5304,10 @@ AT_CHECK([grep "lr_out_chk_dnat_local" lr0flows | sed 's/table=./table=?/' | sor AT_CHECK([grep "lr_out_undnat" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl table=? (lr_out_undnat ), priority=0 , match=(1), action=(next;) table=? (lr_out_undnat ), priority=100 , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat_in_czone;) - table=? (lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src == 10.0.0.4 && tcp.src == 8080)) && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat_in_czone;) - table=? (lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src == 10.0.0.50 && tcp.src == 82) || (ip4.src == 10.0.0.60 && tcp.src == 82)) && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat_in_czone;) - table=? (lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src == 10.0.0.50 && udp.src == 6062) || (ip4.src == 10.0.0.60 && udp.src == 6062)) && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat_in_czone;) - table=? (lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src == 10.0.0.80) || (ip4.src == 10.0.0.81)) && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat_in_czone;) + table=? (lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src == 10.0.0.4 && tcp.src == 8080)) && (inport == "lr0-public" || outport == "lr0-public") && is_chassis_resident("cr-lr0-public")), action=(ct_dnat_in_czone;) + table=? (lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src == 10.0.0.50 && tcp.src == 82) || (ip4.src == 10.0.0.60 && tcp.src == 82)) && (inport == "lr0-public" || outport == "lr0-public") && is_chassis_resident("cr-lr0-public")), action=(ct_dnat_in_czone;) + table=? (lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src == 10.0.0.50 && udp.src == 6062) || (ip4.src == 10.0.0.60 && udp.src == 6062)) && (inport == "lr0-public" || outport == "lr0-public") && is_chassis_resident("cr-lr0-public")), action=(ct_dnat_in_czone;) + table=? (lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src == 10.0.0.80) || (ip4.src == 10.0.0.81)) && (inport == "lr0-public" || outport == "lr0-public") && is_chassis_resident("cr-lr0-public")), action=(ct_dnat_in_czone;) ]) AT_CHECK([grep "lr_out_post_undnat" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl @@ -5277,6 +5325,69 @@ AT_CHECK([grep "lr_out_snat" lr0flows | sed 's/table=./table=?/' | sort], [0], [ table=? (lr_out_snat ), priority=162 , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && reg9[[4]] == 1), action=(reg9[[4]] = 0; ct_snat(172.168.0.20);) ]) +# Separate zones for DGP + +check ovn-nbctl remove nb_global . options use_common_zone +check ovn-nbctl --wait=sb sync + +ovn-sbctl dump-flows lr0 > lr0flows +AT_CAPTURE_FILE([lr0flows]) + +AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) + table=4 (lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.10 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.30 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;) +]) + +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 10.0.0.10), action=(ct_dnat;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.100), action=(ct_dnat;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.200), action=(ct_dnat;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.210), action=(ct_dnat;) +]) + +AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl + table=7 (lr_in_dnat ), priority=0 , match=(1), action=(next;) + table=7 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);) + table=7 (lr_in_dnat ), priority=110 , match=(ct.new && !ct.rel && ip4 && ip4.dst == 172.168.0.200 && is_chassis_resident("cr-lr0-public")), action=(ct_lb_mark(backends=10.0.0.80,10.0.0.81);) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && !ct.rel && ip4 && ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80 && is_chassis_resident("cr-lr0-public")), action=(ct_lb_mark(backends=10.0.0.4:8080);) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && !ct.rel && ip4 && ip4.dst == 172.168.0.100 && tcp && tcp.dst == 8082 && is_chassis_resident("cr-lr0-public")), action=(ct_lb_mark(backends=10.0.0.50:82,10.0.0.60:82);) + table=7 (lr_in_dnat ), priority=120 , match=(ct.new && !ct.rel && ip4 && ip4.dst == 172.168.0.210 && udp && udp.dst == 60 && is_chassis_resident("cr-lr0-public")), action=(ct_lb_mark(backends=10.0.0.50:6062,10.0.0.60:6062);) + table=7 (lr_in_dnat ), priority=50 , match=(ct.est && !ct.rel && !ct.new && ct_mark.natted), action=(next;) + table=7 (lr_in_dnat ), priority=50 , match=(ct.rel && !ct.est && !ct.new), action=(ct_commit_nat;) + table=7 (lr_in_dnat ), priority=70 , match=(ct.est && !ct.rel && !ct.new && ct_mark.natted && ct_mark.force_snat == 1), action=(flags.force_snat_for_lb = 1; next;) + table=7 (lr_in_dnat ), priority=70 , match=(ct.est && !ct.rel && !ct.new && ct_mark.natted && ct_mark.skip_snat == 1), action=(flags.skip_snat_for_lb = 1; next;) + table=7 (lr_in_dnat ), priority=70 , match=(ct.rel && !ct.est && !ct.new && ct_mark.force_snat == 1), action=(flags.force_snat_for_lb = 1; ct_commit_nat;) + table=7 (lr_in_dnat ), priority=70 , match=(ct.rel && !ct.est && !ct.new && ct_mark.skip_snat == 1), action=(flags.skip_snat_for_lb = 1; ct_commit_nat;) +]) + +AT_CHECK([grep "lr_out_chk_dnat_local" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl + table=? (lr_out_chk_dnat_local), priority=0 , match=(1), action=(reg9[[4]] = 0; next;) +]) + +AT_CHECK([grep "lr_out_undnat" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl + table=? (lr_out_undnat ), priority=0 , match=(1), action=(next;) + table=? (lr_out_undnat ), priority=100 , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;) + table=? (lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src == 10.0.0.4 && tcp.src == 8080)) && (inport == "lr0-public" || outport == "lr0-public") && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;) + table=? (lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src == 10.0.0.50 && tcp.src == 82) || (ip4.src == 10.0.0.60 && tcp.src == 82)) && (inport == "lr0-public" || outport == "lr0-public") && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;) + table=? (lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src == 10.0.0.50 && udp.src == 6062) || (ip4.src == 10.0.0.60 && udp.src == 6062)) && (inport == "lr0-public" || outport == "lr0-public") && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;) + table=? (lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src == 10.0.0.80) || (ip4.src == 10.0.0.81)) && (inport == "lr0-public" || outport == "lr0-public") && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;) +]) + +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl + table=? (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) +]) + +AT_CHECK([grep "lr_out_snat" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl + table=? (lr_out_snat ), priority=0 , match=(1), action=(next;) + table=? (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) + table=? (lr_out_snat ), priority=153 , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat(172.168.0.10);) + table=? (lr_out_snat ), priority=161 , match=(ip && ip4.src == 10.0.0.10 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat(172.168.0.30);) + table=? (lr_out_snat ), priority=161 , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat(172.168.0.20);) +]) + # Make the logical router as Gateway router check ovn-nbctl clear logical_router_port lr0-public gateway_chassis check ovn-nbctl set logical_router lr0 options:chassis=gw1 @@ -5318,7 +5429,6 @@ AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl AT_CHECK([grep "lr_out_chk_dnat_local" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl table=? (lr_out_chk_dnat_local), priority=0 , match=(1), action=(reg9[[4]] = 0; next;) - table=? (lr_out_chk_dnat_local), priority=50 , match=(ip && ct_mark.natted == 1), action=(reg9[[4]] = 1; next;) ]) AT_CHECK([grep "lr_out_undnat" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl @@ -5382,7 +5492,6 @@ AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl AT_CHECK([grep "lr_out_chk_dnat_local" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl table=? (lr_out_chk_dnat_local), priority=0 , match=(1), action=(reg9[[4]] = 0; next;) - table=? (lr_out_chk_dnat_local), priority=50 , match=(ip && ct_mark.natted == 1), action=(reg9[[4]] = 1; next;) ]) AT_CHECK([grep "lr_out_undnat" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl @@ -5450,7 +5559,6 @@ AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl AT_CHECK([grep "lr_out_chk_dnat_local" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl table=? (lr_out_chk_dnat_local), priority=0 , match=(1), action=(reg9[[4]] = 0; next;) - table=? (lr_out_chk_dnat_local), priority=50 , match=(ip && ct_mark.natted == 1), action=(reg9[[4]] = 1; next;) ]) AT_CHECK([grep "lr_out_undnat" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl @@ -5530,7 +5638,6 @@ AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl AT_CHECK([grep "lr_out_chk_dnat_local" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl table=? (lr_out_chk_dnat_local), priority=0 , match=(1), action=(reg9[[4]] = 0; next;) - table=? (lr_out_chk_dnat_local), priority=50 , match=(ip && ct_mark.natted == 1), action=(reg9[[4]] = 1; next;) ]) AT_CHECK([grep "lr_out_undnat" lr0flows | sed 's/table=./table=?/' | sort], [0], [dnl @@ -7056,21 +7163,15 @@ AT_CHECK([grep lr_in_ip_input lrflows | grep arp | grep -e 172.16.1.10 -e 10.0.0 ]) AT_CHECK([grep lr_in_unsnat lrflows | grep ct_snat | sed 's/table=../table=??/' | sort], [0], [dnl - table=??(lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && inport == "DR-S2" && flags.loopback == 0 && is_chassis_resident("cr-DR-S2")), action=(ct_snat_in_czone;) - table=??(lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && inport == "DR-S2" && flags.loopback == 1 && flags.use_snat_zone == 1 && is_chassis_resident("cr-DR-S2")), action=(ct_snat;) - table=??(lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 172.16.1.10 && inport == "DR-S1" && flags.loopback == 0 && is_chassis_resident("cr-DR-S1")), action=(ct_snat_in_czone;) - table=??(lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 172.16.1.10 && inport == "DR-S1" && flags.loopback == 1 && flags.use_snat_zone == 1 && is_chassis_resident("cr-DR-S1")), action=(ct_snat;) - table=??(lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 192.168.0.10 && inport == "DR-S3" && flags.loopback == 0 && is_chassis_resident("cr-DR-S3")), action=(ct_snat_in_czone;) - table=??(lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 192.168.0.10 && inport == "DR-S3" && flags.loopback == 1 && flags.use_snat_zone == 1 && is_chassis_resident("cr-DR-S3")), action=(ct_snat;) + table=??(lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && inport == "DR-S2" && is_chassis_resident("cr-DR-S2")), action=(ct_snat;) + table=??(lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 172.16.1.10 && inport == "DR-S1" && is_chassis_resident("cr-DR-S1")), action=(ct_snat;) + table=??(lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 192.168.0.10 && inport == "DR-S3" && is_chassis_resident("cr-DR-S3")), action=(ct_snat;) ]) AT_CHECK([grep lr_out_snat lrflows | grep ct_snat | sed 's/table=../table=??/' | sort], [0], [dnl - table=??(lr_out_snat ), priority=161 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S1" && is_chassis_resident("cr-DR-S1")), action=(ct_snat_in_czone(172.16.1.10);) - table=??(lr_out_snat ), priority=161 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S2" && is_chassis_resident("cr-DR-S2")), action=(ct_snat_in_czone(10.0.0.10);) - table=??(lr_out_snat ), priority=161 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S3" && is_chassis_resident("cr-DR-S3")), action=(ct_snat_in_czone(192.168.0.10);) - table=??(lr_out_snat ), priority=162 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S1" && is_chassis_resident("cr-DR-S1") && reg9[[4]] == 1), action=(reg9[[4]] = 0; ct_snat(172.16.1.10);) - table=??(lr_out_snat ), priority=162 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S2" && is_chassis_resident("cr-DR-S2") && reg9[[4]] == 1), action=(reg9[[4]] = 0; ct_snat(10.0.0.10);) - table=??(lr_out_snat ), priority=162 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S3" && is_chassis_resident("cr-DR-S3") && reg9[[4]] == 1), action=(reg9[[4]] = 0; ct_snat(192.168.0.10);) + table=??(lr_out_snat ), priority=161 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S1" && is_chassis_resident("cr-DR-S1")), action=(ct_snat(172.16.1.10);) + table=??(lr_out_snat ), priority=161 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S2" && is_chassis_resident("cr-DR-S2")), action=(ct_snat(10.0.0.10);) + table=??(lr_out_snat ), priority=161 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S3" && is_chassis_resident("cr-DR-S3")), action=(ct_snat(192.168.0.10);) ]) check ovn-nbctl --wait=sb lr-nat-del DR snat 20.0.0.10 @@ -7099,15 +7200,15 @@ AT_CHECK([grep lr_in_ip_input lrflows | grep arp | grep -e 172.16.1.10 -e 10.0.0 ]) AT_CHECK([grep lr_in_dnat lrflows | grep ct_dnat | sed 's/table=../table=??/' | sort], [0], [dnl - table=??(lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && inport == "DR-S2" && is_chassis_resident("cr-DR-S2")), action=(ct_dnat_in_czone(20.0.0.10);) - table=??(lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.16.1.10 && inport == "DR-S1" && is_chassis_resident("cr-DR-S1")), action=(ct_dnat_in_czone(20.0.0.10);) - table=??(lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.16.1.10 && inport == "DR-S3" && is_chassis_resident("cr-DR-S3")), action=(ct_dnat_in_czone(20.0.0.10);) + table=??(lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && inport == "DR-S2" && is_chassis_resident("cr-DR-S2")), action=(ct_dnat(20.0.0.10);) + table=??(lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.16.1.10 && inport == "DR-S1" && is_chassis_resident("cr-DR-S1")), action=(ct_dnat(20.0.0.10);) + table=??(lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.16.1.10 && inport == "DR-S3" && is_chassis_resident("cr-DR-S3")), action=(ct_dnat(20.0.0.10);) ]) AT_CHECK([grep lr_out_undnat lrflows | grep ct_dnat | sed 's/table=../table=??/' | sort], [0], [dnl - table=??(lr_out_undnat ), priority=100 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S1" && is_chassis_resident("cr-DR-S1")), action=(ct_dnat_in_czone;) - table=??(lr_out_undnat ), priority=100 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S2" && is_chassis_resident("cr-DR-S2")), action=(ct_dnat_in_czone;) - table=??(lr_out_undnat ), priority=100 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S3" && is_chassis_resident("cr-DR-S3")), action=(ct_dnat_in_czone;) + table=??(lr_out_undnat ), priority=100 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S1" && is_chassis_resident("cr-DR-S1")), action=(ct_dnat;) + table=??(lr_out_undnat ), priority=100 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S2" && is_chassis_resident("cr-DR-S2")), action=(ct_dnat;) + table=??(lr_out_undnat ), priority=100 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S3" && is_chassis_resident("cr-DR-S3")), action=(ct_dnat;) ]) check ovn-nbctl --wait=sb lr-nat-del DR dnat @@ -7138,33 +7239,27 @@ AT_CHECK([grep lr_in_ip_input lrflows | grep arp | grep -e 172.16.1.10 -e 10.0.0 ]) AT_CHECK([grep lr_in_unsnat lrflows | grep ct_snat | sed 's/table=../table=??/' | sort], [0], [dnl - table=??(lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && inport == "DR-S2" && flags.loopback == 0 && is_chassis_resident("cr-DR-S2")), action=(ct_snat_in_czone;) - table=??(lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && inport == "DR-S2" && flags.loopback == 1 && flags.use_snat_zone == 1 && is_chassis_resident("cr-DR-S2")), action=(ct_snat;) - table=??(lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 172.16.1.10 && inport == "DR-S1" && flags.loopback == 0 && is_chassis_resident("cr-DR-S1")), action=(ct_snat_in_czone;) - table=??(lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 172.16.1.10 && inport == "DR-S1" && flags.loopback == 1 && flags.use_snat_zone == 1 && is_chassis_resident("cr-DR-S1")), action=(ct_snat;) - table=??(lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 192.168.0.10 && inport == "DR-S3" && flags.loopback == 0 && is_chassis_resident("cr-DR-S3")), action=(ct_snat_in_czone;) - table=??(lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 192.168.0.10 && inport == "DR-S3" && flags.loopback == 1 && flags.use_snat_zone == 1 && is_chassis_resident("cr-DR-S3")), action=(ct_snat;) + table=??(lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && inport == "DR-S2" && is_chassis_resident("cr-DR-S2")), action=(ct_snat;) + table=??(lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 172.16.1.10 && inport == "DR-S1" && is_chassis_resident("cr-DR-S1")), action=(ct_snat;) + table=??(lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 192.168.0.10 && inport == "DR-S3" && is_chassis_resident("cr-DR-S3")), action=(ct_snat;) ]) AT_CHECK([grep lr_out_snat lrflows | grep ct_snat | sed 's/table=../table=??/' | sort], [0], [dnl - table=??(lr_out_snat ), priority=161 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S1" && is_chassis_resident("cr-DR-S1")), action=(ct_snat_in_czone(172.16.1.10);) - table=??(lr_out_snat ), priority=161 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S2" && is_chassis_resident("cr-DR-S2")), action=(ct_snat_in_czone(10.0.0.10);) - table=??(lr_out_snat ), priority=161 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S3" && is_chassis_resident("cr-DR-S3")), action=(ct_snat_in_czone(192.168.0.10);) - table=??(lr_out_snat ), priority=162 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S1" && is_chassis_resident("cr-DR-S1") && reg9[[4]] == 1), action=(reg9[[4]] = 0; ct_snat(172.16.1.10);) - table=??(lr_out_snat ), priority=162 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S2" && is_chassis_resident("cr-DR-S2") && reg9[[4]] == 1), action=(reg9[[4]] = 0; ct_snat(10.0.0.10);) - table=??(lr_out_snat ), priority=162 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S3" && is_chassis_resident("cr-DR-S3") && reg9[[4]] == 1), action=(reg9[[4]] = 0; ct_snat(192.168.0.10);) + table=??(lr_out_snat ), priority=161 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S1" && is_chassis_resident("cr-DR-S1")), action=(ct_snat(172.16.1.10);) + table=??(lr_out_snat ), priority=161 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S2" && is_chassis_resident("cr-DR-S2")), action=(ct_snat(10.0.0.10);) + table=??(lr_out_snat ), priority=161 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S3" && is_chassis_resident("cr-DR-S3")), action=(ct_snat(192.168.0.10);) ]) AT_CHECK([grep lr_in_dnat lrflows | grep ct_dnat | sed 's/table=../table=??/' | sort], [0], [dnl - table=??(lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && inport == "DR-S2" && is_chassis_resident("cr-DR-S2")), action=(ct_dnat_in_czone(20.0.0.10);) - table=??(lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.16.1.10 && inport == "DR-S1" && is_chassis_resident("cr-DR-S1")), action=(ct_dnat_in_czone(20.0.0.10);) - table=??(lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 192.168.0.10 && inport == "DR-S3" && is_chassis_resident("cr-DR-S3")), action=(ct_dnat_in_czone(20.0.0.10);) + table=??(lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && inport == "DR-S2" && is_chassis_resident("cr-DR-S2")), action=(ct_dnat(20.0.0.10);) + table=??(lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.16.1.10 && inport == "DR-S1" && is_chassis_resident("cr-DR-S1")), action=(ct_dnat(20.0.0.10);) + table=??(lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 192.168.0.10 && inport == "DR-S3" && is_chassis_resident("cr-DR-S3")), action=(ct_dnat(20.0.0.10);) ]) AT_CHECK([grep lr_out_undnat lrflows | grep ct_dnat | sed 's/table=../table=??/' | sort], [0], [dnl - table=??(lr_out_undnat ), priority=100 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S1" && is_chassis_resident("cr-DR-S1")), action=(ct_dnat_in_czone;) - table=??(lr_out_undnat ), priority=100 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S2" && is_chassis_resident("cr-DR-S2")), action=(ct_dnat_in_czone;) - table=??(lr_out_undnat ), priority=100 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S3" && is_chassis_resident("cr-DR-S3")), action=(ct_dnat_in_czone;) + table=??(lr_out_undnat ), priority=100 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S1" && is_chassis_resident("cr-DR-S1")), action=(ct_dnat;) + table=??(lr_out_undnat ), priority=100 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S2" && is_chassis_resident("cr-DR-S2")), action=(ct_dnat;) + table=??(lr_out_undnat ), priority=100 , match=(ip && ip4.src == 20.0.0.10 && outport == "DR-S3" && is_chassis_resident("cr-DR-S3")), action=(ct_dnat;) ]) check ovn-nbctl --wait=sb lr-nat-del DR dnat_and_snat diff --git a/tests/ovn.at b/tests/ovn.at index a892691ca..a56e32f55 100644 --- a/tests/ovn.at +++ b/tests/ovn.at @@ -33566,6 +33566,9 @@ check ovn-nbctl lrp-set-gateway-chassis lr0-ext hv1 check ovn-nbctl lr-nat-add lr0 snat 172.16.0.2 10.0.0.0/24 check ovn-nbctl lr-nat-add lr0 dnat 172.16.0.2 10.0.0.2 +# Use common zone +check ovn-nbctl set nb_global . options:use_common_zone="true" + check ovn-nbctl --wait=hv sync # Use constants so that if tables or registers change, this test can # be updated easily. diff --git a/tests/system-ovn-kmod.at b/tests/system-ovn-kmod.at index c1272d5ef..981cc598c 100644 --- a/tests/system-ovn-kmod.at +++ b/tests/system-ovn-kmod.at @@ -928,6 +928,172 @@ EOF OVS_WAIT_UNTIL([test "$(cat server.pcap | wc -l)" = "4"]) +OVS_APP_EXIT_AND_WAIT([ovn-controller]) + +as ovn-sb +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) + +as ovn-nb +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) + +as northd +OVS_APP_EXIT_AND_WAIT([NORTHD_TYPE]) + +as +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d +/connection dropped.*/d"]) +AT_CLEANUP +]) + +OVN_FOR_EACH_NORTHD([ +AT_SETUP([SNAT in separate zone from DNAT]) + +AT_SKIP_IF([test $HAVE_NC = no]) +CHECK_CONNTRACK() +CHECK_CONNTRACK_NAT() +ovn_start +OVS_TRAFFIC_VSWITCHD_START() +ADD_BR([br-int]) + +# Set external-ids in br-int needed for ovn-controller +ovs-vsctl \ + -- set Open_vSwitch . external-ids:system-id=hv1 \ + -- set Open_vSwitch . external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \ + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \ + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \ + -- set bridge br-int fail-mode=secure other-config:disable-in-band=true + +# The goal of this test is to ensure that when traffic is first DNATted +# (by way of a load balancer), and then SNATted, the SNAT happens in a +# separate conntrack zone from the DNAT. + +start_daemon ovn-controller + +check ovn-nbctl ls-add public + +check ovn-nbctl lr-add r1 +check ovn-nbctl lrp-add r1 r1_public 00:de:ad:ff:00:01 172.16.0.1/16 +check ovn-nbctl lrp-add r1 r1_s1 00:de:ad:fe:00:01 173.0.1.1/24 +check ovn-nbctl lrp-set-gateway-chassis r1_public hv1 + +check ovn-nbctl lb-add r1_lb 30.0.0.1 172.16.0.102 +check ovn-nbctl lr-lb-add r1 r1_lb + +check ovn-nbctl ls-add s1 +check ovn-nbctl lsp-add s1 s1_r1 +check ovn-nbctl lsp-set-type s1_r1 router +check ovn-nbctl lsp-set-addresses s1_r1 router +check ovn-nbctl lsp-set-options s1_r1 router-port=r1_s1 + +check ovn-nbctl lsp-add s1 vm1 +check ovn-nbctl lsp-set-addresses vm1 "00:de:ad:01:00:01 173.0.1.2" + +check ovn-nbctl lsp-add public public_r1 +check ovn-nbctl lsp-set-type public_r1 router +check ovn-nbctl lsp-set-addresses public_r1 router +check ovn-nbctl lsp-set-options public_r1 router-port=r1_public nat-addresses=router + +check ovn-nbctl lr-add r2 +check ovn-nbctl lrp-add r2 r2_public 00:de:ad:ff:00:02 172.16.0.2/16 +check ovn-nbctl lrp-add r2 r2_s2 00:de:ad:fe:00:02 173.0.2.1/24 +check ovn-nbctl lr-nat-add r2 dnat_and_snat 172.16.0.102 173.0.2.2 +check ovn-nbctl lrp-set-gateway-chassis r2_public hv1 + +check ovn-nbctl ls-add s2 +check ovn-nbctl lsp-add s2 s2_r2 +check ovn-nbctl lsp-set-type s2_r2 router +check ovn-nbctl lsp-set-addresses s2_r2 router +check ovn-nbctl lsp-set-options s2_r2 router-port=r2_s2 + +check ovn-nbctl lsp-add s2 vm2 +check ovn-nbctl lsp-set-addresses vm2 "00:de:ad:01:00:02 173.0.2.2" + +check ovn-nbctl lsp-add public public_r2 +check ovn-nbctl lsp-set-type public_r2 router +check ovn-nbctl lsp-set-addresses public_r2 router +check ovn-nbctl lsp-set-options public_r2 router-port=r2_public nat-addresses=router + +ADD_NAMESPACES(vm1) +ADD_VETH(vm1, vm1, br-int, "173.0.1.2/24", "00:de:ad:01:00:01", \ + "173.0.1.1") +ADD_NAMESPACES(vm2) +ADD_VETH(vm2, vm2, br-int, "173.0.2.2/24", "00:de:ad:01:00:02", \ + "173.0.2.1") + +check ovn-nbctl lr-nat-add r1 dnat_and_snat 172.16.0.101 173.0.1.2 vm1 00:00:00:01:02:03 + +wait_for_ports_up +check ovn-nbctl --wait=hv sync + +# Create service that listens for TCP and UDP +NETNS_DAEMONIZE([vm2], [nc -l -u 1234], [nc0.pid]) +NETNS_DAEMONIZE([vm2], [nc -l -k 1235], [nc1.pid]) + +test_icmp() { + # Make sure that a ping works as expected + NS_CHECK_EXEC([vm1], [ping -c 3 -i 0.3 -w 2 30.0.0.1 | FORMAT_PING], \ + [0], [dnl +3 packets transmitted, 3 received, 0% packet loss, time 0ms +]) + + # Finally, make sure that conntrack shows two separate zones being used for + # DNAT and SNAT + AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(30.0.0.1) | \ + sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl +icmp,orig=(src=173.0.1.2,dst=30.0.0.1,id=<cleared>,type=8,code=0),reply=(src=172.16.0.102,dst=173.0.1.2,id=<cleared>,type=0,code=0),zone=<cleared>,mark=2 +]) + + AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.0.102) | \ + sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl +icmp,orig=(src=172.16.0.101,dst=172.16.0.102,id=<cleared>,type=8,code=0),reply=(src=173.0.2.2,dst=172.16.0.101,id=<cleared>,type=0,code=0),zone=<cleared> +icmp,orig=(src=173.0.1.2,dst=172.16.0.102,id=<cleared>,type=8,code=0),reply=(src=172.16.0.102,dst=172.16.0.101,id=<cleared>,type=0,code=0),zone=<cleared> +]) +} + +test_udp() { + NS_CHECK_EXEC([vm1], [nc -u 30.0.0.1 1234 -p 1222 -z]) + + AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(30.0.0.1) | \ + sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl +udp,orig=(src=173.0.1.2,dst=30.0.0.1,sport=<cleared>,dport=<cleared>),reply=(src=172.16.0.102,dst=173.0.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,mark=2 +]) + + AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.0.102) | \ + sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl +udp,orig=(src=172.16.0.101,dst=172.16.0.102,sport=<cleared>,dport=<cleared>),reply=(src=173.0.2.2,dst=172.16.0.101,sport=<cleared>,dport=<cleared>),zone=<cleared> +udp,orig=(src=173.0.1.2,dst=172.16.0.102,sport=<cleared>,dport=<cleared>),reply=(src=172.16.0.102,dst=172.16.0.101,sport=<cleared>,dport=<cleared>),zone=<cleared> +]) +} + +test_tcp() { + NS_CHECK_EXEC([vm1], [nc 30.0.0.1 1235 -z]) + + AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(30.0.0.1) | \ + sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl +tcp,orig=(src=173.0.1.2,dst=30.0.0.1,sport=<cleared>,dport=<cleared>),reply=(src=172.16.0.102,dst=173.0.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,mark=2,protoinfo=(state=<cleared>) +]) + + AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.0.102) | \ + sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl +tcp,orig=(src=172.16.0.101,dst=172.16.0.102,sport=<cleared>,dport=<cleared>),reply=(src=173.0.2.2,dst=172.16.0.101,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>) +tcp,orig=(src=173.0.1.2,dst=172.16.0.102,sport=<cleared>,dport=<cleared>),reply=(src=172.16.0.102,dst=172.16.0.101,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>) +]) +} + +for type in icmp udp tcp; do + AS_BOX([Testing $type]) + # First time, when the packet needs to pass through pinctrl buffering + check ovs-appctl dpctl/flush-conntrack + ovn-sbctl --all destroy mac_binding + wait_row_count mac_binding 0 + test_$type + + # Second time with MAC binding being already set + check ovs-appctl dpctl/flush-conntrack + wait_row_count mac_binding 1 ip="172.16.0.102" + test_$type +done + OVS_APP_EXIT_AND_WAIT([ovn-controller]) as ovn-sb diff --git a/tests/system-ovn.at b/tests/system-ovn.at index 939c2c3dc..5ae36daf1 100644 --- a/tests/system-ovn.at +++ b/tests/system-ovn.at @@ -8699,123 +8699,6 @@ OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d AT_CLEANUP ]) -OVN_FOR_EACH_NORTHD([ -AT_SETUP([SNAT in separate zone from DNAT]) - -CHECK_CONNTRACK() -CHECK_CONNTRACK_NAT() -ovn_start -OVS_TRAFFIC_VSWITCHD_START() -ADD_BR([br-int]) - -# Set external-ids in br-int needed for ovn-controller -ovs-vsctl \ - -- set Open_vSwitch . external-ids:system-id=hv1 \ - -- set Open_vSwitch . external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \ - -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \ - -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \ - -- set bridge br-int fail-mode=secure other-config:disable-in-band=true - -# The goal of this test is to ensure that when traffic is first DNATted -# (by way of a load balancer), and then SNATted, the SNAT happens in a -# separate conntrack zone from the DNAT. - -start_daemon ovn-controller - -check ovn-nbctl ls-add public - -check ovn-nbctl lr-add r1 -check ovn-nbctl lrp-add r1 r1_public 00:de:ad:ff:00:01 172.16.0.1/16 -check ovn-nbctl lrp-add r1 r1_s1 00:de:ad:fe:00:01 173.0.1.1/24 -check ovn-nbctl lrp-set-gateway-chassis r1_public hv1 - -check ovn-nbctl lb-add r1_lb 30.0.0.1 172.16.0.102 -check ovn-nbctl lr-lb-add r1 r1_lb - -check ovn-nbctl ls-add s1 -check ovn-nbctl lsp-add s1 s1_r1 -check ovn-nbctl lsp-set-type s1_r1 router -check ovn-nbctl lsp-set-addresses s1_r1 router -check ovn-nbctl lsp-set-options s1_r1 router-port=r1_s1 - -check ovn-nbctl lsp-add s1 vm1 -check ovn-nbctl lsp-set-addresses vm1 "00:de:ad:01:00:01 173.0.1.2" - -check ovn-nbctl lsp-add public public_r1 -check ovn-nbctl lsp-set-type public_r1 router -check ovn-nbctl lsp-set-addresses public_r1 router -check ovn-nbctl lsp-set-options public_r1 router-port=r1_public nat-addresses=router - -check ovn-nbctl lr-add r2 -check ovn-nbctl lrp-add r2 r2_public 00:de:ad:ff:00:02 172.16.0.2/16 -check ovn-nbctl lrp-add r2 r2_s2 00:de:ad:fe:00:02 173.0.2.1/24 -check ovn-nbctl lr-nat-add r2 dnat_and_snat 172.16.0.102 173.0.2.2 -check ovn-nbctl lrp-set-gateway-chassis r2_public hv1 - -check ovn-nbctl ls-add s2 -check ovn-nbctl lsp-add s2 s2_r2 -check ovn-nbctl lsp-set-type s2_r2 router -check ovn-nbctl lsp-set-addresses s2_r2 router -check ovn-nbctl lsp-set-options s2_r2 router-port=r2_s2 - -check ovn-nbctl lsp-add s2 vm2 -check ovn-nbctl lsp-set-addresses vm2 "00:de:ad:01:00:02 173.0.2.2" - -check ovn-nbctl lsp-add public public_r2 -check ovn-nbctl lsp-set-type public_r2 router -check ovn-nbctl lsp-set-addresses public_r2 router -check ovn-nbctl lsp-set-options public_r2 router-port=r2_public nat-addresses=router - -ADD_NAMESPACES(vm1) -ADD_VETH(vm1, vm1, br-int, "173.0.1.2/24", "00:de:ad:01:00:01", \ - "173.0.1.1") -ADD_NAMESPACES(vm2) -ADD_VETH(vm2, vm2, br-int, "173.0.2.2/24", "00:de:ad:01:00:02", \ - "173.0.2.1") - -check ovn-nbctl lr-nat-add r1 dnat_and_snat 172.16.0.101 173.0.1.2 vm1 00:00:00:01:02:03 -check ovn-nbctl --wait=hv sync - -# Next, make sure that a ping works as expected -NS_CHECK_EXEC([vm1], [ping -q -c 3 -i 0.3 -w 2 30.0.0.1 | FORMAT_PING], \ -[0], [dnl -3 packets transmitted, 3 received, 0% packet loss, time 0ms -]) - -# Finally, make sure that conntrack shows two separate zones being used for -# DNAT and SNAT -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(30.0.0.1) | \ -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl -icmp,orig=(src=173.0.1.2,dst=30.0.0.1,id=<cleared>,type=8,code=0),reply=(src=172.16.0.102,dst=173.0.1.2,id=<cleared>,type=0,code=0),zone=<cleared>,mark=2 -]) - -# The final two entries appear identical here. That is because FORMAT_CT -# scrubs the zone numbers. In actuality, the zone numbers are different, -# which is why there are two entries. -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.0.102) | \ -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl -icmp,orig=(src=172.16.0.101,dst=172.16.0.102,id=<cleared>,type=8,code=0),reply=(src=173.0.2.2,dst=172.16.0.101,id=<cleared>,type=0,code=0),zone=<cleared> -icmp,orig=(src=173.0.1.2,dst=172.16.0.102,id=<cleared>,type=8,code=0),reply=(src=172.16.0.102,dst=172.16.0.101,id=<cleared>,type=0,code=0),zone=<cleared> -icmp,orig=(src=173.0.1.2,dst=172.16.0.102,id=<cleared>,type=8,code=0),reply=(src=172.16.0.102,dst=172.16.0.101,id=<cleared>,type=0,code=0),zone=<cleared> -]) - -OVS_APP_EXIT_AND_WAIT([ovn-controller]) - -as ovn-sb -OVS_APP_EXIT_AND_WAIT([ovsdb-server]) - -as ovn-nb -OVS_APP_EXIT_AND_WAIT([ovsdb-server]) - -as northd -OVS_APP_EXIT_AND_WAIT([NORTHD_TYPE]) - -as -OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d -/connection dropped.*/d"]) -AT_CLEANUP -]) - OVN_FOR_EACH_NORTHD([ AT_SETUP([LB - ICMP related traffic]) -- 2.39.2 _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
