On Fri, Mar 2, 2018 at 5:01 AM, Numan Siddique <nusid...@redhat.com> wrote:

>
>
> On Feb 28, 2018 6:12 PM, "Guru Shetty" <g...@ovn.org> wrote:
>
> This is a little more complicated for a quick review. I think it will need
> a round from me to understand the implications for gateway routers and one
> round from Numan (or someone from openstack) that use distributed gateway
> routers.
>
>
>
> Ack. I will take a look at the patch as well.
>
> Thanks
> Numan
>
>
> On 14 February 2018 at 16:42, Ben Pfaff <b...@ovn.org> wrote:
>
> > Hi Guru.  Are you willing to take a look at this patch?
> >
> > Thanks,
> >
> > Ben.
> >
> > On Fri, Feb 09, 2018 at 03:34:55PM +0800, Guoshuai Li wrote:
> > > The main application scenario of this patch is that the user flow wants
> > to
> > > different destination addresses through different external networks.
> > > This scenario requires a distributed route to be associated with
> > > multiple external network logical switches.
> > >
> > > Change l3dgw_port to l3dgw_ports in ovn_datapath,and change
> > > l3redirect_port to ovn_port. Then in a distributed router, the NAT
> > > logical flow table is generated based on the external IP lookup
> > > distributed router port, otherwise not generated. And LB is the same.
> > >
> > > When the destination address of the packet is an external IP of the NAT
> > rule,
> > > and the ingress port is not a gateway, it is necessary to route to the
> > actual
> > > outgoing port.
> > >
> > > Signed-off-by: Guoshuai Li <l...@dtdream.com>
>
>


Hi Guoshuai Li,

I am getting the below compilation errors with "-Werror" option

**********
../ovn/northd/ovn-northd.c: In function ‘build_lrouter_flows’:
../ovn/northd/ovn-northd.c:5394:29: error: declaration of ‘i’ shadows a
previous local [-Werror=shadow]
                 for (size_t i = 0; i < od->n_l3dgw_ports; i++) {
                             ^
../ovn/northd/ovn-northd.c:5139:18: note: shadowed declaration is here
         for (int i = 0; i < od->nbr->n_nat; i++) {
                  ^
../ovn/northd/ovn-northd.c:5395:38: error: declaration of ‘l3dgw_port’
shadows a previous local [-Werror=shadow]
                     struct ovn_port *l3dgw_port = od->l3dgw_ports[i];
                                      ^~~~~~~~~~
../ovn/northd/ovn-northd.c:5197:30: note: shadowed declaration is here
             struct ovn_port *l3dgw_port = find_l3dgw_port(od,
                              ^~~~~~~~~~
../ovn/northd/ovn-northd.c:5413:29: error: declaration of ‘i’ shadows a
previous local [-Werror=shadow]
                 for (size_t i = 0; i < od->n_l3dgw_ports; i++) {
                             ^
../ovn/northd/ovn-northd.c:5139:18: note: shadowed declaration is here
         for (int i = 0; i < od->nbr->n_nat; i++) {
                  ^
../ovn/northd/ovn-northd.c:5414:38: error: declaration of ‘l3dgw_port’
shadows a previous local [-Werror=shadow]
                     struct ovn_port *l3dgw_port = od->l3dgw_ports[i];
                                      ^~~~~~~~~~
../ovn/northd/ovn-northd.c:5197:30: note: shadowed declaration is here
             struct ovn_port *l3dgw_port = find_l3dgw_port(od,
*********

Thanks
Numan



> > > ---
> > >  ovn/northd/ovn-northd.8.xml |  22 +---
> > >  ovn/northd/ovn-northd.c     | 273 +++++++++++++++++++++++++-----
> > --------------
> > >  ovn/ovn-nb.xml              |  12 +-
> > >  tests/system-ovn.at         | 162 +++++++++++++++++++++++---
> > >  4 files changed, 315 insertions(+), 154 deletions(-)
> > >
> > > ---
> > >
> > > I submitted this patch long ago, I think it might be useful,
> > > so resend it for more comments, thanks.
> > >
> > > v1 -> v2:
> > >   1. rebase from master.
> > >   2. add test case.
> > > v2 -> v3:
> > >   fixed build failed.
> > >
> > > ---
> > >
> > > diff --git a/ovn/northd/ovn-northd.8.xml b/ovn/northd/ovn-northd.8.xml
> > > index 6bc2dd6af..ff523614a 100644
> > > --- a/ovn/northd/ovn-northd.8.xml
> > > +++ b/ovn/northd/ovn-northd.8.xml
> > > @@ -1732,16 +1732,6 @@ output;
> > >      <ul>
> > >        <li>
> > >          <p>
> > > -          For distributed logical routers where one of the logical
> > router
> > > -          ports specifies a <code>redirect-chassis</code>, a
> > priority-300
> > > -          logical flow with match <code>REGBIT_NAT_REDIRECT ==
> 1</code>
> > has
> > > -          actions <code>ip.ttl--; next;</code>.  The
> > <code>outport</code>
> > > -          will be set later in the Gateway Redirect table.
> > > -        </p>
> > > -      </li>
> > > -
> > > -      <li>
> > > -        <p>
> > >            IPv4 routing table.  For each route to IPv4 network
> > <var>N</var> with
> > >            netmask <var>M</var>, on router port <var>P</var> with IP
> > address
> > >            <var>A</var> and Ethernet
> > > @@ -1827,12 +1817,12 @@ next;
> > >      <ul>
> > >        <li>
> > >          <p>
> > > -          For distributed logical routers where one of the logical
> > router
> > > -          ports specifies a <code>redirect-chassis</code>, a
> > priority-200
> > > -          logical flow with match <code>REGBIT_NAT_REDIRECT ==
> 1</code>
> > has
> > > -          actions <code>eth.dst = <var>E</var>; next;</code>, where
> > > -          <var>E</var> is the ethernet address of the router's
> > distributed
> > > -          gateway port.
> > > +          For distributed logical routers where router port
> <var>P</var>
> > > +          specifies a <code>redirect-chassis</code>, a priority-200
> > > +          logical flow with match <code>REGBIT_NAT_REDIRECT ==
> 1</code>
> > > +          and outport == <var>P</var> has actions
> > > +          <code>eth.dst = <var>E</var>; next;</code>, where
> <var>E</var>
> > > +          is the ethernet address of the router's distributed gateway
> > port.
> > >          </p>
> > >        </li>
> > >
> > > diff --git a/ovn/northd/ovn-northd.c b/ovn/northd/ovn-northd.c
> > > index 4d95a3d9d..d38efcbed 100644
> > > --- a/ovn/northd/ovn-northd.c
> > > +++ b/ovn/northd/ovn-northd.c
> > > @@ -418,12 +418,10 @@ struct ovn_datapath {
> > >
> > >      /* OVN northd only needs to know about the logical router gateway
> > port for
> > >       * NAT on a distributed router.  This "distributed gateway port"
> is
> > > -     * populated only when there is a "redirect-chassis" specified for
> > one of
> > > -     * the ports on the logical router.  Otherwise this will be NULL.
> */
> > > -    struct ovn_port *l3dgw_port;
> > > -    /* The "derived" OVN port representing the instance of l3dgw_port
> on
> > > -     * the "redirect-chassis". */
> > > -    struct ovn_port *l3redirect_port;
> > > +     * populated only when there is a "redirect-chassis" specified for
> > the
> > > +     * ports on the logical router.  Otherwise this will be NULL. */
> > > +    struct ovn_port **l3dgw_ports;
> > > +    size_t n_l3dgw_ports;
> > >      struct ovn_port *localnet_port;
> > >  };
> > >
> > > @@ -472,6 +470,7 @@ ovn_datapath_destroy(struct hmap *datapaths, struct
> > ovn_datapath *od)
> > >              free(od->ipam_info);
> > >          }
> > >          free(od->router_ports);
> > > +        free(od->l3dgw_ports);
> > >          free(od);
> > >      }
> > >  }
> > > @@ -796,6 +795,10 @@ struct ovn_port {
> > >      bool derived; /* Indicates whether this is an additional port
> > >                     * derived from nbsp or nbrp. */
> > >
> > > +    /* The "derived" OVN port representing the instance of l3dgw_port
> on
> > > +     * the "redirect-chassis". Otherwise this will be NULL. */
> > > +    struct ovn_port *l3redirect_port;
> > > +
> > >      /* The port's peer:
> > >       *
> > >       *     - A switch port S of type "router" has a router port R as a
> > peer,
> > > @@ -1479,7 +1482,7 @@ join_logical_ports(struct northd_context *ctx,
> > >                                       "on L3 gateway router",
> > nbrp->name);
> > >                          continue;
> > >                      }
> > > -                    if (od->l3dgw_port || od->l3redirect_port) {
> > > +                    if (op->l3redirect_port) {
> > >                          static struct vlog_rate_limit rl
> > >                              = VLOG_RATE_LIMIT_INIT(1, 1);
> > >                          VLOG_WARN_RL(&rl, "Bad configuration: multiple
> > ports "
> > > @@ -1506,8 +1509,10 @@ join_logical_ports(struct northd_context *ctx,
> > >
> > >                      /* Set l3dgw_port and l3redirect_port in od, for
> > later
> > >                       * use during flow creation. */
> > > -                    od->l3dgw_port = op;
> > > -                    od->l3redirect_port = crp;
> > > +                    op->l3redirect_port = crp;
> > > +                    od->l3dgw_ports = xrealloc(od->l3dgw_ports,
> > > +                        sizeof *od->l3dgw_ports * (od->n_l3dgw_ports +
> > 1));
> > > +                    od->l3dgw_ports[od->n_l3dgw_ports++] = op;
> > >                  }
> > >              }
> > >          }
> > > @@ -1604,6 +1609,9 @@ get_router_load_balancer_ips(const struct
> > ovn_datapath *od,
> > >      }
> > >  }
> > >
> > > +static const char *
> > > +find_lrp_member_ip(const struct ovn_port *op, const char *ip_s);
> > > +
> > >  /* Returns an array of strings, each consisting of a MAC address
> > followed
> > >   * by one or more IP addresses, and if the port is a distributed
> gateway
> > >   * port, followed by 'is_chassis_resident("LPORT_NAME")', where the
> > > @@ -1646,7 +1654,7 @@ get_nat_addresses(const struct ovn_port *op,
> > size_t *n)
> > >
> > >          /* Determine whether this NAT rule satisfies the conditions
> for
> > >           * distributed NAT processing. */
> > > -        if (op->od->l3redirect_port && !strcmp(nat->type,
> > "dnat_and_snat")
> > > +        if (op->l3redirect_port && !strcmp(nat->type, "dnat_and_snat")
> > >              && nat->logical_port && nat->external_mac) {
> > >              /* Distributed NAT rule. */
> > >              if (eth_addr_from_string(nat->external_mac, &mac)) {
> > > @@ -1660,8 +1668,10 @@ get_nat_addresses(const struct ovn_port *op,
> > size_t *n)
> > >          } else {
> > >              /* Centralized NAT rule, either on gateway router or
> > distributed
> > >               * router. */
> > > -            ds_put_format(&c_addresses, " %s", nat->external_ip);
> > > -            central_ip_address = true;
> > > +            if (find_lrp_member_ip(op, nat->external_ip)) {
> > > +                ds_put_format(&c_addresses, " %s", nat->external_ip);
> > > +                central_ip_address = true;
> > > +            }
> > >          }
> > >      }
> > >
> > > @@ -1680,9 +1690,9 @@ get_nat_addresses(const struct ovn_port *op,
> > size_t *n)
> > >      if (central_ip_address) {
> > >          /* Gratuitous ARP for centralized NAT rules on distributed
> > gateway
> > >           * ports should be restricted to the "redirect-chassis". */
> > > -        if (op->od->l3redirect_port) {
> > > +        if (op->l3redirect_port) {
> > >              ds_put_format(&c_addresses, " is_chassis_resident(%s)",
> > > -                          op->od->l3redirect_port->json_key);
> > > +                          op->l3redirect_port->json_key);
> > >          }
> > >
> > >          addresses[n_nats++] = ds_steal_cstr(&c_addresses);
> > > @@ -2021,8 +2031,7 @@ ovn_port_update_sbrec(struct northd_context *ctx,
> > >              const char *nat_addresses = smap_get(&op->nbsp->options,
> > >                                             "nat-addresses");
> > >              if (nat_addresses && !strcmp(nat_addresses, "router")) {
> > > -                if (op->peer && op->peer->od
> > > -                    && (chassis || op->peer->od->l3redirect_port)) {
> > > +                if (op->peer && (chassis ||
> op->peer->l3redirect_port))
> > {
> > >                      size_t n_nats;
> > >                      char **nats = get_nat_addresses(op->peer,
> &n_nats);
> > >                      if (n_nats) {
> > > @@ -3983,14 +3992,12 @@ build_lswitch_flows(struct hmap *datapaths,
> > struct hmap *ports,
> > >                  ds_clear(&match);
> > >                  ds_put_format(&match, "eth.dst == "ETH_ADDR_FMT,
> > >                                ETH_ADDR_ARGS(mac));
> > > -                if (op->peer->od->l3dgw_port
> > > -                    && op->peer == op->peer->od->l3dgw_port
> > > -                    && op->peer->od->l3redirect_port) {
> > > +                if (op->peer->l3redirect_port) {
> > >                      /* The destination lookup flow for the router's
> > >                       * distributed gateway port MAC address should
> only
> > be
> > >                       * programmed on the "redirect-chassis". */
> > >                      ds_put_format(&match, " &&
> is_chassis_resident(%s)",
> > > -                                  op->peer->od->l3redirect_port-
> > >json_key);
> > > +                                  op->peer->l3redirect_port->jso
> n_key);
> > >                  }
> > >
> > >                  ds_clear(&actions);
> > > @@ -4000,8 +4007,7 @@ build_lswitch_flows(struct hmap *datapaths,
> struct
> > hmap *ports,
> > >
> > >                  /* Add ethernet addresses specified in NAT rules on
> > >                   * distributed logical routers. */
> > > -                if (op->peer->od->l3dgw_port
> > > -                    && op->peer == op->peer->od->l3dgw_port) {
> > > +                if (op->peer->l3redirect_port) {
> > >                      for (int j = 0; j < op->peer->od->nbr->n_nat;
> j++) {
> > >                          const struct nbrec_nat *nat
> > >                                                    =
> > op->peer->od->nbr->nat[j];
> > > @@ -4156,6 +4162,22 @@ find_lrp_member_ip(const struct ovn_port *op,
> > const char *ip_s)
> > >      return NULL;
> > >  }
> > >
> > > +static struct ovn_port *
> > > +find_l3dgw_port(struct ovn_datapath *od, const char *external_ip)
> > > +{
> > > +    for (size_t i = 0; i < od->n_l3dgw_ports; i++) {
> > > +        struct ovn_port *l3dgw_port = od->l3dgw_ports[i];
> > > +        if (find_lrp_member_ip(l3dgw_port, external_ip)) {
> > > +            return l3dgw_port;
> > > +        }
> > > +    }
> > > +    static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> > > +    VLOG_WARN_RL(&rl, "can not find l3dgw port with redirect-chassis "
> > > +                 "for nat, external ip %s in router "UUID_FMT"",
> > > +                 external_ip, UUID_ARGS(&od->key));
> > > +    return NULL;
> > > +}
> > > +
> > >  static void
> > >  add_route(struct hmap *lflows, const struct ovn_port *op,
> > >            const char *lrp_addr_s, const char *network_s, int plen,
> > > @@ -4410,7 +4432,8 @@ static void
> > >  add_router_lb_flow(struct hmap *lflows, struct ovn_datapath *od,
> > >                     struct ds *match, struct ds *actions, int priority,
> > >                     const char *lb_force_snat_ip, char *backend_ips,
> > > -                   bool is_udp, int addr_family)
> > > +                   bool is_udp, int addr_family,
> > > +                   const struct ovn_port *l3dgw_port)
> > >  {
> > >      /* A match and actions for new connections. */
> > >      char *new_match = xasprintf("ct.new && %s", ds_cstr(match));
> > > @@ -4438,7 +4461,7 @@ add_router_lb_flow(struct hmap *lflows, struct
> > ovn_datapath *od,
> > >      free(new_match);
> > >      free(est_match);
> > >
> > > -    if (!od->l3dgw_port || !od->l3redirect_port || !backend_ips
> > > +    if (!l3dgw_port || !l3dgw_port->l3redirect_port || !backend_ips
> > >              || addr_family != AF_INET) {
> > >          return;
> > >      }
> > > @@ -4485,8 +4508,8 @@ add_router_lb_flow(struct hmap *lflows, struct
> > ovn_datapath *od,
> > >      ds_chomp(&undnat_match, '|');
> > >      ds_chomp(&undnat_match, ' ');
> > >      ds_put_format(&undnat_match, ") && outport == %s && "
> > > -                 "is_chassis_resident(%s)", od->l3dgw_port->json_key,
> > > -                 od->l3redirect_port->json_key);
> > > +                 "is_chassis_resident(%s)", l3dgw_port->json_key,
> > > +                 l3dgw_port->l3redirect_port->json_key);
> > >      if (lb_force_snat_ip) {
> > >          ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 120,
> > >                        ds_cstr(&undnat_match),
> > > @@ -4610,12 +4633,11 @@ build_lrouter_flows(struct hmap *datapaths,
> > struct hmap *ports,
> > >          ds_clear(&match);
> > >          ds_put_format(&match, "eth.dst == %s && inport == %s",
> > >                        op->lrp_networks.ea_s, op->json_key);
> > > -        if (op->od->l3dgw_port && op == op->od->l3dgw_port
> > > -            && op->od->l3redirect_port) {
> > > +        if (op->l3redirect_port) {
> > >              /* Traffic with eth.dst = l3dgw_port->lrp_networks.ea_s
> > >               * should only be received on the "redirect-chassis". */
> > >              ds_put_format(&match, " && is_chassis_resident(%s)",
> > > -                          op->od->l3redirect_port->json_key);
> > > +                          op->l3redirect_port->json_key);
> > >          }
> > >          ovn_lflow_add(lflows, op->od, S_ROUTER_IN_ADMISSION, 50,
> > >                        ds_cstr(&match), "next;");
> > > @@ -4724,15 +4746,14 @@ build_lrouter_flows(struct hmap *datapaths,
> > struct hmap *ports,
> > >              ds_put_format(&match,
> > >                            "inport == %s && arp.tpa == %s && arp.op ==
> > 1",
> > >                            op->json_key, op->lrp_networks.ipv4_addrs[i]
> > .addr_s);
> > > -            if (op->od->l3dgw_port && op == op->od->l3dgw_port
> > > -                && op->od->l3redirect_port) {
> > > +            if (op->l3redirect_port) {
> > >                  /* Traffic with eth.src =
> l3dgw_port->lrp_networks.ea_s
> > >                   * should only be sent from the "redirect-chassis", so
> > that
> > >                   * upstream MAC learning points to the
> > "redirect-chassis".
> > >                   * Also need to avoid generation of multiple ARP
> > responses
> > >                   * from different chassis. */
> > >                  ds_put_format(&match, " && is_chassis_resident(%s)",
> > > -                              op->od->l3redirect_port->json_key);
> > > +                              op->l3redirect_port->json_key);
> > >              }
> > >
> > >              ds_clear(&actions);
> > > @@ -4865,7 +4886,7 @@ build_lrouter_flows(struct hmap *datapaths,
> struct
> > hmap *ports,
> > >                  "arp.op = 2; /* ARP reply */ "
> > >                  "arp.tha = arp.sha; ");
> > >
> > > -            if (op->od->l3dgw_port && op == op->od->l3dgw_port) {
> > > +            if (op->l3redirect_port) {
> > >                  struct eth_addr mac;
> > >                  if (nat->external_mac &&
> > >                      eth_addr_from_string(nat->external_mac, &mac)
> > > @@ -4894,10 +4915,8 @@ build_lrouter_flows(struct hmap *datapaths,
> > struct hmap *ports,
> > >                       * upstream MAC learning points to the
> > "redirect-chassis".
> > >                       * Also need to avoid generation of multiple ARP
> > responses
> > >                       * from different chassis. */
> > > -                    if (op->od->l3redirect_port) {
> > > -                        ds_put_format(&match, " &&
> > is_chassis_resident(%s)",
> > > -                                      op->od->l3redirect_port->json_
> > key);
> > > -                    }
> > > +                    ds_put_format(&match, " &&
> is_chassis_resident(%s)",
> > > +                                  op->l3redirect_port->json_key);
> > >                  }
> > >              } else {
> > >                  ds_put_format(&actions,
> > > @@ -5007,15 +5026,14 @@ build_lrouter_flows(struct hmap *datapaths,
> > struct hmap *ports,
> > >                      op->lrp_networks.ipv6_addrs[i].addr_s,
> > >                      op->lrp_networks.ipv6_addrs[i].sn_addr_s,
> > >                      op->lrp_networks.ipv6_addrs[i].addr_s);
> > > -            if (op->od->l3dgw_port && op == op->od->l3dgw_port
> > > -                && op->od->l3redirect_port) {
> > > +            if (op->l3redirect_port) {
> > >                  /* Traffic with eth.src =
> l3dgw_port->lrp_networks.ea_s
> > >                   * should only be sent from the "redirect-chassis", so
> > that
> > >                   * upstream MAC learning points to the
> > "redirect-chassis".
> > >                   * Also need to avoid generation of multiple ND
> replies
> > >                   * from different chassis. */
> > >                  ds_put_format(&match, " && is_chassis_resident(%s)",
> > > -                              op->od->l3redirect_port->json_key);
> > > +                              op->l3redirect_port->json_key);
> > >              }
> > >
> > >              ds_clear(&actions);
> > > @@ -5056,7 +5074,7 @@ build_lrouter_flows(struct hmap *datapaths,
> struct
> > hmap *ports,
> > >          /* NAT rules are only valid on Gateway routers and routers
> with
> > >           * l3dgw_port (router has a port with "redirect-chassis"
> > >           * specified). */
> > > -        if (!smap_get(&od->nbr->options, "chassis") &&
> !od->l3dgw_port)
> > {
> > > +        if (!smap_get(&od->nbr->options, "chassis") &&
> > !od->n_l3dgw_ports) {
> > >              continue;
> > >          }
> > >
> > > @@ -5066,6 +5084,8 @@ build_lrouter_flows(struct hmap *datapaths,
> struct
> > hmap *ports,
> > >          const char *lb_force_snat_ip = get_force_snat_ip(od, "lb",
> > >                                                           &snat_ip);
> > >
> > > +        struct smap nat_external_ip = SMAP_INITIALIZER(&nat_
> > external_ip);
> > > +
> > >          for (int i = 0; i < od->nbr->n_nat; i++) {
> > >              const struct nbrec_nat *nat;
> > >
> > > @@ -5110,7 +5130,7 @@ build_lrouter_flows(struct hmap *datapaths,
> struct
> > hmap *ports,
> > >               * satisfies the conditions for distributed NAT
> processing.
> > */
> > >              bool distributed = false;
> > >              struct eth_addr mac;
> > > -            if (od->l3dgw_port && !strcmp(nat->type, "dnat_and_snat")
> &&
> > > +            if (od->n_l3dgw_ports && !strcmp(nat->type,
> > "dnat_and_snat") &&
> > >                  nat->logical_port && nat->external_mac) {
> > >                  if (eth_addr_from_string(nat->external_mac, &mac)) {
> > >                      distributed = true;
> > > @@ -5123,6 +5143,13 @@ build_lrouter_flows(struct hmap *datapaths,
> > struct hmap *ports,
> > >                  }
> > >              }
> > >
> > > +            /* find l3dgw port by external ip */
> > > +            struct ovn_port *l3dgw_port = find_l3dgw_port(od,
> > > +
> > nat->external_ip);
> > > +
> > > +            bool first_add = smap_add_once(&nat_external_ip,
> > nat->external_ip,
> > > +                                           nat->external_ip);
> > > +
> > >              /* Ingress UNSNAT table: It is for already established
> > connections'
> > >               * reverse traffic. i.e., SNAT has already been done in
> > egress
> > >               * pipeline and now the packet has entered the ingress
> > pipeline as
> > > @@ -5132,16 +5159,16 @@ build_lrouter_flows(struct hmap *datapaths,
> > struct hmap *ports,
> > >               * because when the packet was DNATed in ingress pipeline,
> > it did
> > >               * not know about the possibility of eventual additional
> > SNAT in
> > >               * egress pipeline. */
> > > -            if (!strcmp(nat->type, "snat")
> > > -                || !strcmp(nat->type, "dnat_and_snat")) {
> > > -                if (!od->l3dgw_port) {
> > > +            if ((!strcmp(nat->type, "snat")
> > > +                || !strcmp(nat->type, "dnat_and_snat")) && first_add)
> {
> > > +                if (!od->n_l3dgw_ports) {
> > >                      /* Gateway router. */
> > >                      ds_clear(&match);
> > >                      ds_put_format(&match, "ip && ip4.dst == %s",
> > >                                    nat->external_ip);
> > >                      ovn_lflow_add(lflows, od, S_ROUTER_IN_UNSNAT, 90,
> > >                                    ds_cstr(&match), "ct_snat; next;");
> > > -                } else {
> > > +                } else if (l3dgw_port) {
> > >                      /* Distributed router. */
> > >
> > >                      /* Traffic received on l3dgw_port is subject to
> > NAT. */
> > > @@ -5149,12 +5176,12 @@ build_lrouter_flows(struct hmap *datapaths,
> > struct hmap *ports,
> > >                      ds_put_format(&match, "ip && ip4.dst == %s"
> > >                                            " && inport == %s",
> > >                                    nat->external_ip,
> > > -                                  od->l3dgw_port->json_key);
> > > -                    if (!distributed && od->l3redirect_port) {
> > > +                                  l3dgw_port->json_key);
> > > +                    if (!distributed && l3dgw_port->l3redirect_port) {
> > >                          /* Flows for NAT rules that are centralized
> are
> > only
> > >                           * programmed on the "redirect-chassis". */
> > >                          ds_put_format(&match, " &&
> > is_chassis_resident(%s)",
> > > -                                      od->l3redirect_port->json_key);
> > > +                                     l3dgw_port->l3redirect_port->
> > json_key);
> > >                      }
> > >                      ovn_lflow_add(lflows, od, S_ROUTER_IN_UNSNAT, 100,
> > >                                    ds_cstr(&match), "ct_snat;");
> > > @@ -5176,7 +5203,7 @@ build_lrouter_flows(struct hmap *datapaths,
> struct
> > hmap *ports,
> > >               * to a logical IP address. */
> > >              if (!strcmp(nat->type, "dnat")
> > >                  || !strcmp(nat->type, "dnat_and_snat")) {
> > > -                if (!od->l3dgw_port) {
> > > +                if (!od->n_l3dgw_ports) {
> > >                      /* Gateway router. */
> > >                      /* Packet when it goes from the initiator to
> > destination.
> > >                       * We need to set flags.loopback because the
> router
> > can
> > > @@ -5196,7 +5223,7 @@ build_lrouter_flows(struct hmap *datapaths,
> struct
> > hmap *ports,
> > >                                    nat->logical_ip);
> > >                      ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 100,
> > >                                    ds_cstr(&match), ds_cstr(&actions));
> > > -                } else {
> > > +                } else if (l3dgw_port) {
> > >                      /* Distributed router. */
> > >
> > >                      /* Traffic received on l3dgw_port is subject to
> > NAT. */
> > > @@ -5204,12 +5231,12 @@ build_lrouter_flows(struct hmap *datapaths,
> > struct hmap *ports,
> > >                      ds_put_format(&match, "ip && ip4.dst == %s"
> > >                                            " && inport == %s",
> > >                                    nat->external_ip,
> > > -                                  od->l3dgw_port->json_key);
> > > -                    if (!distributed && od->l3redirect_port) {
> > > +                                  l3dgw_port->json_key);
> > > +                    if (!distributed && l3dgw_port->l3redirect_port) {
> > >                          /* Flows for NAT rules that are centralized
> are
> > only
> > >                           * programmed on the "redirect-chassis". */
> > >                          ds_put_format(&match, " &&
> > is_chassis_resident(%s)",
> > > -                                      od->l3redirect_port->json_key);
> > > +                                     l3dgw_port->l3redirect_port->
> > json_key);
> > >                      }
> > >                      ds_clear(&actions);
> > >                      ds_put_format(&actions, "ct_dnat(%s);",
> > > @@ -5237,18 +5264,18 @@ build_lrouter_flows(struct hmap *datapaths,
> > struct hmap *ports,
> > >               * Note that this only applies for NAT on a distributed
> > router.
> > >               * Undo DNAT on a gateway router is done in the ingress
> DNAT
> > >               * pipeline stage. */
> > > -            if (od->l3dgw_port && (!strcmp(nat->type, "dnat")
> > > +            if (l3dgw_port && (!strcmp(nat->type, "dnat")
> > >                  || !strcmp(nat->type, "dnat_and_snat"))) {
> > >                  ds_clear(&match);
> > >                  ds_put_format(&match, "ip && ip4.src == %s"
> > >                                        " && outport == %s",
> > >                                nat->logical_ip,
> > > -                              od->l3dgw_port->json_key);
> > > -                if (!distributed && od->l3redirect_port) {
> > > +                              l3dgw_port->json_key);
> > > +                if (!distributed && l3dgw_port->l3redirect_port) {
> > >                      /* Flows for NAT rules that are centralized are
> only
> > >                       * programmed on the "redirect-chassis". */
> > >                      ds_put_format(&match, " &&
> is_chassis_resident(%s)",
> > > -                                  od->l3redirect_port->json_key);
> > > +                                  l3dgw_port->l3redirect_port->
> > json_key);
> > >                  }
> > >                  ds_clear(&actions);
> > >                  if (distributed) {
> > > @@ -5265,7 +5292,7 @@ build_lrouter_flows(struct hmap *datapaths,
> struct
> > hmap *ports,
> > >               * address. */
> > >              if (!strcmp(nat->type, "snat")
> > >                  || !strcmp(nat->type, "dnat_and_snat")) {
> > > -                if (!od->l3dgw_port) {
> > > +                if (!od->n_l3dgw_ports) {
> > >                      /* Gateway router. */
> > >                      ds_clear(&match);
> > >                      ds_put_format(&match, "ip && ip4.src == %s",
> > > @@ -5279,18 +5306,18 @@ build_lrouter_flows(struct hmap *datapaths,
> > struct hmap *ports,
> > >                      ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT,
> > >                                    count_1bits(ntohl(mask)) + 1,
> > >                                    ds_cstr(&match), ds_cstr(&actions));
> > > -                } else {
> > > +                } else if (l3dgw_port) {
> > >                      /* Distributed router. */
> > >                      ds_clear(&match);
> > >                      ds_put_format(&match, "ip && ip4.src == %s"
> > >                                            " && outport == %s",
> > >                                    nat->logical_ip,
> > > -                                  od->l3dgw_port->json_key);
> > > -                    if (!distributed && od->l3redirect_port) {
> > > +                                  l3dgw_port->json_key);
> > > +                    if (!distributed && l3dgw_port->l3redirect_port) {
> > >                          /* Flows for NAT rules that are centralized
> are
> > only
> > >                           * programmed on the "redirect-chassis". */
> > >                          ds_put_format(&match, " &&
> > is_chassis_resident(%s)",
> > > -                                      od->l3redirect_port->json_key);
> > > +                                     l3dgw_port->l3redirect_port->
> > json_key);
> > >                      }
> > >                      ds_clear(&actions);
> > >                      if (distributed) {
> > > @@ -5314,15 +5341,18 @@ build_lrouter_flows(struct hmap *datapaths,
> > struct hmap *ports,
> > >               * on the l3dgw_port instance where nat->logical_port is
> > >               * resident. */
> > >              if (distributed) {
> > > -                ds_clear(&match);
> > > -                ds_put_format(&match,
> > > -                              "eth.dst == "ETH_ADDR_FMT" && inport ==
> > %s"
> > > -                              " && is_chassis_resident(\"%s\")",
> > > -                              ETH_ADDR_ARGS(mac),
> > > -                              od->l3dgw_port->json_key,
> > > -                              nat->logical_port);
> > > -                ovn_lflow_add(lflows, od, S_ROUTER_IN_ADMISSION, 50,
> > > -                              ds_cstr(&match), "next;");
> > > +                for (size_t i = 0; i < od->n_l3dgw_ports; i++) {
> > > +                    struct ovn_port *l3dgw_port = od->l3dgw_ports[i];
> > > +                    ds_clear(&match);
> > > +                    ds_put_format(&match,
> > > +                                  "eth.dst == "ETH_ADDR_FMT" && inport
> > == %s"
> > > +                                  " && is_chassis_resident(\"%s\")",
> > > +                                  ETH_ADDR_ARGS(mac),
> > > +                                  l3dgw_port->json_key,
> > > +                                  nat->logical_port);
> > > +                    ovn_lflow_add(lflows, od, S_ROUTER_IN_ADMISSION,
> 50,
> > > +                                  ds_cstr(&match), "next;");
> > > +                }
> > >              }
> > >
> > >              /* Ingress Gateway Redirect Table: For NAT on a
> distributed
> > > @@ -5330,12 +5360,15 @@ build_lrouter_flows(struct hmap *datapaths,
> > struct hmap *ports,
> > >               * flows indicate the presence of an applicable NAT rule
> > that
> > >               * can be applied in a distributed manner. */
> > >              if (distributed) {
> > > -                ds_clear(&match);
> > > -                ds_put_format(&match, "ip4.src == %s && outport ==
> %s",
> > > -                              nat->logical_ip,
> > > -                              od->l3dgw_port->json_key);
> > > -                ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT,
> 100,
> > > -                              ds_cstr(&match), "next;");
> > > +                for (size_t i = 0; i < od->n_l3dgw_ports; i++) {
> > > +                    struct ovn_port *l3dgw_port = od->l3dgw_ports[i];
> > > +                    ds_clear(&match);
> > > +                    ds_put_format(&match, "ip4.src == %s && outport ==
> > %s",
> > > +                                  nat->logical_ip,
> > > +                                  l3dgw_port->json_key);
> > > +                    ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT,
> > 100,
> > > +                                  ds_cstr(&match), "next;");
> > > +                }
> > >              }
> > >
> > >              /* Egress Loopback table: For NAT on a distributed router.
> > > @@ -5343,12 +5376,12 @@ build_lrouter_flows(struct hmap *datapaths,
> > struct hmap *ports,
> > >               * gateway port have ip.dst matching a NAT external IP,
> then
> > >               * loop a clone of the packet back to the beginning of the
> > >               * ingress pipeline with inport = outport. */
> > > -            if (od->l3dgw_port) {
> > > +            if (l3dgw_port && first_add) {
> > >                  /* Distributed router. */
> > >                  ds_clear(&match);
> > >                  ds_put_format(&match, "ip4.dst == %s && outport ==
> %s",
> > >                                nat->external_ip,
> > > -                              od->l3dgw_port->json_key);
> > > +                              l3dgw_port->json_key);
> > >                  ds_clear(&actions);
> > >                  ds_put_format(&actions,
> > >                                "clone { ct_clear; "
> > > @@ -5365,7 +5398,7 @@ build_lrouter_flows(struct hmap *datapaths,
> struct
> > hmap *ports,
> > >          }
> > >
> > >          /* Handle force SNAT options set in the gateway router. */
> > > -        if (dnat_force_snat_ip && !od->l3dgw_port) {
> > > +        if (dnat_force_snat_ip && !od->n_l3dgw_ports) {
> > >              /* If a packet with destination IP address as that of the
> > >               * gateway router (as set in options:dnat_force_snat_ip)
> is
> > seen,
> > >               * UNSNAT it. */
> > > @@ -5384,7 +5417,7 @@ build_lrouter_flows(struct hmap *datapaths,
> struct
> > hmap *ports,
> > >              ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 100,
> > >                            ds_cstr(&match), ds_cstr(&actions));
> > >          }
> > > -        if (lb_force_snat_ip && !od->l3dgw_port) {
> > > +        if (lb_force_snat_ip && !od->n_l3dgw_ports) {
> > >              /* If a packet with destination IP address as that of the
> > >               * gateway router (as set in options:lb_force_snat_ip) is
> > seen,
> > >               * UNSNAT it. */
> > > @@ -5403,7 +5436,7 @@ build_lrouter_flows(struct hmap *datapaths,
> struct
> > hmap *ports,
> > >                            ds_cstr(&match), ds_cstr(&actions));
> > >          }
> > >
> > > -        if (!od->l3dgw_port) {
> > > +        if (!od->n_l3dgw_ports) {
> > >              /* For gateway router, re-circulate every packet through
> > >              * the DNAT zone.  This helps with two things.
> > >              *
> > > @@ -5422,40 +5455,38 @@ build_lrouter_flows(struct hmap *datapaths,
> > struct hmap *ports,
> > >                            "ip", "flags.loopback = 1; ct_dnat;");
> > >          } else {
> > >              /* For NAT on a distributed router, add flows to Ingress
> > > -             * IP Routing table, Ingress ARP Resolution table, and
> > > -             * Ingress Gateway Redirect Table that are not specific
> to a
> > > -             * NAT rule. */
> > > -
> > > -            /* The highest priority IN_IP_ROUTING rule matches packets
> > > -             * with REGBIT_NAT_REDIRECT (set in DNAT or UNSNAT
> stages),
> > > -             * with action "ip.ttl--; next;".  The IN_GW_REDIRECT
> table
> > > -             * will take care of setting the outport. */
> > > -            ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 300,
> > > -                          REGBIT_NAT_REDIRECT" == 1", "ip.ttl--;
> > next;");
> > > -
> > > -            /* The highest priority IN_ARP_RESOLVE rule matches
> packets
> > > -             * with REGBIT_NAT_REDIRECT (set in DNAT or UNSNAT
> stages),
> > > -             * then sets eth.dst to the distributed gateway port's
> > > -             * ethernet address. */
> > > -            ds_clear(&actions);
> > > -            ds_put_format(&actions, "eth.dst = %s; next;",
> > > -                          od->l3dgw_port->lrp_networks.ea_s);
> > > -            ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 200,
> > > -                          REGBIT_NAT_REDIRECT" == 1",
> > ds_cstr(&actions));
> > > -
> > > -            /* The highest priority IN_GW_REDIRECT rule redirects
> > packets
> > > -             * with REGBIT_NAT_REDIRECT (set in DNAT or UNSNAT stages)
> > to
> > > -             * the central instance of the l3dgw_port for NAT
> > processing. */
> > > -            ds_clear(&actions);
> > > -            ds_put_format(&actions, "outport = %s; next;",
> > > -                          od->l3redirect_port->json_key);
> > > -            ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 200,
> > > -                          REGBIT_NAT_REDIRECT" == 1",
> > ds_cstr(&actions));
> > > +             * ARP Resolution table, and Ingress Gateway Redirect
> Table
> > > +             * that are not specific to a NAT rule. */
> > > +            for (size_t i = 0; i < od->n_l3dgw_ports; i++) {
> > > +                struct ovn_port *l3dgw_port = od->l3dgw_ports[i];
> > > +                /* The highest priority IN_ARP_RESOLVE rule matches
> > packets
> > > +                 * with REGBIT_NAT_REDIRECT (set in DNAT or UNSNAT
> > stages),
> > > +                 * then sets eth.dst to the distributed gateway port's
> > > +                 * ethernet address. */
> > > +                ds_clear(&match);
> > > +                ds_put_format(&match, REGBIT_NAT_REDIRECT" == 1 && "
> > > +                              "outport == %s", l3dgw_port->json_key);
> > > +                ds_clear(&actions);
> > > +                ds_put_format(&actions, "eth.dst = %s; next;",
> > > +                              l3dgw_port->lrp_networks.ea_s);
> > > +                ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_RESOLVE,
> 200,
> > > +                              ds_cstr(&match), ds_cstr(&actions));
> > > +
> > > +                /* The highest priority IN_GW_REDIRECT rule redirects
> > packets
> > > +                 * with REGBIT_NAT_REDIRECT (set in DNAT or UNSNAT
> > stages) to
> > > +                 * the central instance of the l3dgw_port for NAT
> > processing.
> > > +                 */
> > > +                ds_clear(&actions);
> > > +                ds_put_format(&actions, "outport = %s; next;",
> > > +                              l3dgw_port->l3redirect_port->json_key);
> > > +                ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT,
> 200,
> > > +                              ds_cstr(&match), ds_cstr(&actions));
> > > +            }
> > >          }
> > >
> > >          /* Load balancing and packet defrag are only valid on
> > >           * Gateway routers or router with gateway port. */
> > > -        if (!smap_get(&od->nbr->options, "chassis") &&
> !od->l3dgw_port)
> > {
> > > +        if (!smap_get(&od->nbr->options, "chassis") &&
> > !od->n_l3dgw_ports) {
> > >              continue;
> > >          }
> > >
> > > @@ -5517,6 +5548,9 @@ build_lrouter_flows(struct hmap *datapaths,
> struct
> > hmap *ports,
> > >                      ds_put_format(&match, "ip && ip6.dst == %s",
> > >                                  ip_address);
> > >                  }
> > > +                /* find l3dgw port by lb vip */
> > > +                const struct ovn_port *l3dgw_port
> > > +                    = find_l3dgw_port(od, ip_address);
> > >                  free(ip_address);
> > >
> > >                  int prio = 110;
> > > @@ -5533,13 +5567,13 @@ build_lrouter_flows(struct hmap *datapaths,
> > struct hmap *ports,
> > >                      prio = 120;
> > >                  }
> > >
> > > -                if (od->l3redirect_port) {
> > > +                if (l3dgw_port) {
> > >                      ds_put_format(&match, " &&
> is_chassis_resident(%s)",
> > > -                                  od->l3redirect_port->json_key);
> > > +                                  l3dgw_port->l3redirect_port->
> > json_key);
> > >                  }
> > >                  add_router_lb_flow(lflows, od, &match, &actions, prio,
> > >                                     lb_force_snat_ip, node->value,
> > is_udp,
> > > -                                   addr_family);
> > > +                                   addr_family, l3dgw_port);
> > >              }
> > >          }
> > >          sset_destroy(&all_ips);
> > > @@ -5898,17 +5932,18 @@ build_lrouter_flows(struct hmap *datapaths,
> > struct hmap *ports,
> > >          if (!od->nbr) {
> > >              continue;
> > >          }
> > > -        if (od->l3dgw_port && od->l3redirect_port) {
> > > +        for (size_t i = 0; i < od->n_l3dgw_ports; i++) {
> > > +            struct ovn_port *l3dgw_port = od->l3dgw_ports[i];
> > >              /* For traffic with outport == l3dgw_port, if the
> > >               * packet did not match any higher priority redirect
> > >               * rule, then the traffic is redirected to the central
> > >               * instance of the l3dgw_port. */
> > >              ds_clear(&match);
> > >              ds_put_format(&match, "outport == %s",
> > > -                          od->l3dgw_port->json_key);
> > > +                          l3dgw_port->json_key);
> > >              ds_clear(&actions);
> > >              ds_put_format(&actions, "outport = %s; next;",
> > > -                          od->l3redirect_port->json_key);
> > > +                          l3dgw_port->l3redirect_port->json_key);
> > >              ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 50,
> > >                            ds_cstr(&match), ds_cstr(&actions));
> > >
> > > diff --git a/ovn/ovn-nb.xml b/ovn/ovn-nb.xml
> > > index b7a5b6bf2..c0c6a45f0 100644
> > > --- a/ovn/ovn-nb.xml
> > > +++ b/ovn/ovn-nb.xml
> > > @@ -1430,8 +1430,7 @@
> > >          <p>
> > >            If set, this indicates that this logical router port
> > represents
> > >            a distributed gateway port that connects this router to a
> > logical
> > > -          switch with a localnet port.  There may be at most one such
> > > -          logical router port on each logical router.
> > > +          switch with a localnet port.
> > >          </p>
> > >
> > >          <p>
> > > @@ -1617,7 +1616,14 @@
> > >      </column>
> > >
> > >      <column name="external_ip">
> > > -      An IPv4 address.
> > > +      <p>
> > > +        An IPv4 address.
> > > +      </p>
> > > +
> > > +      <p>
> > > +        On distributed router, This address must be within the subnet
> of
> > > +        the gateway port instance on the
> <code>redirect-chassis</code>.
> > > +      </p>
> > >      </column>
> > >
> > >      <column name="external_mac">
> > > diff --git a/tests/system-ovn.at b/tests/system-ovn.at
> > > index 638c0b661..6f30c1ec6 100644
> > > --- a/tests/system-ovn.at
> > > +++ b/tests/system-ovn.at
> > > @@ -1097,23 +1097,27 @@ start_daemon ovn-controller
> > >
> > >  # Logical network:
> > >  # One LR R1 with switches foo (192.168.1.0/24), bar (192.168.2.0/24),
> > > -# and alice (172.16.1.0/24) connected to it.  The port between R1 and
> > > -# alice is the router gateway port where the R1 LB rules are applied.
> > > +# alice (172.16.1.0/24) and outsite (172.16.2.0/24) connected to it.
> > > +# The port between R1 and alice/outsite is the router gateway port
> > > +# where the R1 LB rules are applied.
> > >  #
> > > -#    foo -- R1 -- bar
> > > -#           |
> > > -#    alice ----
> > > +#     foo ---+--- bar
> > > +#            R1
> > > +#    alice --+-- outsite
> > >
> > >  ovn-nbctl lr-add R1
> > >
> > >  ovn-nbctl ls-add foo
> > >  ovn-nbctl ls-add bar
> > >  ovn-nbctl ls-add alice
> > > +ovn-nbctl ls-add outsite
> > >
> > >  ovn-nbctl lrp-add R1 foo 00:00:01:01:02:03 192.168.1.1/24
> > >  ovn-nbctl lrp-add R1 bar 00:00:01:01:02:04 192.168.2.1/24
> > >  ovn-nbctl lrp-add R1 alice 00:00:02:01:02:03 172.16.1.1/24 \
> > >      -- set Logical_Router_Port alice options:redirect-chassis=hv1
> > > +ovn-nbctl lrp-add R1 outsite 00:00:03:01:02:01 172.16.2.1/24 \
> > > +    -- set Logical_Router_Port outsite options:redirect-chassis=hv1
> > >
> > >  # Connect foo to R1
> > >  ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo \
> > > @@ -1130,6 +1134,11 @@ ovn-nbctl lsp-add alice rp-alice -- set
> > Logical_Switch_Port rp-alice \
> > >      type=router options:router-port=alice \
> > >      -- lsp-set-addresses rp-alice router
> > >
> > > +# Connect outsite to R1
> > > +ovn-nbctl lsp-add outsite rp-outsite -- set Logical_Switch_Port
> > rp-outsite \
> > > +    type=router options:router-port=outsite \
> > > +    -- lsp-set-addresses rp-outsite router
> > > +
> > >  # Logical port 'foo1' in switch 'foo'.
> > >  ADD_NAMESPACES(foo1)
> > >  ADD_VETH(foo1, foo1, br-int, "192.168.1.2/24", "f0:00:00:01:02:03", \
> > > @@ -1158,12 +1167,21 @@ ADD_VETH(alice1, alice1, br-int, "
> 172.16.1.2/24",
> > "f0:00:00:01:02:05", \
> > >  ovn-nbctl lsp-add alice alice1 \
> > >  -- lsp-set-addresses alice1 "f0:00:00:01:02:05 172.16.1.2"
> > >
> > > +# Logical port 'outsite1' in switch 'outsite'.
> > > +ADD_NAMESPACES(outsite1)
> > > +ADD_VETH(outsite1, outsite1, br-int, "172.16.2.2/24",
> > "f0:00:00:01:02:07", \
> > > +         "172.16.2.1")
> > > +ovn-nbctl lsp-add outsite outsite1 \
> > > +-- lsp-set-addresses outsite1 "f0:00:00:01:02:07 172.16.2.2"
> > > +
> > >  # Config OVN load-balancer with a VIP.
> > >  uuid=`ovn-nbctl  create load_balancer vips:172.16.1.10="192.168.1.2,
> > 192.168.2.2"`
> > > -ovn-nbctl set logical_router R1 load_balancer=$uuid
> > > +uuid2=`ovn-nbctl  create load_balancer vips:172.16.2.10="192.168.1.2,
> > 192.168.2.2"`
> > > +ovn-nbctl set logical_router R1 load_balancer=$uuid,$uuid2
> > >
> > >  # Config OVN load-balancer with another VIP (this time with ports).
> > >  ovn-nbctl set load_balancer $uuid vips:'"172.16.1.11:8000"'='"19
> > 2.168.1.2:80,192.168.2.2:80"'
> > > +ovn-nbctl set load_balancer $uuid2 vips:'"172.16.2.11:8000"'='"19
> > 2.168.1.2:80,192.168.2.2:80"'
> > >
> > >  # Wait for ovn-controller to catch up.
> > >  ovn-nbctl --wait=hv sync
> > > @@ -1179,6 +1197,10 @@ for i in `seq 1 20`; do
> > >      echo Request $i
> > >      NS_CHECK_EXEC([alice1], [wget 172.16.1.10 -t 5 -T 1
> > --retry-connrefused -v -o wget$i.log])
> > >  done
> > > +for i in `seq 1 20`; do
> > > +    echo Request $i
> > > +    NS_CHECK_EXEC([outsite1], [wget 172.16.2.10 -t 5 -T 1
> > --retry-connrefused -v -o wget$i.log])
> > > +done
> > >
> > >  dnl Each server should have at least one connection.
> > >  AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.1.10) |
> > > @@ -1186,12 +1208,21 @@ sed -e 's/zone=[[0-9]]*/zone=<cleared>/'],
> [0],
> > [dnl
> > >  tcp,orig=(src=172.16.1.2,dst=172.16.1.10,sport=<cleared>,
> > dport=<cleared>),reply=(src=192.168.1.2,dst=172.16.1.2,
> > sport=<cleared>,dport=<cleared>),zone=<cleared>,
> > protoinfo=(state=<cleared>)
> > >  tcp,orig=(src=172.16.1.2,dst=172.16.1.10,sport=<cleared>,
> > dport=<cleared>),reply=(src=192.168.2.2,dst=172.16.1.2,
> > sport=<cleared>,dport=<cleared>),zone=<cleared>,
> > protoinfo=(state=<cleared>)
> > >  ])
> > > +AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.2.10) |
> > > +sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
> > > +tcp,orig=(src=172.16.2.2,dst=172.16.2.10,sport=<cleared>,
> > dport=<cleared>),reply=(src=192.168.1.2,dst=172.16.2.2,
> > sport=<cleared>,dport=<cleared>),zone=<cleared>,
> > protoinfo=(state=<cleared>)
> > > +tcp,orig=(src=172.16.2.2,dst=172.16.2.10,sport=<cleared>,
> > dport=<cleared>),reply=(src=192.168.2.2,dst=172.16.2.2,
> > sport=<cleared>,dport=<cleared>),zone=<cleared>,
> > protoinfo=(state=<cleared>)
> > > +])
> > >
> > >  dnl Test load-balancing that includes L4 ports in NAT.
> > >  for i in `seq 1 20`; do
> > >      echo Request $i
> > >      NS_CHECK_EXEC([alice1], [wget 172.16.1.11:8000 -t 5 -T 1
> > --retry-connrefused -v -o wget$i.log])
> > >  done
> > > +for i in `seq 1 20`; do
> > > +    echo Request $i
> > > +    NS_CHECK_EXEC([outsite1], [wget 172.16.2.11:8000 -t 5 -T 1
> > --retry-connrefused -v -o wget$i.log])
> > > +done
> > >
> > >  dnl Each server should have at least one connection.
> > >  AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.1.11) |
> > > @@ -1199,6 +1230,11 @@ sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0],
> > [dnl
> > >  tcp,orig=(src=172.16.1.2,dst=172.16.1.11,sport=<cleared>,
> > dport=<cleared>),reply=(src=192.168.1.2,dst=172.16.1.2,
> > sport=<cleared>,dport=<cleared>),zone=<cleared>,
> > protoinfo=(state=<cleared>)
> > >  tcp,orig=(src=172.16.1.2,dst=172.16.1.11,sport=<cleared>,
> > dport=<cleared>),reply=(src=192.168.2.2,dst=172.16.1.2,
> > sport=<cleared>,dport=<cleared>),zone=<cleared>,
> > protoinfo=(state=<cleared>)
> > >  ])
> > > +AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.2.11) |
> > > +sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
> > > +tcp,orig=(src=172.16.2.2,dst=172.16.2.11,sport=<cleared>,
> > dport=<cleared>),reply=(src=192.168.1.2,dst=172.16.2.2,
> > sport=<cleared>,dport=<cleared>),zone=<cleared>,
> > protoinfo=(state=<cleared>)
> > > +tcp,orig=(src=172.16.2.2,dst=172.16.2.11,sport=<cleared>,
> > dport=<cleared>),reply=(src=192.168.2.2,dst=172.16.2.2,
> > sport=<cleared>,dport=<cleared>),zone=<cleared>,
> > protoinfo=(state=<cleared>)
> > > +])
> > >
> > >  OVS_APP_EXIT_AND_WAIT([ovn-controller])
> > >
> > > @@ -1238,23 +1274,27 @@ start_daemon ovn-controller
> > >
> > >  # Logical network:
> > >  # One LR R1 with switches foo (192.168.1.0/24), bar (192.168.2.0/24),
> > > -# and alice (172.16.1.0/24) connected to it.  The port between R1 and
> > > -# alice is the router gateway port where the R1 NAT rules are applied.
> > > +# alice (172.16.1.0/24) and outsite (172.16.2.0/24) connected to it.
> > > +# The port between R1 and alice/outsite is the router gateway port
> > > +# where the R1 NAT rules are applied.
> > >  #
> > > -#    foo -- R1 -- alice
> > > -#           |
> > > -#    bar ----
> > > +#     foo ---+--- bar
> > > +#            R1
> > > +#    alice --+-- outsite
> > >
> > >  ovn-nbctl lr-add R1
> > >
> > >  ovn-nbctl ls-add foo
> > >  ovn-nbctl ls-add bar
> > >  ovn-nbctl ls-add alice
> > > +ovn-nbctl ls-add outsite
> > >
> > >  ovn-nbctl lrp-add R1 foo 00:00:01:01:02:03 192.168.1.1/24
> > >  ovn-nbctl lrp-add R1 bar 00:00:01:01:02:04 192.168.2.1/24
> > >  ovn-nbctl lrp-add R1 alice 00:00:02:01:02:03 172.16.1.1/24 \
> > >      -- set Logical_Router_Port alice options:redirect-chassis=hv1
> > > +ovn-nbctl lrp-add R1 outsite 00:00:03:01:02:03 172.16.2.1/24 \
> > > +    -- set Logical_Router_Port outsite options:redirect-chassis=hv1
> > >
> > >  # Connect foo to R1
> > >  ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo \
> > > @@ -1271,6 +1311,11 @@ ovn-nbctl lsp-add alice rp-alice -- set
> > Logical_Switch_Port rp-alice \
> > >      type=router options:router-port=alice \
> > >      -- lsp-set-addresses rp-alice router
> > >
> > > +# Connect outsite to R1
> > > +ovn-nbctl lsp-add outsite rp-outsite -- set Logical_Switch_Port
> > rp-outsite \
> > > +    type=router options:router-port=outsite \
> > > +    -- lsp-set-addresses rp-outsite router
> > > +
> > >  # Logical port 'foo1' in switch 'foo'.
> > >  ADD_NAMESPACES(foo1)
> > >  ADD_VETH(foo1, foo1, br-int, "192.168.1.2/24", "f0:00:00:01:02:03", \
> > > @@ -1299,15 +1344,26 @@ ADD_VETH(alice1, alice1, br-int, "
> 172.16.1.2/24",
> > "f0:00:00:01:02:05", \
> > >  ovn-nbctl lsp-add alice alice1 \
> > >  -- lsp-set-addresses alice1 "f0:00:00:01:02:05 172.16.1.2"
> > >
> > > +# Logical port 'outsite1' in switch 'outsite'.
> > > +ADD_NAMESPACES(outsite1)
> > > +ADD_VETH(outsite1, outsite1, br-int, "172.16.2.2/24",
> > "f0:00:00:01:02:07", \
> > > +         "172.16.2.1")
> > > +ovn-nbctl lsp-add outsite outsite1 \
> > > +-- lsp-set-addresses outsite1 "f0:00:00:01:02:07 172.16.2.2"
> > > +
> > >  # Add DNAT rules
> > >  AT_CHECK([ovn-nbctl lr-nat-add R1 dnat_and_snat 172.16.1.3 192.168.1.2
> > foo1 00:00:02:02:03:04])
> > >  AT_CHECK([ovn-nbctl lr-nat-add R1 dnat_and_snat 172.16.1.4 192.168.1.3
> > foo2 00:00:02:02:03:05])
> > > +AT_CHECK([ovn-nbctl lr-nat-add R1 dnat_and_snat 172.16.2.3 192.168.1.2
> > foo1 00:00:02:02:03:04])
> > > +AT_CHECK([ovn-nbctl lr-nat-add R1 dnat_and_snat 172.16.2.4 192.168.1.3
> > foo2 00:00:02:02:03:05])
> > >
> > >  # Add a SNAT rule
> > >  AT_CHECK([ovn-nbctl lr-nat-add R1 snat 172.16.1.1 192.168.0.0/16])
> > > +AT_CHECK([ovn-nbctl lr-nat-add R1 snat 172.16.2.1 192.168.2.0/24])
> > >
> > >  ovn-nbctl --wait=hv sync
> > >  OVS_WAIT_UNTIL([ovs-ofctl dump-flows br-int | grep
> > 'nat(src=172.16.1.1)'])
> > > +OVS_WAIT_UNTIL([ovs-ofctl dump-flows br-int | grep
> > 'nat(src=172.16.2.1)'])
> > >
> > >  # North-South DNAT: 'alice1' pings 'foo1' using 172.16.1.3.
> > >  NS_CHECK_EXEC([alice1], [ping -q -c 3 -i 0.3 -w 2 172.16.1.3 |
> > FORMAT_PING], \
> > > @@ -1315,11 +1371,21 @@ NS_CHECK_EXEC([alice1], [ping -q -c 3 -i 0.3 -w
> > 2 172.16.1.3 | FORMAT_PING], \
> > >  3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > >  ])
> > >
> > > +# North-South DNAT: 'outsite1' pings 'foo1' using 172.16.2.3.
> > > +NS_CHECK_EXEC([outsite1], [ping -q -c 3 -i 0.3 -w 2 172.16.2.3 |
> > FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > >  # We verify that DNAT indeed happened via 'dump-conntrack' command.
> > >  AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.1.3) | \
> > >  sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
> > >  icmp,orig=(src=172.16.1.2,dst=172.16.1.3,id=<cleared>,type=
> > 8,code=0),reply=(src=192.168.1.2,dst=172.16.1.2,id=<
> > cleared>,type=0,code=0),zone=<cleared>
> > >  ])
> > > +AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.2.3) | \
> > > +sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
> > > +icmp,orig=(src=172.16.2.2,dst=172.16.2.3,id=<cleared>,
> > type=8,code=0),reply=(src=192.168.1.2,dst=172.16.2.2,id=<
> > cleared>,type=0,code=0),zone=<cleared>
> > > +])
> > >
> > >  # South-North SNAT: 'foo2' pings 'alice1'. But 'alice1' receives
> traffic
> > >  # from 172.16.1.4
> > > @@ -1328,11 +1394,22 @@ NS_CHECK_EXEC([foo2], [ping -q -c 3 -i 0.3 -w 2
> > 172.16.1.2 | FORMAT_PING], \
> > >  3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > >  ])
> > >
> > > +# South-North SNAT: 'foo2' pings 'outsite1'. But 'outsite1' receives
> > traffic
> > > +# from 172.16.2.4
> > > +NS_CHECK_EXEC([foo2], [ping -q -c 3 -i 0.3 -w 2 172.16.2.2 |
> > FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > >  # We verify that SNAT indeed happened via 'dump-conntrack' command.
> > >  AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.1.4) | \
> > >  sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
> > >  icmp,orig=(src=192.168.1.3,dst=172.16.1.2,id=<cleared>,
> > type=8,code=0),reply=(src=172.16.1.2,dst=172.16.1.4,id=<
> > cleared>,type=0,code=0),zone=<cleared>
> > >  ])
> > > +AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.2.4) | \
> > > +sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
> > > +icmp,orig=(src=192.168.1.3,dst=172.16.2.2,id=<cleared>,
> > type=8,code=0),reply=(src=172.16.2.2,dst=172.16.2.4,id=<
> > cleared>,type=0,code=0),zone=<cleared>
> > > +])
> > >
> > >  # South-North SNAT: 'bar1' pings 'alice1'. But 'alice1' receives
> traffic
> > >  # from 172.16.1.1
> > > @@ -1341,11 +1418,22 @@ NS_CHECK_EXEC([bar1], [ping -q -c 3 -i 0.3 -w 2
> > 172.16.1.2 | FORMAT_PING], \
> > >  3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > >  ])
> > >
> > > +# South-North SNAT: 'bar1' pings 'outsite1'. But 'outsite1' receives
> > traffic
> > > +# from 172.16.2.1
> > > +NS_CHECK_EXEC([bar1], [ping -q -c 3 -i 0.3 -w 2 172.16.2.2 |
> > FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > >  # We verify that SNAT indeed happened via 'dump-conntrack' command.
> > >  AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.1.1) | \
> > >  sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
> > >  icmp,orig=(src=192.168.2.2,dst=172.16.1.2,id=<cleared>,
> > type=8,code=0),reply=(src=172.16.1.2,dst=172.16.1.1,id=<
> > cleared>,type=0,code=0),zone=<cleared>
> > >  ])
> > > +AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.2.1) | \
> > > +sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
> > > +icmp,orig=(src=192.168.2.2,dst=172.16.2.2,id=<cleared>,
> > type=8,code=0),reply=(src=172.16.2.2,dst=172.16.2.1,id=<
> > cleared>,type=0,code=0),zone=<cleared>
> > > +])
> > >
> > >  OVS_APP_EXIT_AND_WAIT([ovn-controller])
> > >
> > > @@ -1385,23 +1473,27 @@ start_daemon ovn-controller
> > >
> > >  # Logical network:
> > >  # One LR R1 with switches foo (192.168.1.0/24), bar (192.168.2.0/24),
> > > -# and alice (172.16.1.0/24) connected to it.  The port between R1 and
> > > -# alice is the router gateway port where the R1 NAT rules are applied.
> > > +# alice (172.16.1.0/24) and outsite (172.16.1.0/24) connected to it.
> > > +# The port between R1 and alice/outsite is the router gateway port
> > > +# where the R1 NAT rules are applied.
> > >  #
> > > -#    foo -- R1 -- alice
> > > -#           |
> > > -#    bar ----
> > > +#     foo ---+--- bar
> > > +#            R1
> > > +#    alice --+-- outsite
> > >
> > >  ovn-nbctl lr-add R1
> > >
> > >  ovn-nbctl ls-add foo
> > >  ovn-nbctl ls-add bar
> > >  ovn-nbctl ls-add alice
> > > +ovn-nbctl ls-add outsite
> > >
> > >  ovn-nbctl lrp-add R1 foo 00:00:01:01:02:03 192.168.1.1/24
> > >  ovn-nbctl lrp-add R1 bar 00:00:01:01:02:04 192.168.2.1/24
> > >  ovn-nbctl lrp-add R1 alice 00:00:02:01:02:03 172.16.1.1/24 \
> > >      -- set Logical_Router_Port alice options:redirect-chassis=hv1
> > > +ovn-nbctl lrp-add R1 outsite 00:00:03:01:02:01 172.16.2.1/24 \
> > > +    -- set Logical_Router_Port outsite options:redirect-chassis=hv1
> > >
> > >  # Connect foo to R1
> > >  ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo \
> > > @@ -1418,6 +1510,11 @@ ovn-nbctl lsp-add alice rp-alice -- set
> > Logical_Switch_Port rp-alice \
> > >      type=router options:router-port=alice \
> > >      -- lsp-set-addresses rp-alice router
> > >
> > > +# Connect outsite to R1
> > > +ovn-nbctl lsp-add outsite rp-outsite -- set Logical_Switch_Port
> > rp-outsite \
> > > +    type=router options:router-port=outsite \
> > > +    -- lsp-set-addresses rp-outsite router
> > > +
> > >  # Logical port 'foo1' in switch 'foo'.
> > >  ADD_NAMESPACES(foo1)
> > >  ADD_VETH(foo1, foo1, br-int, "192.168.1.2/24", "f0:00:00:01:02:03", \
> > > @@ -1446,15 +1543,26 @@ ADD_VETH(alice1, alice1, br-int, "
> 172.16.1.2/24",
> > "f0:00:00:01:02:05", \
> > >  ovn-nbctl lsp-add alice alice1 \
> > >  -- lsp-set-addresses alice1 "f0:00:00:01:02:05 172.16.1.2"
> > >
> > > +# Logical port 'outsite1' in switch 'outsite'.
> > > +ADD_NAMESPACES(outsite1)
> > > +ADD_VETH(outsite1, outsite1, br-int, "172.16.2.2/24",
> > "f0:00:00:01:02:07", \
> > > +         "172.16.2.1")
> > > +ovn-nbctl lsp-add outsite outsite1 \
> > > +-- lsp-set-addresses outsite1 "f0:00:00:01:02:07 172.16.2.2"
> > > +
> > >  # Add DNAT rules
> > >  AT_CHECK([ovn-nbctl lr-nat-add R1 dnat_and_snat 172.16.1.3 192.168.1.2
> > foo1 00:00:02:02:03:04])
> > >  AT_CHECK([ovn-nbctl lr-nat-add R1 dnat_and_snat 172.16.1.4 192.168.2.2
> > bar1 00:00:02:02:03:05])
> > > +AT_CHECK([ovn-nbctl lr-nat-add R1 dnat_and_snat 172.16.2.3 192.168.1.2
> > foo1 00:00:02:02:03:04])
> > > +AT_CHECK([ovn-nbctl lr-nat-add R1 dnat_and_snat 172.16.2.4 192.168.2.2
> > bar1 00:00:02:02:03:05])
> > >
> > >  # Add a SNAT rule
> > >  AT_CHECK([ovn-nbctl lr-nat-add R1 snat 172.16.1.1 192.168.0.0/16])
> > > +AT_CHECK([ovn-nbctl lr-nat-add R1 snat 172.16.2.1 192.168.1.0/24])
> > >
> > >  ovn-nbctl --wait=hv sync
> > >  OVS_WAIT_UNTIL([ovs-ofctl dump-flows br-int | grep
> > 'nat(src=172.16.1.1)'])
> > > +OVS_WAIT_UNTIL([ovs-ofctl dump-flows br-int | grep
> > 'nat(src=172.16.2.1)'])
> > >
> > >  echo "------ hv dump ------"
> > >  ovs-ofctl show br-int
> > > @@ -1500,6 +1608,12 @@ NS_CHECK_EXEC([foo1], [ping -q -c 3 -i 0.3 -w 2
> > 172.16.1.4 | FORMAT_PING], \
> > >  3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > >  ])
> > >
> > > +# East-West NAT: 'foo1' pings 'bar1' using 172.16.2.4.
> > > +NS_CHECK_EXEC([foo1], [ping -q -c 3 -i 0.3 -w 2 172.16.2.4 |
> > FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > >  # Check conntrack entries.  First SNAT of 'foo1' address happens.
> > >  # Then DNAT of 'bar1' address happens (listed first below).
> > >  AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.1.3) | \
> > > @@ -1507,6 +1621,11 @@ sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0],
> > [dnl
> > >  icmp,orig=(src=172.16.1.3,dst=172.16.1.4,id=<cleared>,type=
> > 8,code=0),reply=(src=192.168.2.2,dst=172.16.1.3,id=<
> > cleared>,type=0,code=0),zone=<cleared>
> > >  icmp,orig=(src=192.168.1.2,dst=172.16.1.4,id=<cleared>,
> > type=8,code=0),reply=(src=172.16.1.4,dst=172.16.1.3,id=<
> > cleared>,type=0,code=0),zone=<cleared>
> > >  ])
> > > +AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.2.3) | \
> > > +sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
> > > +icmp,orig=(src=172.16.2.3,dst=172.16.2.4,id=<cleared>,
> > type=8,code=0),reply=(src=192.168.2.2,dst=172.16.2.3,id=<
> > cleared>,type=0,code=0),zone=<cleared>
> > > +icmp,orig=(src=192.168.1.2,dst=172.16.2.4,id=<cleared>,
> > type=8,code=0),reply=(src=172.16.2.4,dst=172.16.2.3,id=<
> > cleared>,type=0,code=0),zone=<cleared>
> > > +])
> > >
> > >  # East-West NAT: 'foo2' pings 'bar1' using 172.16.1.4.
> > >  NS_CHECK_EXEC([foo2], [ping -q -c 3 -i 0.3 -w 2 172.16.1.4 |
> > FORMAT_PING], \
> > > @@ -1514,6 +1633,12 @@ NS_CHECK_EXEC([foo2], [ping -q -c 3 -i 0.3 -w 2
> > 172.16.1.4 | FORMAT_PING], \
> > >  3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > >  ])
> > >
> > > +# East-West NAT: 'foo2' pings 'bar1' using 172.16.2.4.
> > > +NS_CHECK_EXEC([foo2], [ping -q -c 3 -i 0.3 -w 2 172.16.2.4 |
> > FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > >  # Check conntrack entries.  First SNAT of 'foo2' address happens.
> > >  # Then DNAT of 'bar1' address happens (listed first below).
> > >  AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.1.1) | \
> > > @@ -1521,6 +1646,11 @@ sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0],
> > [dnl
> > >  icmp,orig=(src=172.16.1.1,dst=172.16.1.4,id=<cleared>,type=
> > 8,code=0),reply=(src=192.168.2.2,dst=172.16.1.1,id=<
> > cleared>,type=0,code=0),zone=<cleared>
> > >  icmp,orig=(src=192.168.1.3,dst=172.16.1.4,id=<cleared>,
> > type=8,code=0),reply=(src=172.16.1.4,dst=172.16.1.1,id=<
> > cleared>,type=0,code=0),zone=<cleared>
> > >  ])
> > > +AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.2.1) | \
> > > +sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
> > > +icmp,orig=(src=172.16.2.1,dst=172.16.2.4,id=<cleared>,
> > type=8,code=0),reply=(src=192.168.2.2,dst=172.16.2.1,id=<
> > cleared>,type=0,code=0),zone=<cleared>
> > > +icmp,orig=(src=192.168.1.3,dst=172.16.2.4,id=<cleared>,
> > type=8,code=0),reply=(src=172.16.2.4,dst=172.16.2.1,id=<
> > cleared>,type=0,code=0),zone=<cleared>
> > > +])
> > >
> > >  OVS_APP_EXIT_AND_WAIT([ovn-controller])
> > >
> > > --
> > > 2.13.2.windows.1
> > >
> > > _______________________________________________
> > > dev mailing list
> > > d...@openvswitch.org
> > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
> >
> _______________________________________________
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>
>
>
_______________________________________________
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Reply via email to