Re: [ovs-dev] [PATCH ovn v4] northd: Make ECMP group and members independent of processing order.

Fri, 16 Oct 2026 13:12:11 -0700 

Hi Lucas,
Thanks so much for the new version! Overall I like the design. Since it is a complete rework and thus entirely new code, I have some comments below. 

On 10/15/26 4:02 PM, Lucas Vargas Dias wrote:

The ECMP group id and the member ids are written into the
lr_in_ip_routing and lr_in_ip_routing_ecmp logical flows, but they
were assigned from the order in which routes reached
en_group_ecmp_route: the group id was hmap_count() at creation time
and the member id was the insertion position.  On a full recompute
the routes are walked in hmap order, while incremental processing
appends them in event order, so the same set of routes could get a
different numbering.  Every forced or fallback recompute could then
delete and re-insert Logical_Flow rows in the Southbound DB even
though nothing had changed.

Derive the ids from hashes instead.  The group id comes from the
group key, i.e. prefix, prefix length, route table, source or
destination route and source class, which does not change as members
are added or removed.  The member id comes from the UUID of the
route's source row, its nexthop and its output port: a NAT or LB row
produces one route per path to the peer router, so the UUID alone is
not unique within a group.

When two elements derive the same id, all of them retry in the next
round with an id derived from their hash and the round number, so
that the ids only depend on the set of routes and not on the order
in which they were processed.  The members are listed in the select()
action in id order for the same reason.

Besides making a recompute produce the same flows as incremental
processing, adding or removing a route no longer changes the ids,
and so the flows, of unrelated groups or members.

Fixes: 6979a96138a2 ("northd: Support I+P for group_ecmp_route engine.")
Fixes: 5395e12f247c ("northd: Drop traffic for ECMP group with "discard" 
route.")
Suggested-by: Rosemarie O'Riorden <[email protected]>
Assisted-by: Claude Opus 5.5, Claude Code
Signed-off-by: Lucas Vargas Dias <[email protected]>
---
v4:
   - Derive the ECMP group and member ids from hashes instead of sorting,
     so that adding or removing a route doesn't change the ids of
     unrelated groups or members (suggested by Rosemarie).
   - Added a test with NAT and LB routes reachable over several paths,
     whose ECMP members share their source row.
   - Added ovn_strip_ecmp_ids(), ecmp_ids() and ecmp_member_id() for the
     tests.
v3:
   - Dropped nullable_strcmp() and the patch converting its callers.
   - Removed the unused 'route_count' field from struct ecmp_groups_node.
v2:
   - Moved nullable_strcmp() to lib/ovn-util.h and documented its NULL
     handling.
   - Clarified in ecmp_groups_node_cmp() that it can't return 0 for two
     distinct groups.

  northd/en-group-ecmp-route.c | 247 +++++++++++++++++++++++++---
  northd/en-group-ecmp-route.h |   1 -
  tests/ovn-macros.at          |  34 ++++
  tests/ovn-northd.at          | 300 +++++++++++++++++++++++++++--------
  4 files changed, 486 insertions(+), 96 deletions(-)

diff --git a/northd/en-group-ecmp-route.c b/northd/en-group-ecmp-route.c
index aca197318..bf553dded 100644
--- a/northd/en-group-ecmp-route.c
+++ b/northd/en-group-ecmp-route.c
@@ -24,6 +24,7 @@
  #include "en-advertised-route-sync.h"
  #include "en-group-ecmp-route.h"
  #include "en-learned-route-sync.h"
+#include "hash.h"
  #include "hmapx.h"
  #include "openvswitch/hmap.h"

  
@@ -228,14 +229,12 @@ ecmp_groups_add_route(struct ecmp_groups_node *group,

                        const struct parsed_route *route)
  {
      static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
-    if (group->route_count == UINT16_MAX) {
+    if (vector_len(&group->route_list) == UINT16_MAX) {
          VLOG_WARN_RL(&rl, "too many routes in a single ecmp group.");
          return;
      }

  
      if (route->is_discard_route) {

-        group->has_discard_route = true;
-
          char *prefix = normalize_v46_prefix(&route->prefix, route->plen);
          VLOG_WARN_RL(&rl, "The ECMP route \"%s\" contains \"discard\" "
                       "route, the whole group will drop traffic.", prefix);
@@ -244,22 +243,13 @@ ecmp_groups_add_route(struct ecmp_groups_node *group,

  
      struct ecmp_route_list_node er = (struct ecmp_route_list_node) {

          .route = route,
-        .id = ++group->route_count,
      };

  
-    if (group->route_count == 1) {

-        sset_clone(&group->selection_fields, &route->ecmp_selection_fields);
-    } else {
-        sset_intersect(&group->selection_fields,
-                       &route->ecmp_selection_fields);
-    }
-
      vector_push(&group->route_list, &er);
  }

  
-/* Removes a route from an ecmp group. If the ecmp group should persist

- * afterwards you must call ecmp_groups_update_ids before any further
- * insertions. */
+/* Removes a route from an ecmp group.  The ids of the group and of its
+ * members are refreshed by group_ecmp_datapath_finalize(). */
  static const struct parsed_route *
  ecmp_groups_remove_route(struct ecmp_groups_node *group,
                           const struct parsed_route *pr)
@@ -278,15 +268,221 @@ ecmp_groups_remove_route(struct ecmp_groups_node *group,
      return NULL;
  }

  
+/* NAT and LB routes for the same prefix share a group, see

+ * route_sources_ecmp_compatible(). */
+static enum route_source
+ecmp_group_source_class(enum route_source source)
+{
+    return source == ROUTE_SOURCE_LB ? ROUTE_SOURCE_NAT : source;



So, this function relies follows the rules of 
route_sources_ecmp_compatible(), which makes them independent yet 
dependent in a way, because whenever someone updates one they also have 
to update the other... I think it would make sense to make one actually 
call the other or something like that to logically intertwine them, to 
make it so only one function has to be changed in that case. Because 
they're quite simple.



I guess the same is kind of true of ecmp_group_hash() and 
ecmp_group_hash(), that they need to stay consistent and updates need to 
be made in two separate places. But I don't see any way to easily 
logically link them so oh well.



+}
+
+/* Hashes the key of 'eg', which is unique within a datapath, see
+ * ecmp_groups_find().  Unlike the members, the key does not change when
+ * routes are added to or removed from the group. */
+static uint32_t
+ecmp_group_hash(const struct ecmp_groups_node *eg)
+{
+    uint32_t hash = hash_bytes(&eg->prefix, sizeof eg->prefix, 0);
+    hash = hash_add(hash, eg->plen);
+    hash = hash_add(hash, eg->route_table_id);
+    hash = hash_add(hash, eg->is_src_route);
+    hash = hash_add(hash, ecmp_group_source_class(eg->source));
+    return hash_finish(hash, 0);
+}
+
+/* Hashes what makes 'route' unique within its ecmp group.  A NAT or LB row
+ * produces one route per path to the peer router, so its UUID alone is not
+ * enough. */
+static uint32_t
+ecmp_member_hash(const struct parsed_route *route)
+{
+    uint32_t hash = uuid_hash(&route->source_hint->uuid);
+    if (route->nexthop) {
+        hash = hash_bytes(route->nexthop, sizeof *route->nexthop, hash);
+    }
+    if (route->out_port) {
+        hash = hash_string(route->out_port->key, hash);
+    }



Shouldn't the hash be finished here like it is in the other hashing 
function?



+    return hash;
+}
+
+/* Number of rounds after which ecmp_assign_ids() stops retrying hashes that
+ * collide and falls back to the next free id. */
+#define ECMP_ID_MAX_ROUNDS 32
+
+struct ecmp_id_slot {
+    struct hmap_node hmap_node;
+    uint16_t id;
+    bool taken;         /* Assigned to an element. */
+    size_t n_claims;    /* Elements that derived 'id' in the current round. */
+};
+
+static struct ecmp_id_slot *
+ecmp_id_slot_get(struct hmap *slots, uint16_t id)
+{
+    struct ecmp_id_slot *slot;
+    HMAP_FOR_EACH_WITH_HASH (slot, hmap_node, id, slots) {
+        if (slot->id == id) {
+            return slot;
+        }
+    }
+    slot = xzalloc(sizeof *slot);
+    slot->id = id;
+    hmap_insert(slots, &slot->hmap_node, id);
+    return slot;
+}
+
+static uint16_t
+ecmp_id_from_hash(uint32_t hash, unsigned int round)
+{
+    return hash_int(round, hash) % UINT16_MAX + 1;
+}
+
+/* Gives each of the 'n' elements a distinct id in [1, UINT16_MAX] derived
+ * from 'hashes', so that the id of an element does not depend on the order
+ * in which the elements are passed, nor on the ids of unrelated elements.
+ *
+ * An element keeps the id derived from its hash unless that id is already
+ * taken or another element derives the same id in the same round.  In that
+ * case all of them retry in the next round, with an id derived from their
+ * hash and the round number.  The result therefore only depends on the set
+ * of hashes.  Elements whose hashes are equal can't be told apart and, after
+ * ECMP_ID_MAX_ROUNDS rounds, take the next free id in the order they are
+ * passed. */
  static void
-ecmp_group_update_ids(struct ecmp_groups_node *group)
+ecmp_assign_ids(const uint32_t *hashes, uint16_t *ids, size_t n)
  {
+    struct hmap slots = HMAP_INITIALIZER(&slots);
+    struct ecmp_id_slot *slot;
+    size_t n_left = n;
+
+    memset(ids, 0, n * sizeof *ids);
+    for (unsigned int round = 0; n_left && round < ECMP_ID_MAX_ROUNDS;
+         round++) {
+        for (size_t i = 0; i < n; i++) {
+            if (!ids[i]) {
+                uint16_t id = ecmp_id_from_hash(hashes[i], round);
+                ecmp_id_slot_get(&slots, id)->n_claims++;
+            }
+        }
+        for (size_t i = 0; i < n; i++) {
+            if (!ids[i]) {
+                uint16_t id = ecmp_id_from_hash(hashes[i], round);



Nit: The id values are being calculated twice per round. This could be 
done at the beginning of each round. Although it's not a huge deal 
because since collisions are so unlikely there will usually only be one 
round. But you could still call it once per round rather than twice.



+                slot = ecmp_id_slot_get(&slots, id);
+                if (!slot->taken && slot->n_claims == 1) {
+                    slot->taken = true;
+                    ids[i] = id;
+                    n_left--;
+                }
+            }
+        }
+        HMAP_FOR_EACH (slot, hmap_node, &slots) {
+            slot->n_claims = 0;
+        }
+    }
+
+    for (size_t i = 0; n_left && i < n; i++) {
+        if (ids[i]) {
+            continue;
+        }
+        uint16_t id = ecmp_id_from_hash(hashes[i], 0);
+        while ((slot = ecmp_id_slot_get(&slots, id))->taken) {
+            id = id % UINT16_MAX + 1;
+        }
+        slot->taken = true;
+        ids[i] = id;
+        n_left--;
+    }
+
+    HMAP_FOR_EACH_POP (slot, hmap_node, &slots) {
+        free(slot);
+    }
+    hmap_destroy(&slots);
+}
+
+static int
+ecmp_route_list_node_id_cmp(const void *a_, const void *b_)
+{
+    const struct ecmp_route_list_node *a = a_;
+    const struct ecmp_route_list_node *b = b_;
+
+    return a->id < b->id ? -1 : a->id > b->id;
+}
+
+/* Recomputes everything in 'group' that is derived from its members, so that
+ * the result only depends on the set of members. */
+static void
+ecmp_group_finalize(struct ecmp_groups_node *group)
+{
+    size_t n = vector_len(&group->route_list);
+    uint32_t *hashes = xmalloc(n * sizeof *hashes);
+    uint16_t *ids = xmalloc(n * sizeof *ids);
      struct ecmp_route_list_node *er;
      size_t i = 0;
+
      VECTOR_FOR_EACH_PTR (&group->route_list, er) {
-        er->id = i++;
+        hashes[i++] = ecmp_member_hash(er->route);
+    }
+    ecmp_assign_ids(hashes, ids, n);
+    i = 0;
+    VECTOR_FOR_EACH_PTR (&group->route_list, er) {
+        er->id = ids[i++];
+    }
+    free(hashes);
+    free(ids);
+
+    /* The members are listed in this order in the select() action. */
+    vector_qsort(&group->route_list, ecmp_route_list_node_id_cmp);
+
+    bool first = true;
+    group->has_discard_route = false;
+    VECTOR_FOR_EACH_PTR (&group->route_list, er) {
+        if (er->route->is_discard_route) {
+            group->has_discard_route = true;
+        }
+        if (first) {
+            sset_destroy(&group->selection_fields);
+            sset_clone(&group->selection_fields,
+                       &er->route->ecmp_selection_fields);
+            group->source = er->route->source;
+            first = false;
+        } else {
+            sset_intersect(&group->selection_fields,
+                           &er->route->ecmp_selection_fields);
+        }
      }
-    group->route_count = i;
+}
+
+/* Assigns the group and member ids of all ecmp groups of 'gn'.  Must be
+ * called after the ecmp groups of 'gn' changed, both on full recompute and on
+ * incremental processing, so that both yield the same ids for the same set
+ * of routes and the generated logical flows do not change on recompute. */
+static void
+group_ecmp_datapath_finalize(struct group_ecmp_datapath *gn)
+{
+    size_t n = hmap_count(&gn->ecmp_groups);
+    if (!n) {
+        return;
+    }
+
+    struct ecmp_groups_node **groups = xmalloc(n * sizeof *groups);
+    uint32_t *hashes = xmalloc(n * sizeof *hashes);
+    uint16_t *ids = xmalloc(n * sizeof *ids);
+    struct ecmp_groups_node *eg;
+    size_t i = 0;
+
+    HMAP_FOR_EACH (eg, hmap_node, &gn->ecmp_groups) {
+        ecmp_group_finalize(eg);
+        groups[i] = eg;
+        hashes[i++] = ecmp_group_hash(eg);
+    }
+    ecmp_assign_ids(hashes, ids, n);
+    for (i = 0; i < n; i++) {
+        groups[i]->id = ids[i];
+    }
+    free(groups);
+    free(hashes);
+    free(ids);
  }

  
  static struct ecmp_groups_node *

@@ -302,7 +498,6 @@ ecmp_groups_add(struct group_ecmp_datapath *gn,
      struct ecmp_groups_node *eg = xzalloc(sizeof *eg);
      hmap_insert(&gn->ecmp_groups, &eg->hmap_node, route->hash);

  
-    eg->id = hmap_count(&gn->ecmp_groups);

      eg->prefix = route->prefix;
      eg->plen = route->plen;
      eg->is_src_route = route->is_src_route;
@@ -396,6 +591,10 @@ group_ecmp_route(struct group_ecmp_route_data *data,
          gn = group_ecmp_datapath_lookup_or_add(data, pr->od);
          add_route(gn, pr);
      }
+
+    HMAP_FOR_EACH (gn, hmap_node, &data->datapaths) {
+        group_ecmp_datapath_finalize(gn);
+    }
  }

  
  enum engine_node_state

@@ -472,9 +671,7 @@ handle_deleted_route(struct group_ecmp_route_data *data,
               * unique route. Otherwise it stays an ecmp group with just one
               * member. */
              ecmp_groups_remove_route(eg, pr);
-            if (ecmp_group_has_symmetric_reply(eg)) {
-                ecmp_group_update_ids(eg);
-            } else {
+            if (!ecmp_group_has_symmetric_reply(eg)) {
                  const struct ecmp_route_list_node *er =
                      vector_get_ptr(&eg->route_list, 0);
                  unique_routes_add(node, er->route);
@@ -482,11 +679,8 @@ handle_deleted_route(struct group_ecmp_route_data *data,
                  ecmp_groups_node_free(eg);
              }
          } else {
-            /* We can just remove the member from the group. We need to update
-             * the indices of all routes so that future insertions directly
-             * have a new index. */
+            /* We can just remove the member from the group. */
              ecmp_groups_remove_route(eg, pr);
-            ecmp_group_update_ids(eg);
          }
      }

  
@@ -537,6 +731,7 @@ group_ecmp_route_learned_route_change_handler(struct engine_node *eng_node,

              hmapx_add(&data->trk_data.deleted_datapath_routes, node);
              hmap_remove(&data->datapaths, &node->hmap_node);
          } else {
+            group_ecmp_datapath_finalize(node);
              hmapx_add(&data->trk_data.crupdated_datapath_routes, node);
          }
      }
@@ -589,6 +784,7 @@ group_ecmp_route_routes_change_handler(struct engine_node 
*eng_node,
              hmapx_add(&data->trk_data.deleted_datapath_routes, node);
              hmap_remove(&data->datapaths, &node->hmap_node);
          } else {
+            group_ecmp_datapath_finalize(node);
              hmapx_add(&data->trk_data.crupdated_datapath_routes, node);
          }
      }
@@ -645,6 +841,7 @@ group_ecmp_route_dynamic_routes_change_handler(struct 
engine_node *eng_node,
              hmapx_add(&gdata->trk_data.deleted_datapath_routes, node);
              hmap_remove(&gdata->datapaths, &node->hmap_node);
          } else {
+            group_ecmp_datapath_finalize(node);
              hmapx_add(&gdata->trk_data.crupdated_datapath_routes, node);
          }
      }
diff --git a/northd/en-group-ecmp-route.h b/northd/en-group-ecmp-route.h
index b40935be2..13f84a036 100644
--- a/northd/en-group-ecmp-route.h
+++ b/northd/en-group-ecmp-route.h
@@ -37,7 +37,6 @@ struct ecmp_groups_node {
      bool has_discard_route;
      enum route_source source;
      uint32_t route_table_id;
-    uint16_t route_count;
      struct vector route_list; /* Contains ecmp_route_list_node */
      struct sset selection_fields;
  };
diff --git a/tests/ovn-macros.at b/tests/ovn-macros.at
index 7a19c0130..1f36f127a 100644
--- a/tests/ovn-macros.at
+++ b/tests/ovn-macros.at
@@ -1568,6 +1568,40 @@ ovn_strip_lflows() {
       sed 's/table=[[0-9]]\{1,2\}\s\?/table=??/g' | sort
  }

  
+# The ECMP group and member ids are derived from hashes of the routes, replace

+# them with <gid> and <mid>.
+ovn_strip_ecmp_ids() {
+    sed -e 's/\(reg8\[[0\.\.15\]] =\{1,2\} \)[[1-9]][[0-9]]*/\1<gid>/' \
+        -e 's/\(reg8\[[16\.\.31\]] =\{1,2\} \)[[1-9]][[0-9]]*/\1<mid>/' \
+        -e ':a' \
+        -e 's/\(select[[^)]]*[[(, ]]\)[[0-9]][[0-9]]*/\1<mid>/' \
+        -e 'ta'
+}
+
+# ecmp_ids LR PREFIX: prints the group id and the comma separated member ids
+# of the ECMP group of PREFIX on LR, and fails if the members listed in its
+# select() action are not the ones that have a flow in lr_in_ip_routing_ecmp.
+ecmp_ids() {
+    local flows gid sel mids
+    flows=$(ovn-sbctl dump-flows $1)
+    gid=$(echo "$flows" | sed -n "s|.*ip[[46]]\.dst == $2).*reg8\[[0\.\.15\]] = 
\([[0-9]]*\);.*|\1|p")
+    sel=$(echo "$flows" | sed -n "s|.*ip[[46]]\.dst == 
$2).*select(\([[^)]]*\));.*|\1|p" | tr -d ' ')
+    mids=$(echo "$flows" | sed -n "s|.*(reg8\[[0\.\.15\]] == $gid && 
reg8\[[16\.\.31\]] == \([[0-9]]*\)).*|\1|p" | sort -n | paste -sd, -)
+    if test -z "$gid" || test "$sel" != "$mids"; then
+        echo "$2: group '$gid', select($sel) != members($mids)" >&2
+        return 1
+    fi
+    echo "$gid $sel"
+}
+
+# ecmp_member_id LR PREFIX NEXTHOP: prints the member id of NEXTHOP in the
+# ECMP group of PREFIX on LR.
+ecmp_member_id() {
+    local gid
+    gid=$(ecmp_ids $1 $2 | cut -d' ' -f1)
+    ovn-sbctl dump-flows $1 | sed -n "s|.*(reg8\[[0\.\.15\]] == $gid && 
reg8\[[16\.\.31\]] == \([[0-9]]*\)), action=(reg0 = $3;.*|\1|p"
+}
+
  ovn_strip_collector_set() {
      sed 's/collector_set=[[0-9]]*,\?/collector_set=??,/g'
  }
diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
index 7a0a400c4..e00627e42 100644
--- a/tests/ovn-northd.at
+++ b/tests/ovn-northd.at
@@ -7799,18 +7799,18 @@ check_row_count ECMP_Nexthop 1

  
  ovn-sbctl dump-flows lr0 > lr0flows
  
-AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | ovn_strip_lflows], [0], [dnl

+AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | ovn_strip_ecmp_ids | 
ovn_strip_lflows], [0], [dnl
    table=??(lr_in_ip_routing   ), priority=0    , match=(1), action=(drop;)
    table=??(lr_in_ip_routing   ), priority=10300, match=(ct_mark.ecmp_reply_port == 1 && reg7 == 
0 && ip4.dst == 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; eth.src = 00:00:20:20:12:13; 
reg5 = 192.168.0.1; outport = "lr0-public"; next;)
    table=??(lr_in_ip_routing   ), priority=10550, match=(nd_rs || nd_ra), 
action=(drop;)
    table=??(lr_in_ip_routing   ), priority=1842 , match=(ip4.dst == 192.168.0.0/24), 
action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg5 = 192.168.0.1; eth.src = 
00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 1; reg9[[9]] = 1; 
next;)
-  table=??(lr_in_ip_routing   ), priority=1936 , match=(reg7 == 0 && ip4.dst 
== 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; reg8[[16..31]] 
= 1; next;)
+  table=??(lr_in_ip_routing   ), priority=1936 , match=(reg7 == 0 && ip4.dst == 
1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = <gid>; reg8[[16..31]] = 
<mid>; next;)
    table=??(lr_in_ip_routing   ), priority=2322 , match=(inport == "lr0-public" && 
ip6.dst == fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; xxreg0 = ip6.dst; xxreg1 = 
fe80::200:20ff:fe20:1213; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 
1; reg9[[9]] = 0; next;)
  ])

  
-AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | ovn_strip_lflows], [0], [dnl

+AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | ovn_strip_ecmp_ids | 
ovn_strip_lflows], [0], [dnl
    table=??(lr_in_ip_routing_ecmp), priority=0    , match=(1), action=(drop;)
-  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == 1 && 
reg8[[16..31]] == 1), action=(reg0 = 192.168.0.10; reg5 = 192.168.0.1; eth.src = 
00:00:20:20:12:13; outport = "lr0-public"; reg9[[9]] = 1; reg9[[10]] = 1; next;)
+  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == <gid> && 
reg8[[16..31]] == <mid>), action=(reg0 = 192.168.0.10; reg5 = 192.168.0.1; eth.src = 00:00:20:20:12:13; 
outport = "lr0-public"; reg9[[9]] = 1; reg9[[10]] = 1; next;)
    table=??(lr_in_ip_routing_ecmp), priority=150  , match=(reg8[[0..15]] == 
0), action=(next;)
  ])

  
@@ -7825,18 +7825,18 @@ check_column 192.168.0.10 ECMP_Nexthop nexthop nexthop=192.168.0.10

  check_column 192.168.0.20 ECMP_Nexthop nexthop nexthop=192.168.0.20

  
  ovn-sbctl dump-flows lr0 > lr0flows

-AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | ovn_strip_lflows], [0], [dnl
+AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | ovn_strip_ecmp_ids | 
ovn_strip_lflows], [0], [dnl
    table=??(lr_in_ip_routing   ), priority=0    , match=(1), action=(drop;)
    table=??(lr_in_ip_routing   ), priority=10300, match=(ct_mark.ecmp_reply_port == 1 && reg7 == 
0 && ip4.dst == 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; eth.src = 00:00:20:20:12:13; 
reg5 = 192.168.0.1; outport = "lr0-public"; next;)
    table=??(lr_in_ip_routing   ), priority=10550, match=(nd_rs || nd_ra), 
action=(drop;)
    table=??(lr_in_ip_routing   ), priority=1842 , match=(ip4.dst == 192.168.0.0/24), 
action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg5 = 192.168.0.1; eth.src = 
00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 1; reg9[[9]] = 1; 
next;)
-  table=??(lr_in_ip_routing   ), priority=1936 , match=(reg7 == 0 && ip4.dst 
== 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; reg8[[16..31]] 
= select(1, 2);)
+  table=??(lr_in_ip_routing   ), priority=1936 , match=(reg7 == 0 && ip4.dst == 1.0.0.1/32), 
action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, 
<mid>);)
    table=??(lr_in_ip_routing   ), priority=2322 , match=(inport == "lr0-public" && 
ip6.dst == fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; xxreg0 = ip6.dst; xxreg1 = 
fe80::200:20ff:fe20:1213; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 
1; reg9[[9]] = 0; next;)
  ])
-AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed 
's/192\.168\.0\..0/192.168.0.??/' | ovn_strip_lflows], [0], [dnl
+AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed 
's/192\.168\.0\..0/192.168.0.??/' | ovn_strip_ecmp_ids | ovn_strip_lflows], [0], [dnl
    table=??(lr_in_ip_routing_ecmp), priority=0    , match=(1), action=(drop;)
-  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == 1 && 
reg8[[16..31]] == 1), action=(reg0 = 192.168.0.??; reg5 = 192.168.0.1; eth.src = 
00:00:20:20:12:13; outport = "lr0-public"; reg9[[9]] = 1; reg9[[10]] = 1; next;)
-  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == 1 && 
reg8[[16..31]] == 2), action=(reg0 = 192.168.0.??; reg5 = 192.168.0.1; eth.src = 
00:00:20:20:12:13; outport = "lr0-public"; reg9[[9]] = 1; reg9[[10]] = 1; next;)
+  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == <gid> && 
reg8[[16..31]] == <mid>), action=(reg0 = 192.168.0.??; reg5 = 192.168.0.1; eth.src = 00:00:20:20:12:13; 
outport = "lr0-public"; reg9[[9]] = 1; reg9[[10]] = 1; next;)
+  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == <gid> && 
reg8[[16..31]] == <mid>), action=(reg0 = 192.168.0.??; reg5 = 192.168.0.1; eth.src = 00:00:20:20:12:13; 
outport = "lr0-public"; reg9[[9]] = 1; reg9[[10]] = 1; next;)
    table=??(lr_in_ip_routing_ecmp), priority=150  , match=(reg8[[0..15]] == 
0), action=(next;)
  ])

  
@@ -7867,18 +7867,18 @@ check_column 192.168.0.10 ECMP_Nexthop nexthop nexthop=192.168.0.10

  check_column 192.168.0.20 ECMP_Nexthop nexthop nexthop=192.168.0.20

  
  ovn-sbctl dump-flows lr0 > lr0flows

-AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | ovn_strip_lflows], [0], [dnl
+AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | ovn_strip_ecmp_ids | 
ovn_strip_lflows], [0], [dnl
    table=??(lr_in_ip_routing   ), priority=0    , match=(1), action=(drop;)
    table=??(lr_in_ip_routing   ), priority=10300, match=(ct_mark.ecmp_reply_port == 1 && reg7 == 
0 && ip4.dst == 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; eth.src = 00:00:20:20:12:13; 
reg5 = 192.168.0.1; outport = "lr0-public"; next;)
    table=??(lr_in_ip_routing   ), priority=10550, match=(nd_rs || nd_ra), 
action=(drop;)
    table=??(lr_in_ip_routing   ), priority=1842 , match=(ip4.dst == 192.168.0.0/24), 
action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg5 = 192.168.0.1; eth.src = 
00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 1; reg9[[9]] = 1; 
next;)
-  table=??(lr_in_ip_routing   ), priority=1936 , match=(reg7 == 0 && ip4.dst 
== 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; reg8[[16..31]] 
= select(1, 2);)
+  table=??(lr_in_ip_routing   ), priority=1936 , match=(reg7 == 0 && ip4.dst == 1.0.0.1/32), 
action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, 
<mid>);)
    table=??(lr_in_ip_routing   ), priority=2322 , match=(inport == "lr0-public" && 
ip6.dst == fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; xxreg0 = ip6.dst; xxreg1 = 
fe80::200:20ff:fe20:1213; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 
1; reg9[[9]] = 0; next;)
  ])
-AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed 
's/192\.168\.0\..0/192.168.0.??/' | ovn_strip_lflows], [0], [dnl
+AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed 
's/192\.168\.0\..0/192.168.0.??/' | ovn_strip_ecmp_ids | ovn_strip_lflows], [0], [dnl
    table=??(lr_in_ip_routing_ecmp), priority=0    , match=(1), action=(drop;)
-  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == 1 && 
reg8[[16..31]] == 1), action=(reg0 = 192.168.0.??; reg5 = 192.168.0.1; eth.src = 
00:00:20:20:12:13; outport = "lr0-public"; reg9[[9]] = 1; reg9[[10]] = 1; next;)
-  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == 1 && 
reg8[[16..31]] == 2), action=(reg0 = 192.168.0.??; reg5 = 192.168.0.1; eth.src = 
00:00:20:20:12:13; outport = "lr0-public"; reg9[[9]] = 1; reg9[[10]] = 1; next;)
+  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == <gid> && 
reg8[[16..31]] == <mid>), action=(reg0 = 192.168.0.??; reg5 = 192.168.0.1; eth.src = 00:00:20:20:12:13; 
outport = "lr0-public"; reg9[[9]] = 1; reg9[[10]] = 1; next;)
+  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == <gid> && 
reg8[[16..31]] == <mid>), action=(reg0 = 192.168.0.??; reg5 = 192.168.0.1; eth.src = 00:00:20:20:12:13; 
outport = "lr0-public"; reg9[[9]] = 1; reg9[[10]] = 1; next;)
    table=??(lr_in_ip_routing_ecmp), priority=150  , match=(reg8[[0..15]] == 
0), action=(next;)
  ])

  
@@ -7916,8 +7916,8 @@ check ovn-nbctl set logical_router_static_route $route2_uuid selection_fields="i
  
  check ovn-nbctl --wait=sb sync

  ovn-sbctl dump-flows lr0 > lr0flows
-AT_CHECK([grep -e "(lr_in_ip_routing   ).*3.3.0.0" lr0flows | sed 
's/table=../table=??/' | sort], [0], [dnl
-  table=??(lr_in_ip_routing   ), priority=1744 , match=(reg7 == 0 && ip4.dst == 
3.3.0.0/16), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; reg8[[16..31]] = 
select(values=(1, 2); hash_fields="ip_dst,ip_proto,ip_src");)
+AT_CHECK([grep -e "(lr_in_ip_routing   ).*3.3.0.0" lr0flows | 
ovn_strip_ecmp_ids | sed 's/table=../table=??/' | sort], [0], [dnl
+  table=??(lr_in_ip_routing   ), priority=1744 , match=(reg7 == 0 && ip4.dst == 3.3.0.0/16), 
action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = <gid>; reg8[[16..31]] = select(values=(<mid>, 
<mid>); hash_fields="ip_dst,ip_proto,ip_src");)
  ])

  
  check ovn-nbctl set logical_router_static_route $route1_uuid selection_fields="ip_src,ip_dst,tp_src,tp_dst"

@@ -7925,11 +7925,11 @@ check ovn-nbctl set logical_router_static_route $route2_uuid 
selection_fields="i

  
  check ovn-nbctl --wait=sb sync

  ovn-sbctl dump-flows lr0 > lr0flows
-AT_CHECK([grep -e "(lr_in_ip_routing   ).*3.3.0.0" lr0flows | sed 
's/table=../table=??/' | sort], [0], [dnl
-  table=??(lr_in_ip_routing   ), priority=1744 , match=(reg7 == 0 && ip4.dst == 
3.3.0.0/16), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; reg8[[16..31]] = 
select(values=(1, 2); hash_fields="ip_dst,ip_proto,ip_src");)
-  table=??(lr_in_ip_routing   ), priority=1745 , match=(reg7 == 0 && ip4.dst == 3.3.0.0/16 
&& sctp), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; reg8[[16..31]] = 
select(values=(1, 2); hash_fields="ip_dst,ip_proto,ip_src,sctp_dst,sctp_src");)
-  table=??(lr_in_ip_routing   ), priority=1745 , match=(reg7 == 0 && ip4.dst == 3.3.0.0/16 
&& tcp), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; reg8[[16..31]] = 
select(values=(1, 2); hash_fields="ip_dst,ip_proto,ip_src,tcp_dst,tcp_src");)
-  table=??(lr_in_ip_routing   ), priority=1745 , match=(reg7 == 0 && ip4.dst == 3.3.0.0/16 
&& udp), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; reg8[[16..31]] = 
select(values=(1, 2); hash_fields="ip_dst,ip_proto,ip_src,udp_dst,udp_src");)
+AT_CHECK([grep -e "(lr_in_ip_routing   ).*3.3.0.0" lr0flows | 
ovn_strip_ecmp_ids | sed 's/table=../table=??/' | sort], [0], [dnl
+  table=??(lr_in_ip_routing   ), priority=1744 , match=(reg7 == 0 && ip4.dst == 3.3.0.0/16), 
action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = <gid>; reg8[[16..31]] = select(values=(<mid>, 
<mid>); hash_fields="ip_dst,ip_proto,ip_src");)
+  table=??(lr_in_ip_routing   ), priority=1745 , match=(reg7 == 0 && ip4.dst == 3.3.0.0/16 && sctp), 
action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = <gid>; reg8[[16..31]] = select(values=(<mid>, 
<mid>); hash_fields="ip_dst,ip_proto,ip_src,sctp_dst,sctp_src");)
+  table=??(lr_in_ip_routing   ), priority=1745 , match=(reg7 == 0 && ip4.dst == 3.3.0.0/16 && tcp), 
action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = <gid>; reg8[[16..31]] = select(values=(<mid>, 
<mid>); hash_fields="ip_dst,ip_proto,ip_src,tcp_dst,tcp_src");)
+  table=??(lr_in_ip_routing   ), priority=1745 , match=(reg7 == 0 && ip4.dst == 3.3.0.0/16 && udp), 
action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = <gid>; reg8[[16..31]] = select(values=(<mid>, 
<mid>); hash_fields="ip_dst,ip_proto,ip_src,udp_dst,udp_src");)
  ])

  
  AT_CHECK([grep -e "lr_in_ecmp_stateful_egr" lr0flows | ovn_strip_lflows], [0], [dnl

@@ -17440,19 +17440,19 @@ datapath=$(fetch_column datapath_binding _uuid 
external_ids:name=lr0)

  
  # Validate the routes.

  ovn-sbctl dump-flows lr0 > lr0flows
-AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | ovn_strip_lflows], [0], [dnl
+AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | ovn_strip_ecmp_ids | 
ovn_strip_lflows], [0], [dnl
    table=??(lr_in_ip_routing   ), priority=0    , match=(1), action=(drop;)
    table=??(lr_in_ip_routing   ), priority=10550, match=(nd_rs || nd_ra), 
action=(drop;)
-  table=??(lr_in_ip_routing   ), priority=1840 , match=(reg7 == 0 && ip4.dst 
== 192.168.0.0/24), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; 
reg8[[16..31]] = select(1, 2);)
+  table=??(lr_in_ip_routing   ), priority=1840 , match=(reg7 == 0 && ip4.dst == 192.168.0.0/24), 
action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, 
<mid>);)
    table=??(lr_in_ip_routing   ), priority=1842 , match=(ip4.dst == 10.0.0.0/24), 
action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg5 = 10.0.0.1; eth.src = 
00:00:00:00:ff:01; outport = "lr0-sw0"; flags.loopback = 1; reg9[[9]] = 1; 
next;)
    table=??(lr_in_ip_routing   ), priority=1842 , match=(ip4.dst == 10.0.1.0/24), 
action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg5 = 10.0.1.1; eth.src = 
00:00:00:00:ff:02; outport = "lr0-sw1"; flags.loopback = 1; reg9[[9]] = 1; 
next;)
    table=??(lr_in_ip_routing   ), priority=2322 , match=(inport == "lr0-sw0" && ip6.dst 
== fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; xxreg0 = ip6.dst; xxreg1 = fe80::200:ff:fe00:ff01; 
eth.src = 00:00:00:00:ff:01; outport = "lr0-sw0"; flags.loopback = 1; reg9[[9]] = 0; next;)
    table=??(lr_in_ip_routing   ), priority=2322 , match=(inport == "lr0-sw1" && ip6.dst 
== fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; xxreg0 = ip6.dst; xxreg1 = fe80::200:ff:fe00:ff02; 
eth.src = 00:00:00:00:ff:02; outport = "lr0-sw1"; flags.loopback = 1; reg9[[9]] = 0; next;)
  ])
-AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed -e 
's/10\.0\..\./10.0.??./g' -e 's/lr0-sw./lr0-sw??/' -e 's/00:ff:0./00:ff:0?/' | 
ovn_strip_lflows], [0], [dnl
+AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed -e 
's/10\.0\..\./10.0.??./g' -e 's/lr0-sw./lr0-sw??/' -e 's/00:ff:0./00:ff:0?/' | 
ovn_strip_ecmp_ids | ovn_strip_lflows], [0], [dnl
    table=??(lr_in_ip_routing_ecmp), priority=0    , match=(1), action=(drop;)
-  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == 1 && 
reg8[[16..31]] == 1), action=(reg0 = 10.0.??.10; reg5 = 10.0.??.1; eth.src = 00:00:00:00:ff:0?; 
outport = "lr0-sw??"; reg9[[9]] = 1; next;)
-  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == 1 && 
reg8[[16..31]] == 2), action=(reg0 = 10.0.??.10; reg5 = 10.0.??.1; eth.src = 00:00:00:00:ff:0?; 
outport = "lr0-sw??"; reg9[[9]] = 1; next;)
+  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == <gid> && 
reg8[[16..31]] == <mid>), action=(reg0 = 10.0.??.10; reg5 = 10.0.??.1; eth.src = 00:00:00:00:ff:0?; 
outport = "lr0-sw??"; reg9[[9]] = 1; next;)
+  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == <gid> && 
reg8[[16..31]] == <mid>), action=(reg0 = 10.0.??.10; reg5 = 10.0.??.1; eth.src = 00:00:00:00:ff:0?; 
outport = "lr0-sw??"; reg9[[9]] = 1; next;)
    table=??(lr_in_ip_routing_ecmp), priority=150  , match=(reg8[[0..15]] == 
0), action=(next;)
  ])

  
@@ -17466,20 +17466,20 @@ check ovn-nbctl --wait=sb sync

  check_row_count Advertised_Route 4
  check_row_count Learned_Route 1
  ovn-sbctl dump-flows lr0 > lr0flows
-AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | ovn_strip_lflows], [0], [dnl
+AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | ovn_strip_ecmp_ids | 
ovn_strip_lflows], [0], [dnl
    table=??(lr_in_ip_routing   ), priority=0    , match=(1), action=(drop;)
    table=??(lr_in_ip_routing   ), priority=10550, match=(nd_rs || nd_ra), 
action=(drop;)
    table=??(lr_in_ip_routing   ), priority=1838 , match=(reg7 == 0 && ip4.dst == 
192.168.1.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 10.0.0.20; reg5 = 10.0.0.1; eth.src 
= 00:00:00:00:ff:01; outport = "lr0-sw0"; flags.loopback = 1; reg9[[9]] = 1; next;)
-  table=??(lr_in_ip_routing   ), priority=1840 , match=(reg7 == 0 && ip4.dst 
== 192.168.0.0/24), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; 
reg8[[16..31]] = select(1, 2);)
+  table=??(lr_in_ip_routing   ), priority=1840 , match=(reg7 == 0 && ip4.dst == 192.168.0.0/24), 
action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, 
<mid>);)
    table=??(lr_in_ip_routing   ), priority=1842 , match=(ip4.dst == 10.0.0.0/24), 
action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg5 = 10.0.0.1; eth.src = 
00:00:00:00:ff:01; outport = "lr0-sw0"; flags.loopback = 1; reg9[[9]] = 1; 
next;)
    table=??(lr_in_ip_routing   ), priority=1842 , match=(ip4.dst == 10.0.1.0/24), 
action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg5 = 10.0.1.1; eth.src = 
00:00:00:00:ff:02; outport = "lr0-sw1"; flags.loopback = 1; reg9[[9]] = 1; 
next;)
    table=??(lr_in_ip_routing   ), priority=2322 , match=(inport == "lr0-sw0" && ip6.dst 
== fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; xxreg0 = ip6.dst; xxreg1 = fe80::200:ff:fe00:ff01; 
eth.src = 00:00:00:00:ff:01; outport = "lr0-sw0"; flags.loopback = 1; reg9[[9]] = 0; next;)
    table=??(lr_in_ip_routing   ), priority=2322 , match=(inport == "lr0-sw1" && ip6.dst 
== fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; xxreg0 = ip6.dst; xxreg1 = fe80::200:ff:fe00:ff02; 
eth.src = 00:00:00:00:ff:02; outport = "lr0-sw1"; flags.loopback = 1; reg9[[9]] = 0; next;)
  ])
-AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed -e 
's/10\.0\..\./10.0.??./g' -e 's/lr0-sw./lr0-sw??/' -e 's/00:ff:0./00:ff:0?/' | 
ovn_strip_lflows], [0], [dnl
+AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed -e 
's/10\.0\..\./10.0.??./g' -e 's/lr0-sw./lr0-sw??/' -e 's/00:ff:0./00:ff:0?/' | 
ovn_strip_ecmp_ids | ovn_strip_lflows], [0], [dnl
    table=??(lr_in_ip_routing_ecmp), priority=0    , match=(1), action=(drop;)
-  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == 1 && 
reg8[[16..31]] == 1), action=(reg0 = 10.0.??.10; reg5 = 10.0.??.1; eth.src = 00:00:00:00:ff:0?; 
outport = "lr0-sw??"; reg9[[9]] = 1; next;)
-  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == 1 && 
reg8[[16..31]] == 2), action=(reg0 = 10.0.??.10; reg5 = 10.0.??.1; eth.src = 00:00:00:00:ff:0?; 
outport = "lr0-sw??"; reg9[[9]] = 1; next;)
+  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == <gid> && 
reg8[[16..31]] == <mid>), action=(reg0 = 10.0.??.10; reg5 = 10.0.??.1; eth.src = 00:00:00:00:ff:0?; 
outport = "lr0-sw??"; reg9[[9]] = 1; next;)
+  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == <gid> && 
reg8[[16..31]] == <mid>), action=(reg0 = 10.0.??.10; reg5 = 10.0.??.1; eth.src = 00:00:00:00:ff:0?; 
outport = "lr0-sw??"; reg9[[9]] = 1; next;)
    table=??(lr_in_ip_routing_ecmp), priority=150  , match=(reg8[[0..15]] == 
0), action=(next;)
  ])

  
@@ -17494,22 +17494,22 @@ check ovn-nbctl --wait=sb sync

  check_row_count Advertised_Route 4
  check_row_count Learned_Route 2
  ovn-sbctl dump-flows lr0 > lr0flows
-AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | ovn_strip_lflows], [0], [dnl
+AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | ovn_strip_ecmp_ids | 
ovn_strip_lflows], [0], [dnl
    table=??(lr_in_ip_routing   ), priority=0    , match=(1), action=(drop;)
    table=??(lr_in_ip_routing   ), priority=10550, match=(nd_rs || nd_ra), 
action=(drop;)
-  table=??(lr_in_ip_routing   ), priority=1838 , match=(reg7 == 0 && ip4.dst 
== 192.168.1.0/24), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 2; 
reg8[[16..31]] = select(1, 2);)
-  table=??(lr_in_ip_routing   ), priority=1840 , match=(reg7 == 0 && ip4.dst 
== 192.168.0.0/24), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; 
reg8[[16..31]] = select(1, 2);)
+  table=??(lr_in_ip_routing   ), priority=1838 , match=(reg7 == 0 && ip4.dst == 192.168.1.0/24), 
action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, 
<mid>);)
+  table=??(lr_in_ip_routing   ), priority=1840 , match=(reg7 == 0 && ip4.dst == 192.168.0.0/24), 
action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, 
<mid>);)
    table=??(lr_in_ip_routing   ), priority=1842 , match=(ip4.dst == 10.0.0.0/24), 
action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg5 = 10.0.0.1; eth.src = 
00:00:00:00:ff:01; outport = "lr0-sw0"; flags.loopback = 1; reg9[[9]] = 1; 
next;)
    table=??(lr_in_ip_routing   ), priority=1842 , match=(ip4.dst == 10.0.1.0/24), 
action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg5 = 10.0.1.1; eth.src = 
00:00:00:00:ff:02; outport = "lr0-sw1"; flags.loopback = 1; reg9[[9]] = 1; 
next;)
    table=??(lr_in_ip_routing   ), priority=2322 , match=(inport == "lr0-sw0" && ip6.dst 
== fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; xxreg0 = ip6.dst; xxreg1 = fe80::200:ff:fe00:ff01; 
eth.src = 00:00:00:00:ff:01; outport = "lr0-sw0"; flags.loopback = 1; reg9[[9]] = 0; next;)
    table=??(lr_in_ip_routing   ), priority=2322 , match=(inport == "lr0-sw1" && ip6.dst 
== fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; xxreg0 = ip6.dst; xxreg1 = fe80::200:ff:fe00:ff02; 
eth.src = 00:00:00:00:ff:02; outport = "lr0-sw1"; flags.loopback = 1; reg9[[9]] = 0; next;)
  ])
-AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed -e 
's/10\.0\..\./10.0.??./g' -e 's/lr0-sw./lr0-sw??/' -e 's/00:ff:0./00:ff:0?/' | 
ovn_strip_lflows], [0], [dnl
+AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed -e 
's/10\.0\..\./10.0.??./g' -e 's/lr0-sw./lr0-sw??/' -e 's/00:ff:0./00:ff:0?/' | 
ovn_strip_ecmp_ids | ovn_strip_lflows], [0], [dnl
    table=??(lr_in_ip_routing_ecmp), priority=0    , match=(1), action=(drop;)
-  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == 1 && 
reg8[[16..31]] == 1), action=(reg0 = 10.0.??.10; reg5 = 10.0.??.1; eth.src = 00:00:00:00:ff:0?; 
outport = "lr0-sw??"; reg9[[9]] = 1; next;)
-  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == 1 && 
reg8[[16..31]] == 2), action=(reg0 = 10.0.??.10; reg5 = 10.0.??.1; eth.src = 00:00:00:00:ff:0?; 
outport = "lr0-sw??"; reg9[[9]] = 1; next;)
-  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == 2 && 
reg8[[16..31]] == 1), action=(reg0 = 10.0.??.20; reg5 = 10.0.??.1; eth.src = 00:00:00:00:ff:0?; 
outport = "lr0-sw??"; reg9[[9]] = 1; next;)
-  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == 2 && 
reg8[[16..31]] == 2), action=(reg0 = 10.0.??.20; reg5 = 10.0.??.1; eth.src = 00:00:00:00:ff:0?; 
outport = "lr0-sw??"; reg9[[9]] = 1; next;)
+  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == <gid> && 
reg8[[16..31]] == <mid>), action=(reg0 = 10.0.??.10; reg5 = 10.0.??.1; eth.src = 00:00:00:00:ff:0?; 
outport = "lr0-sw??"; reg9[[9]] = 1; next;)
+  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == <gid> && 
reg8[[16..31]] == <mid>), action=(reg0 = 10.0.??.10; reg5 = 10.0.??.1; eth.src = 00:00:00:00:ff:0?; 
outport = "lr0-sw??"; reg9[[9]] = 1; next;)
+  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == <gid> && 
reg8[[16..31]] == <mid>), action=(reg0 = 10.0.??.20; reg5 = 10.0.??.1; eth.src = 00:00:00:00:ff:0?; 
outport = "lr0-sw??"; reg9[[9]] = 1; next;)
+  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == <gid> && 
reg8[[16..31]] == <mid>), action=(reg0 = 10.0.??.20; reg5 = 10.0.??.1; eth.src = 00:00:00:00:ff:0?; 
outport = "lr0-sw??"; reg9[[9]] = 1; next;)
    table=??(lr_in_ip_routing_ecmp), priority=150  , match=(reg8[[0..15]] == 
0), action=(next;)
  ])

  
@@ -17536,12 +17536,12 @@ check ovn-nbctl --wait=sb sync

  check_row_count Advertised_Route 4
  check_row_count Learned_Route 4
  ovn-sbctl dump-flows lr0 > lr0flows
-AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | sed -e 's/reg8\[[0..15\]] = 
[[123]]/reg8\[[0..15\]] = ??/' | ovn_strip_lflows], [0], [dnl
+AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | ovn_strip_ecmp_ids | 
ovn_strip_lflows], [0], [dnl
    table=??(lr_in_ip_routing   ), priority=0    , match=(1), action=(drop;)
    table=??(lr_in_ip_routing   ), priority=10550, match=(nd_rs || nd_ra), 
action=(drop;)
-  table=??(lr_in_ip_routing   ), priority=1838 , match=(reg7 == 0 && ip4.dst 
== 192.168.0.0/24), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = ??; 
reg8[[16..31]] = select(1, 2);)
-  table=??(lr_in_ip_routing   ), priority=1838 , match=(reg7 == 0 && ip4.dst 
== 192.168.1.0/24), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = ??; 
reg8[[16..31]] = select(1, 2);)
-  table=??(lr_in_ip_routing   ), priority=1840 , match=(reg7 == 0 && ip4.dst 
== 192.168.0.0/24), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = ??; 
reg8[[16..31]] = select(1, 2);)
+  table=??(lr_in_ip_routing   ), priority=1838 , match=(reg7 == 0 && ip4.dst == 192.168.0.0/24), 
action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, 
<mid>);)
+  table=??(lr_in_ip_routing   ), priority=1838 , match=(reg7 == 0 && ip4.dst == 192.168.1.0/24), 
action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, 
<mid>);)
+  table=??(lr_in_ip_routing   ), priority=1840 , match=(reg7 == 0 && ip4.dst == 192.168.0.0/24), 
action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, 
<mid>);)
    table=??(lr_in_ip_routing   ), priority=1842 , match=(ip4.dst == 10.0.0.0/24), 
action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg5 = 10.0.0.1; eth.src = 
00:00:00:00:ff:01; outport = "lr0-sw0"; flags.loopback = 1; reg9[[9]] = 1; 
next;)
    table=??(lr_in_ip_routing   ), priority=1842 , match=(ip4.dst == 10.0.1.0/24), 
action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg5 = 10.0.1.1; eth.src = 
00:00:00:00:ff:02; outport = "lr0-sw1"; flags.loopback = 1; reg9[[9]] = 1; 
next;)
    table=??(lr_in_ip_routing   ), priority=2322 , match=(inport == "lr0-sw0" && ip6.dst 
== fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; xxreg0 = ip6.dst; xxreg1 = fe80::200:ff:fe00:ff01; 
eth.src = 00:00:00:00:ff:01; outport = "lr0-sw0"; flags.loopback = 1; reg9[[9]] = 0; next;)
@@ -17586,13 +17586,13 @@ check_uuid ovn-sbctl create Learned_Route \
  check ovn-nbctl --wait=sb sync

  
  ovn-sbctl dump-flows lr0 > lr0flows

-AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | grep -F "42.42.42.42" | 
ovn_strip_lflows], [0], [dnl
-  table=??(lr_in_ip_routing   ), priority=1934 , match=(reg7 == 0 && ip4.dst 
== 42.42.42.42/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; 
reg8[[16..31]] = select(1, 2);)
+AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | grep -F "42.42.42.42" | 
ovn_strip_ecmp_ids | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_ip_routing   ), priority=1934 , match=(reg7 == 0 && ip4.dst == 42.42.42.42/32), 
action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, 
<mid>);)
  ])

  
-AT_CHECK([grep "lr_in_ip_routing_ecmp" lr0flows | grep "42" | sed -e 's/reg8\[[16..31\]] == [[12]]/reg8\[[16..31\]] == ??/g' | ovn_strip_lflows], [0], [dnl

-  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == 1 && 
reg8[[16..31]] == ??), action=(reg0 = 192.168.2.42; eth.src = 00:00:00:00:ff:02; outport = 
"lrp2"; reg9[[9]] = 1; next;)
-  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == 1 && 
reg8[[16..31]] == ??), action=(reg0 = 192.168.3.42; eth.src = 00:00:00:00:ff:03; outport = 
"lrp3"; reg9[[9]] = 1; next;)
+AT_CHECK([grep "lr_in_ip_routing_ecmp" lr0flows | grep "42" | 
ovn_strip_ecmp_ids | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == <gid> && 
reg8[[16..31]] == <mid>), action=(reg0 = 192.168.2.42; eth.src = 00:00:00:00:ff:02; outport = 
"lrp2"; reg9[[9]] = 1; next;)
+  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == <gid> && 
reg8[[16..31]] == <mid>), action=(reg0 = 192.168.3.42; eth.src = 00:00:00:00:ff:03; outport = 
"lrp3"; reg9[[9]] = 1; next;)
  ])

  
  OVN_CLEANUP_NORTHD

@@ -19350,8 +19350,8 @@ check_row_count Advertised_Route 1 \
  # One ECMP selection flow covers the prefix and two member flows carry the
  # respective neighbor nexthops.
  ovn-sbctl lflow-list lr0 > lr0_flows
-AT_CHECK([grep 'lr_in_ip_routing.*172.16.1.10/32' lr0_flows | \
-          grep -c 'select(1, 2)'], [0], [1
+AT_CHECK([grep 'lr_in_ip_routing.*172.16.1.10/32' lr0_flows | 
ovn_strip_ecmp_ids | \
+          grep -c 'select(<mid>, <mid>)'], [0], [1
  ])
  AT_CHECK([grep 'lr_in_ip_routing_ecmp' lr0_flows | \
            grep -E -c 'reg0 = 10\.0\.0\.[[12]];'], [0], [2
@@ -19370,6 +19370,85 @@ OVN_CLEANUP_NORTHD
  AT_CLEANUP
  ])

  
+OVN_FOR_EACH_NORTHD_NO_HV([

+AT_SETUP([dynamic-routing - NAT and LB reachable over several paths use ECMP])
+AT_KEYWORDS([dynamic-routing ecmp])
+ovn_start
+
+# lr0 reaches lr1 over the switches "a" and "b" and redistributes NAT and LB
+# addresses on both ports.  The NAT and the LB of lr1 then produce one
+# forwarding route per path on lr0.  The two routes of each ECMP group share
+# their source row and only differ in the nexthop and output port, they must
+# still get distinct ids that don't depend on the processing order.
+
+check ovn-nbctl lr-add lr0 -- set Logical_Router lr0 \
+    options:dynamic-routing=true options:chassis=hv1
+check ovn-nbctl lr-add lr1 -- set Logical_Router lr1 options:chassis=hv1
+i=0
+for s in a b; do
+    i=$((i + 1))
+    check ovn-nbctl ls-add $s
+    check ovn-nbctl lrp-add lr0 lr0-$s 00:00:00:00:0$i:01 10.0.$i.1/24
+    check ovn-nbctl lrp-set-options lr0-$s dynamic-routing-redistribute=lb,nat
+    check ovn-nbctl lsp-add-router-port $s $s-lr0 lr0-$s
+    check ovn-nbctl lrp-add lr1 lr1-$s 00:00:00:00:0$i:02 10.0.$i.2/24
+    check ovn-nbctl lsp-add-router-port $s $s-lr1 lr1-$s
+done
+check ovn-nbctl lr-nat-add lr1 dnat_and_snat 172.16.1.10 192.168.1.10
+check ovn-nbctl lb-add lb1 172.16.1.20:80 192.168.1.20:80
+check ovn-nbctl --wait=sb lr-lb-add lr1 lb1
+
+ovn-sbctl dump-flows lr0 > lr0flows
+AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | grep "172\.16\.1\." | 
ovn_strip_ecmp_ids | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_ip_routing   ), priority=1935 , match=(ip4.dst == 172.16.1.10/32), 
action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = <gid>; reg8[[16..31]] = 
select(<mid>, <mid>);)
+  table=??(lr_in_ip_routing   ), priority=1935 , match=(ip4.dst == 172.16.1.20/32), 
action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = <gid>; reg8[[16..31]] = 
select(<mid>, <mid>);)
+])
+AT_CHECK([grep "lr_in_ip_routing_ecmp" lr0flows | grep "reg0 = 10\.0\." | 
ovn_strip_ecmp_ids | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == <gid> && 
reg8[[16..31]] == <mid>), action=(reg0 = 10.0.1.2; reg5 = 10.0.1.1; eth.src = 00:00:00:00:01:01; 
outport = "lr0-a"; reg9[[9]] = 1; next;)
+  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == <gid> && 
reg8[[16..31]] == <mid>), action=(reg0 = 10.0.1.2; reg5 = 10.0.1.1; eth.src = 00:00:00:00:01:01; 
outport = "lr0-a"; reg9[[9]] = 1; next;)
+  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == <gid> && 
reg8[[16..31]] == <mid>), action=(reg0 = 10.0.2.2; reg5 = 10.0.2.1; eth.src = 00:00:00:00:02:01; 
outport = "lr0-b"; reg9[[9]] = 1; next;)
+  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == <gid> && 
reg8[[16..31]] == <mid>), action=(reg0 = 10.0.2.2; reg5 = 10.0.2.1; eth.src = 00:00:00:00:02:01; 
outport = "lr0-b"; reg9[[9]] = 1; next;)
+])
+
+dnl Both members of each group come from the same NAT or LB row.
+nat=$(fetch_column nb:NAT _uuid external_ip=172.16.1.10)
+lb=$(fetch_column nb:Load_Balancer _uuid name=lb1)
+check_row_count Logical_Flow 2 external_ids:stage-name=lr_in_ip_routing_ecmp \
+    external_ids:stage-hint=${nat:0:8}
+check_row_count Logical_Flow 2 external_ids:stage-name=lr_in_ip_routing_ecmp \
+    external_ids:stage-hint=${lb:0:8}
+
+dnl ecmp_ids also checks that the members have distinct ids.
+nat_ids=$(ecmp_ids lr0 172.16.1.10/32)
+lb_ids=$(ecmp_ids lr0 172.16.1.20/32)
+AT_CHECK([test -n "$nat_ids" && test -n "$lb_ids"])
+CHECK_NO_CHANGE_AFTER_RECOMPUTE
+
+dnl A third path adds a member to each group without changing the ids of the
+dnl existing members.
+check ovn-nbctl ls-add c
+check ovn-nbctl lrp-add lr0 lr0-c 00:00:00:00:03:01 10.0.3.1/24
+check ovn-nbctl lrp-set-options lr0-c dynamic-routing-redistribute=lb,nat
+check ovn-nbctl lsp-add-router-port c c-lr0 lr0-c
+check ovn-nbctl lrp-add lr1 lr1-c 00:00:00:00:03:02 10.0.3.2/24
+check ovn-nbctl --wait=sb lsp-add-router-port c c-lr1 lr1-c
+CHECK_NO_CHANGE_AFTER_RECOMPUTE
+
+for old in "$nat_ids 172.16.1.10/32" "$lb_ids 172.16.1.20/32"; do
+    set -- $old
+    new=$(ecmp_ids lr0 $3)
+    AT_CHECK([test "${new%% *}" = "$1"])
+    AT_CHECK([echo "${new#* }" | tr , '\n' | wc -l], [0], [3
+])
+    for m in $(echo $2 | tr , ' '); do
+        AT_CHECK([echo "${new#* }" | tr , '\n' | grep -qx $m])
+    done
+done
+
+OVN_CLEANUP_NORTHD
+AT_CLEANUP
+])
+
  OVN_FOR_EACH_NORTHD_NO_HV([
  AT_SETUP([dynamic-routing - LB redistribute uses backend LSP as tracked_port])
  AT_KEYWORDS([dynamic-routing])
@@ -24092,6 +24171,87 @@ done
  OVN_CLEANUP_NORTHD
  AT_CLEANUP

  
+OVN_FOR_EACH_NORTHD_NO_HV([

+AT_SETUP([Static routes - ECMP ids stable across recompute])
+AT_KEYWORDS([ecmp])
+ovn_start
+
+check ovn-nbctl lr-add lr0
+for i in 1 2 3 4; do
+    check ovn-nbctl lrp-add lr0 lr0-sw$i 00:00:00:00:0$i:01 10.0.$i.1/24
+done
+check ovn-nbctl --wait=sb sync
+
+dnl Add the routes in "reverse" order, the ids must not depend on it.
+for p in 192.168.30.0/24 192.168.20.0/24 192.168.10.0/24; do
+    for i in 3 2 1; do
+        check ovn-nbctl --wait=sb --ecmp lr-route-add lr0 $p 10.0.$i.10
+    done
+done
+CHECK_NO_CHANGE_AFTER_RECOMPUTE
+
+AT_CHECK([ovn-sbctl dump-flows lr0 | grep -w lr_in_ip_routing | grep select | 
ovn_strip_ecmp_ids | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_ip_routing   ), priority=1840 , match=(reg7 == 0 && ip4.dst == 192.168.10.0/24), 
action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, 
<mid>, <mid>);)
+  table=??(lr_in_ip_routing   ), priority=1840 , match=(reg7 == 0 && ip4.dst == 192.168.20.0/24), 
action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, 
<mid>, <mid>);)
+  table=??(lr_in_ip_routing   ), priority=1840 , match=(reg7 == 0 && ip4.dst == 192.168.30.0/24), 
action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, 
<mid>, <mid>);)
+])
+g10=$(ecmp_ids lr0 192.168.10.0/24)
+g20=$(ecmp_ids lr0 192.168.20.0/24)
+g30=$(ecmp_ids lr0 192.168.30.0/24)
+AT_CHECK([test -n "$g10" && test -n "$g20" && test -n "$g30"])
+AT_CHECK([printf '%s\n' "$g10" "$g20" "$g30" | cut -d' ' -f1 | sort -u | wc 
-l], [0], [3
+])
+
+dnl Remove a member from the middle of a group with more than 2 members, the
+dnl ids of the other members and groups must not change.
+m2=$(ecmp_member_id lr0 192.168.10.0/24 10.0.2.10)
+check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats
+check ovn-nbctl --wait=sb lr-route-del lr0 192.168.10.0/24 10.0.2.10
+check_engine_compute group_ecmp_route incremental
+CHECK_NO_CHANGE_AFTER_RECOMPUTE
+g10_ids=$(echo "${g10#* }" | tr , '\n' | grep -vx "$m2" | paste -sd, -)
+AT_CHECK_UNQUOTED([ecmp_ids lr0 192.168.10.0/24], [0], [${g10%% *} $g10_ids
+])
+AT_CHECK_UNQUOTED([ecmp_ids lr0 192.168.20.0/24], [0], [$g20
+])
+AT_CHECK_UNQUOTED([ecmp_ids lr0 192.168.30.0/24], [0], [$g30
+])
+g10=$(ecmp_ids lr0 192.168.10.0/24)
+
+dnl Remove a whole group and then add a new one, the ids of the other groups
+dnl must not change and the group ids must stay unique.
+check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats
+check ovn-nbctl --wait=sb lr-route-del lr0 192.168.20.0/24
+check ovn-nbctl --wait=sb --ecmp lr-route-add lr0 192.168.40.0/24 10.0.1.10
+check ovn-nbctl --wait=sb --ecmp lr-route-add lr0 192.168.40.0/24 10.0.4.10
+check_engine_compute group_ecmp_route incremental
+CHECK_NO_CHANGE_AFTER_RECOMPUTE
+AT_CHECK_UNQUOTED([ecmp_ids lr0 192.168.10.0/24], [0], [$g10
+])
+AT_CHECK_UNQUOTED([ecmp_ids lr0 192.168.30.0/24], [0], [$g30
+])
+g40=$(ecmp_ids lr0 192.168.40.0/24)
+AT_CHECK([test -n "$g40"])
+AT_CHECK([printf '%s\n' "$g10" "$g30" "$g40" | cut -d' ' -f1 | sort -u | wc 
-l], [0], [3
+])
+
+dnl Once no member is a discard route anymore the group must stop dropping.
+route=$(ovn-nbctl --bare --columns _uuid find Logical_Router_Static_Route \
+        ip_prefix="192.168.30.0/24" nexthop="10.0.3.10")
+check ovn-nbctl --wait=sb set Logical_Router_Static_Route $route 
nexthop=discard
+AT_CHECK([ovn-sbctl dump-flows lr0 | grep lr_in_ip_routing_ecmp | grep -c 
"drop;"], [0], [4
+])
+check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats
+check ovn-nbctl --wait=sb set Logical_Router_Static_Route $route 
nexthop=10.0.3.10
+check_engine_compute group_ecmp_route incremental
+AT_CHECK([ovn-sbctl dump-flows lr0 | grep lr_in_ip_routing_ecmp | grep -c 
"drop;"], [0], [1
+])
+CHECK_NO_CHANGE_AFTER_RECOMPUTE
+
+OVN_CLEANUP_NORTHD
+AT_CLEANUP
+])
+
  OVN_FOR_EACH_NORTHD_NO_HV([
  AT_SETUP([Static routes - ECMP with discard])
  ovn_start
@@ -24106,14 +24266,14 @@ check ovn-nbctl                                       
                   \
  check ovn-nbctl --wait=sb sync

  
  ovn-sbctl dump-flows lr0 > lr0flows

-AT_CHECK([grep 'lr_in_ip_routing' lr0flows | grep 'select' | 
ovn_strip_lflows], [0], [dnl
-  table=??(lr_in_ip_routing   ), priority=1840 , match=(reg7 == 0 && ip4.dst 
== 192.168.10.0/24), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; 
reg8[[16..31]] = select(1, 2);)
+AT_CHECK([grep 'lr_in_ip_routing' lr0flows | grep 'select' | 
ovn_strip_ecmp_ids | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_ip_routing   ), priority=1840 , match=(reg7 == 0 && ip4.dst == 
192.168.10.0/24), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = <gid>; reg8[[16..31]] = 
select(<mid>, <mid>);)
  ])

  
-AT_CHECK([grep 'lr_in_ip_routing_ecmp' lr0flows | sed -E 's/== [[1-9]]+\)/== \?)/' | ovn_strip_lflows], [0], [dnl

+AT_CHECK([grep 'lr_in_ip_routing_ecmp' lr0flows | ovn_strip_ecmp_ids | 
ovn_strip_lflows], [0], [dnl
    table=??(lr_in_ip_routing_ecmp), priority=0    , match=(1), action=(drop;)
-  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == 1 && 
reg8[[16..31]] == ?), action=(reg0 = 10.0.10.10; reg5 = 10.0.10.1; eth.src = 00:00:00:00:10:01; 
outport = "lr0-sw1"; reg9[[9]] = 1; next;)
-  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == 1 && 
reg8[[16..31]] == ?), action=(reg0 = 10.0.20.10; reg5 = 10.0.20.1; eth.src = 00:00:00:00:20:01; 
outport = "lr0-sw2"; reg9[[9]] = 1; next;)
+  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == <gid> && 
reg8[[16..31]] == <mid>), action=(reg0 = 10.0.10.10; reg5 = 10.0.10.1; eth.src = 00:00:00:00:10:01; 
outport = "lr0-sw1"; reg9[[9]] = 1; next;)
+  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == <gid> && 
reg8[[16..31]] == <mid>), action=(reg0 = 10.0.20.10; reg5 = 10.0.20.1; eth.src = 00:00:00:00:20:01; 
outport = "lr0-sw2"; reg9[[9]] = 1; next;)
    table=??(lr_in_ip_routing_ecmp), priority=150  , match=(reg8[[0..15]] == 
0), action=(next;)
  ])

  
@@ -24124,14 +24284,14 @@ ecmp2=$(fetch_column nb:Logical_Router_Static_Route _uuid nexthop="10.0.20.10")

  check ovn-nbctl --wait=sb set Logical_Router_Static_Route $ecmp1 
nexthop="discard"

  
  ovn-sbctl dump-flows lr0 > lr0flows

-AT_CHECK([grep 'lr_in_ip_routing' lr0flows | grep 'select' | 
ovn_strip_lflows], [0], [dnl
-  table=??(lr_in_ip_routing   ), priority=1840 , match=(reg7 == 0 && ip4.dst 
== 192.168.10.0/24), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; 
reg8[[16..31]] = select(1, 2);)
+AT_CHECK([grep 'lr_in_ip_routing' lr0flows | grep 'select' | 
ovn_strip_ecmp_ids | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_ip_routing   ), priority=1840 , match=(reg7 == 0 && ip4.dst == 
192.168.10.0/24), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = <gid>; reg8[[16..31]] = 
select(<mid>, <mid>);)
  ])

  
-AT_CHECK([grep 'lr_in_ip_routing_ecmp' lr0flows | sed -E 's/== [[1-9]]+\)/== \?)/' | ovn_strip_lflows], [0], [dnl

+AT_CHECK([grep 'lr_in_ip_routing_ecmp' lr0flows | ovn_strip_ecmp_ids | 
ovn_strip_lflows], [0], [dnl
    table=??(lr_in_ip_routing_ecmp), priority=0    , match=(1), action=(drop;)
-  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == 1 
&& reg8[[16..31]] == ?), action=(drop;)
-  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == 1 
&& reg8[[16..31]] == ?), action=(drop;)
+  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == <gid> && 
reg8[[16..31]] == <mid>), action=(drop;)
+  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == <gid> && 
reg8[[16..31]] == <mid>), action=(drop;)
    table=??(lr_in_ip_routing_ecmp), priority=150  , match=(reg8[[0..15]] == 
0), action=(next;)
  ])

  
@@ -24139,14 +24299,14 @@ AT_CHECK([grep 'lr_in_ip_routing_ecmp' lr0flows | sed -E 's/== [[1-9]]+\)/== \?)

  check ovn-nbctl --wait=sb set Logical_Router_Static_Route $ecmp2 
nexthop="discard"

  
  ovn-sbctl dump-flows lr0 > lr0flows

-AT_CHECK([grep 'lr_in_ip_routing' lr0flows | grep 'select' | 
ovn_strip_lflows], [0], [dnl
-  table=??(lr_in_ip_routing   ), priority=1840 , match=(reg7 == 0 && ip4.dst 
== 192.168.10.0/24), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; 
reg8[[16..31]] = select(1, 2);)
+AT_CHECK([grep 'lr_in_ip_routing' lr0flows | grep 'select' | 
ovn_strip_ecmp_ids | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_ip_routing   ), priority=1840 , match=(reg7 == 0 && ip4.dst == 
192.168.10.0/24), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = <gid>; reg8[[16..31]] = 
select(<mid>, <mid>);)
  ])

  
-AT_CHECK([grep 'lr_in_ip_routing_ecmp' lr0flows | sed -E 's/== [[1-9]]+\)/== \?)/' | ovn_strip_lflows], [0], [dnl

+AT_CHECK([grep 'lr_in_ip_routing_ecmp' lr0flows | ovn_strip_ecmp_ids | 
ovn_strip_lflows], [0], [dnl
    table=??(lr_in_ip_routing_ecmp), priority=0    , match=(1), action=(drop;)
-  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == 1 
&& reg8[[16..31]] == ?), action=(drop;)
-  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == 1 
&& reg8[[16..31]] == ?), action=(drop;)
+  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == <gid> && 
reg8[[16..31]] == <mid>), action=(drop;)
+  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == <gid> && 
reg8[[16..31]] == <mid>), action=(drop;)
    table=??(lr_in_ip_routing_ecmp), priority=150  , match=(reg8[[0..15]] == 
0), action=(next;)
  ])

  
@@ -24155,14 +24315,14 @@ check ovn-nbctl set Logical_Router_Static_Route $ecmp1 nexthop="10.0.10.10"

  check ovn-nbctl --wait=sb set Logical_Router_Static_Route $ecmp2 
nexthop="10.0.20.10"

  
  ovn-sbctl dump-flows lr0 > lr0flows

-AT_CHECK([grep 'lr_in_ip_routing' lr0flows | grep 'select' | 
ovn_strip_lflows], [0], [dnl
-  table=??(lr_in_ip_routing   ), priority=1840 , match=(reg7 == 0 && ip4.dst 
== 192.168.10.0/24), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1; 
reg8[[16..31]] = select(1, 2);)
+AT_CHECK([grep 'lr_in_ip_routing' lr0flows | grep 'select' | 
ovn_strip_ecmp_ids | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_ip_routing   ), priority=1840 , match=(reg7 == 0 && ip4.dst == 
192.168.10.0/24), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = <gid>; reg8[[16..31]] = 
select(<mid>, <mid>);)
  ])

  
-AT_CHECK([grep 'lr_in_ip_routing_ecmp' lr0flows | sed -E 's/== [[1-9]]+\)/== \?)/' | ovn_strip_lflows], [0], [dnl

+AT_CHECK([grep 'lr_in_ip_routing_ecmp' lr0flows | ovn_strip_ecmp_ids | 
ovn_strip_lflows], [0], [dnl
    table=??(lr_in_ip_routing_ecmp), priority=0    , match=(1), action=(drop;)
-  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == 1 && 
reg8[[16..31]] == ?), action=(reg0 = 10.0.10.10; reg5 = 10.0.10.1; eth.src = 00:00:00:00:10:01; 
outport = "lr0-sw1"; reg9[[9]] = 1; next;)
-  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == 1 && 
reg8[[16..31]] == ?), action=(reg0 = 10.0.20.10; reg5 = 10.0.20.1; eth.src = 00:00:00:00:20:01; 
outport = "lr0-sw2"; reg9[[9]] = 1; next;)
+  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == <gid> && 
reg8[[16..31]] == <mid>), action=(reg0 = 10.0.10.10; reg5 = 10.0.10.1; eth.src = 00:00:00:00:10:01; 
outport = "lr0-sw1"; reg9[[9]] = 1; next;)
+  table=??(lr_in_ip_routing_ecmp), priority=100  , match=(reg8[[0..15]] == <gid> && 
reg8[[16..31]] == <mid>), action=(reg0 = 10.0.20.10; reg5 = 10.0.20.1; eth.src = 00:00:00:00:20:01; 
outport = "lr0-sw2"; reg9[[9]] = 1; next;)
    table=??(lr_in_ip_routing_ecmp), priority=150  , match=(reg8[[0..15]] == 
0), action=(next;)
  ])

  


_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev






















					Reply via email to