On 10/26/26 4:54 PM, Rosemarie O'Riorden via dev wrote: > Hi Lucas, thanks for the new verison! I have two tiny nits below that > could be addressed upon merge. So if I were you I'd wait for other > comments before sending another version, as it might not be needed. >
Hi Lucas, Rosemarie, Thanks for the fix and reviews! > Otherwise the new addition to the algorithm looks good to me. > > As I said previously, other reviewers may have comments since I've been > the main reviewer over all the versions, but: > > Acked-by: Rosemarie O'Riorden <[email protected]> > I took care of the nits and applied the patch to main and 26.09. The patch didn't apply cleanly to 26.03 but if you need it there please send a 26.03-specific backport patch. Please also see below a reply to your other hash related comment. Regards, Dumitru > On 10/22/26 3:52 PM, Lucas Vargas Dias wrote: >> The ECMP group id and the member ids are written into the >> lr_in_ip_routing and lr_in_ip_routing_ecmp logical flows, but they >> were assigned from the order in which routes reached >> en_group_ecmp_route: the group id was hmap_count() at creation time >> and the member id was the insertion position. On a full recompute >> the routes are walked in hmap order, while incremental processing >> appends them in event order, so the same set of routes could get a >> different numbering. Every forced or fallback recompute could then >> delete and re-insert Logical_Flow rows in the Southbound DB even >> though nothing had changed. >> >> Derive the ids from hashes instead. The group id comes from the >> group key, i.e. prefix, prefix length, route table, source or >> destination route and source class, which does not change as members >> are added or removed. The member id comes from the UUID of the >> route's source row, its nexthop and its output port: a NAT or LB row >> produces one route per path to the peer router, so the UUID alone is >> not unique within a group. >> >> When two elements derive the same id, all of them retry in the next >> round with an id derived from their hash and the round number, so >> that the ids only depend on the set of routes and not on the order >> in which they were processed. Elements with equal hashes can't be >> told apart, so they skip the retries and take the next free id. The >> members are listed in the select() action in id order for the same >> reason. >> >> Besides making a recompute produce the same flows as incremental >> processing, adding or removing a route no longer changes the ids, >> and so the flows, of unrelated groups or members. >> >> Fixes: 6979a96138a2 ("northd: Support I+P for group_ecmp_route engine.") >> Fixes: 5395e12f247c ("northd: Drop traffic for ECMP group with "discard" >> route.") >> Suggested-by: Rosemarie O'Riorden <[email protected]> >> Assisted-by: Claude Opus 5.5, Claude Code >> Signed-off-by: Lucas Vargas Dias <[email protected]> >> --- >> v6: >> - Elements with equal hashes skip the retry rounds in >> ecmp_assign_ids() and go straight to the fallback. >> - Renamed 'n' to 'n_elements' and 'n_left' to 'n_unassigned' in >> ecmp_assign_ids(), and 'n' to 'n_members' and 'n_groups' in its >> callers. >> v5: >> - Implement route_sources_ecmp_compatible() on top of >> ecmp_group_source_class() so that the NAT/LB rule lives in one place. >> - Hash the whole source row UUID with hash_bytes() in >> ecmp_member_hash(). >> - Derive each id once per round in ecmp_assign_ids(). >> - Cross-reference ecmp_group_hash() and ecmp_groups_find(). >> v4: >> - Derive the ECMP group and member ids from hashes instead of sorting, >> so that adding or removing a route doesn't change the ids of >> unrelated groups or members (suggested by Rosemarie). >> - Added a test with NAT and LB routes reachable over several paths, >> whose ECMP members share their source row. >> - Added ovn_strip_ecmp_ids(), ecmp_ids() and ecmp_member_id() for the >> tests. >> v3: >> - Dropped nullable_strcmp() and the patch converting its callers. >> - Removed the unused 'route_count' field from struct ecmp_groups_node. >> v2: >> - Moved nullable_strcmp() to lib/ovn-util.h and documented its NULL >> handling. >> - Clarified in ecmp_groups_node_cmp() that it can't return 0 for two >> distinct groups. >> >> northd/en-group-ecmp-route.c | 326 +++++++++++++++++++++++++++++++---- >> northd/en-group-ecmp-route.h | 1 - >> tests/ovn-macros.at | 34 ++++ >> tests/ovn-northd.at | 300 ++++++++++++++++++++++++-------- >> 4 files changed, 558 insertions(+), 103 deletions(-) >> >> diff --git a/northd/en-group-ecmp-route.c b/northd/en-group-ecmp-route.c >> index aca197318..4785100f4 100644 >> --- a/northd/en-group-ecmp-route.c >> +++ b/northd/en-group-ecmp-route.c >> @@ -24,6 +24,7 @@ >> #include "en-advertised-route-sync.h" >> #include "en-group-ecmp-route.h" >> #include "en-learned-route-sync.h" >> +#include "hash.h" >> #include "hmapx.h" >> #include "openvswitch/hmap.h" >> >> @@ -182,22 +183,26 @@ unique_routes_destroy(struct hmap *unique_routes) >> hmap_destroy(unique_routes); >> } >> >> -/* Allow LB and NAT forwarding routes for the same prefix to coexist as >> +/* Routes for the same prefix share an ecmp group if their sources map to >> the >> + * same class. >> + * >> + * LB and NAT forwarding routes are in the same class so that they coexist >> as >> * ECMP members instead of one silently replacing the other. This is an >> * edge case: it arises only when two different peer LRs independently >> * use the same IP as a VIP and a NAT external IP, and a CMS would >> * typically prevent such overlap. Without this guard, one route >> * would silently win and traffic to the other service would be >> * black-holed. */ >> +static enum route_source >> +ecmp_group_source_class(enum route_source source) >> +{ >> + return source == ROUTE_SOURCE_LB ? ROUTE_SOURCE_NAT : source; >> +} >> + >> static bool >> route_sources_ecmp_compatible(enum route_source a, enum route_source b) >> { >> - if (a == b) { >> - return true; >> - } >> - >> - return (a == ROUTE_SOURCE_NAT || a == ROUTE_SOURCE_LB) && >> - (b == ROUTE_SOURCE_NAT || b == ROUTE_SOURCE_LB); >> + return ecmp_group_source_class(a) == ecmp_group_source_class(b); >> } >> >> /* Remove the unique_routes_node from the group, and return the parsed_route >> @@ -228,14 +233,12 @@ ecmp_groups_add_route(struct ecmp_groups_node *group, >> const struct parsed_route *route) >> { >> static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); >> - if (group->route_count == UINT16_MAX) { >> + if (vector_len(&group->route_list) == UINT16_MAX) { >> VLOG_WARN_RL(&rl, "too many routes in a single ecmp group."); >> return; >> } >> >> if (route->is_discard_route) { >> - group->has_discard_route = true; >> - >> char *prefix = normalize_v46_prefix(&route->prefix, route->plen); >> VLOG_WARN_RL(&rl, "The ECMP route \"%s\" contains \"discard\" " >> "route, the whole group will drop traffic.", prefix); >> @@ -244,22 +247,13 @@ ecmp_groups_add_route(struct ecmp_groups_node *group, >> >> struct ecmp_route_list_node er = (struct ecmp_route_list_node) { >> .route = route, >> - .id = ++group->route_count, >> }; >> >> - if (group->route_count == 1) { >> - sset_clone(&group->selection_fields, &route->ecmp_selection_fields); >> - } else { >> - sset_intersect(&group->selection_fields, >> - &route->ecmp_selection_fields); >> - } >> - >> vector_push(&group->route_list, &er); >> } >> >> -/* Removes a route from an ecmp group. If the ecmp group should persist >> - * afterwards you must call ecmp_groups_update_ids before any further >> - * insertions. */ >> +/* Removes a route from an ecmp group. The ids of the group and of its >> + * members are refreshed by group_ecmp_datapath_finalize(). */ >> static const struct parsed_route * >> ecmp_groups_remove_route(struct ecmp_groups_node *group, >> const struct parsed_route *pr) >> @@ -278,15 +272,281 @@ ecmp_groups_remove_route(struct ecmp_groups_node >> *group, >> return NULL; >> } >> >> +/* Hashes the key of 'eg', which is unique within a datapath. It must cover >> + * the fields that ecmp_groups_find() compares. Unlike the members, the key >> + * does not change when routes are added to or removed from the group. */ >> +static uint32_t >> +ecmp_group_hash(const struct ecmp_groups_node *eg) >> +{ >> + uint32_t hash = hash_bytes(&eg->prefix, sizeof eg->prefix, 0); >> + hash = hash_add(hash, eg->plen); >> + hash = hash_add(hash, eg->route_table_id); >> + hash = hash_add(hash, eg->is_src_route); >> + hash = hash_add(hash, ecmp_group_source_class(eg->source)); >> + return hash_finish(hash, 0); >> +} >> + >> +/* Hashes what makes 'route' unique within its ecmp group. A NAT or LB row >> + * produces one route per path to the peer router, so its UUID alone is not >> + * enough. */ >> +static uint32_t >> +ecmp_member_hash(const struct parsed_route *route) >> +{ >> + const struct uuid *uuid = &route->source_hint->uuid; >> + uint32_t hash = hash_bytes(uuid, sizeof *uuid, 0); >> + if (route->nexthop) { >> + hash = hash_bytes(route->nexthop, sizeof *route->nexthop, hash); >> + } >> + if (route->out_port) { >> + hash = hash_string(route->out_port->key, hash); >> + } >> + return hash; >> +} >> + >> +/* Number of rounds after which ecmp_assign_ids() stops retrying hashes that >> + * collide and falls back to the next free id. */ >> +#define ECMP_ID_MAX_ROUNDS 32 >> + >> +struct ecmp_id_slot { >> + struct hmap_node hmap_node; >> + uint16_t id; >> + bool taken; /* Assigned to an element. */ >> + size_t n_claims; /* Elements that derived 'id' in the current round. >> */ >> +}; >> + >> +static struct ecmp_id_slot * >> +ecmp_id_slot_get(struct hmap *slots, uint16_t id) >> +{ >> + struct ecmp_id_slot *slot; >> + HMAP_FOR_EACH_WITH_HASH (slot, hmap_node, id, slots) { >> + if (slot->id == id) { >> + return slot; >> + } >> + } >> + slot = xzalloc(sizeof *slot); >> + slot->id = id; >> + hmap_insert(slots, &slot->hmap_node, id); >> + return slot; >> +} >> + >> +struct ecmp_hash_count { >> + struct hmap_node hmap_node; >> + uint32_t hash; >> + size_t n; >> +}; >> + >> +static struct ecmp_hash_count * >> +ecmp_hash_count_get(struct hmap *counts, uint32_t hash) >> +{ >> + struct ecmp_hash_count *count; >> + HMAP_FOR_EACH_WITH_HASH (count, hmap_node, hash, counts) { >> + if (count->hash == hash) { > > Btw, this if-condition will always be true. HMAP_FOR_EACH_WITH_HASH > already checks for the hash match. So this is doing that comparison > twice. So I'd recommend removing the if-condition and just returning > count when a node is found. I don't think you're right, Rosemarie. It's a bit confusing that we're using the hash value as hashmap "hash" too. But you could have two ecmp hash values that "hash" to the same bucket. See: /* Returns the first node in 'hmap' with the given 'hash', or a null pointer if * no nodes have that hash value. */ static inline struct hmap_node * hmap_first_with_hash(const struct hmap *hmap, size_t hash) { return hmap_next_with_hash__(hmap->buckets[hash & hmap->mask], hash); } So I actually think Lucas' version is the correct one. > > For context: > > static inline struct hmap_node * > hmap_next_with_hash__(const struct hmap_node *node, size_t hash) > { > while (node != NULL && node->hash != hash) { > node = node->next; > } > return CONST_CAST(struct hmap_node *, node); > } > >> + return count; >> + } >> + } >> + count = xzalloc(sizeof *count); >> + count->hash = hash; >> + hmap_insert(counts, &count->hmap_node, hash); >> + return count; >> +} >> + >> +/* Sets 'same_hash[i]' for each of the 'n_elements' elements that has no id >> + * yet and shares its hash with another such element. */ >> static void >> -ecmp_group_update_ids(struct ecmp_groups_node *group) >> +ecmp_find_same_hash(const uint32_t *hashes, const uint16_t *ids, >> + bool *same_hash, size_t n_elements) >> { >> + struct hmap counts = HMAP_INITIALIZER(&counts); >> + struct ecmp_hash_count *count; >> + >> + for (size_t i = 0; i < n_elements; i++) { >> + if (!ids[i]) { >> + ecmp_hash_count_get(&counts, hashes[i])->n++; >> + } >> + } >> + for (size_t i = 0; i < n_elements; i++) { >> + if (!ids[i]) { >> + same_hash[i] = ecmp_hash_count_get(&counts, hashes[i])->n > 1; >> + } >> + } >> + HMAP_FOR_EACH_POP (count, hmap_node, &counts) { >> + free(count); >> + } >> + hmap_destroy(&counts); >> +} >> + >> +static uint16_t >> +ecmp_id_from_hash(uint32_t hash, unsigned int round) > > Same unsigned int nit applies here. (see below) > >> +{ >> + return hash_int(round, hash) % UINT16_MAX + 1; >> +} >> + >> +/* Gives each of the 'n_elements' elements a distinct id in [1, UINT16_MAX] >> + * derived from 'hashes', so that the id of an element does not depend on >> the >> + * order in which the elements are passed, nor on the ids of unrelated >> + * elements. >> + * >> + * An element keeps the id derived from its hash unless that id is already >> + * taken or another element derives the same id in the same round. In that >> + * case all of them retry in the next round, with an id derived from their >> + * hash and the round number. The result therefore only depends on the set >> + * of hashes. >> + * >> + * Elements whose hashes are equal derive the same id in every round, so >> they >> + * can't be told apart. They don't retry and, like elements that still >> + * collide after ECMP_ID_MAX_ROUNDS rounds, take the next free id in the >> + * order they are passed. */ >> +static void >> +ecmp_assign_ids(const uint32_t *hashes, uint16_t *ids, size_t n_elements) >> +{ >> + struct hmap slots = HMAP_INITIALIZER(&slots); >> + /* The slot that each element claims in the current round. */ >> + struct ecmp_id_slot **claims = xmalloc(n_elements * sizeof *claims); >> + /* Whether another element has the same hash. */ >> + bool *same_hash = xzalloc(n_elements * sizeof *same_hash); >> + struct ecmp_id_slot *slot; >> + size_t n_unassigned = n_elements; >> + size_t n_retrying = n_elements; >> + >> + memset(ids, 0, n_elements * sizeof *ids); >> + for (unsigned int round = 0; n_retrying && round < ECMP_ID_MAX_ROUNDS; > > Nit: Any particular reason why round is an unsigned int instead of > size_t? size_t is used for all the other loops as usual. > >> + round++) { >> + for (size_t i = 0; i < n_elements; i++) { >> + if (!ids[i] && !same_hash[i]) { >> + uint16_t id = ecmp_id_from_hash(hashes[i], round); >> + claims[i] = ecmp_id_slot_get(&slots, id); >> + claims[i]->n_claims++; >> + } >> + } >> + for (size_t i = 0; i < n_elements; i++) { >> + if (!ids[i] && !same_hash[i]) { >> + slot = claims[i]; >> + if (!slot->taken && slot->n_claims == 1) { >> + slot->taken = true; >> + ids[i] = slot->id; >> + n_unassigned--; >> + n_retrying--; >> + } >> + } >> + } >> + HMAP_FOR_EACH (slot, hmap_node, &slots) { >> + slot->n_claims = 0; >> + } >> + >> + if (!round && n_unassigned) { >> + /* Elements with the same hash always claim the same id, so none >> + * of them got one in this round. */ >> + ecmp_find_same_hash(hashes, ids, same_hash, n_elements); >> + for (size_t i = 0; i < n_elements; i++) { >> + n_retrying -= same_hash[i]; >> + } >> + } >> + } >> + free(claims); >> + free(same_hash); >> + >> + for (size_t i = 0; n_unassigned && i < n_elements; i++) { >> + if (ids[i]) { >> + continue; >> + } >> + uint16_t id = ecmp_id_from_hash(hashes[i], 0); >> + while ((slot = ecmp_id_slot_get(&slots, id))->taken) { >> + id = id % UINT16_MAX + 1; >> + } >> + slot->taken = true; >> + ids[i] = id; >> + n_unassigned--; >> + } >> + >> + HMAP_FOR_EACH_POP (slot, hmap_node, &slots) { >> + free(slot); >> + } >> + hmap_destroy(&slots); >> +} >> + >> +static int >> +ecmp_route_list_node_id_cmp(const void *a_, const void *b_) >> +{ >> + const struct ecmp_route_list_node *a = a_; >> + const struct ecmp_route_list_node *b = b_; >> + >> + return a->id < b->id ? -1 : a->id > b->id; >> +} >> + >> +/* Recomputes everything in 'group' that is derived from its members, so >> that >> + * the result only depends on the set of members. */ >> +static void >> +ecmp_group_finalize(struct ecmp_groups_node *group) >> +{ >> + size_t n_members = vector_len(&group->route_list); >> + uint32_t *hashes = xmalloc(n_members * sizeof *hashes); >> + uint16_t *ids = xmalloc(n_members * sizeof *ids); >> struct ecmp_route_list_node *er; >> size_t i = 0; >> + >> VECTOR_FOR_EACH_PTR (&group->route_list, er) { >> - er->id = i++; >> + hashes[i++] = ecmp_member_hash(er->route); >> + } >> + ecmp_assign_ids(hashes, ids, n_members); >> + i = 0; >> + VECTOR_FOR_EACH_PTR (&group->route_list, er) { >> + er->id = ids[i++]; >> + } >> + free(hashes); >> + free(ids); >> + >> + /* The members are listed in this order in the select() action. */ >> + vector_qsort(&group->route_list, ecmp_route_list_node_id_cmp); >> + >> + bool first = true; >> + group->has_discard_route = false; >> + VECTOR_FOR_EACH_PTR (&group->route_list, er) { >> + if (er->route->is_discard_route) { >> + group->has_discard_route = true; >> + } >> + if (first) { >> + sset_destroy(&group->selection_fields); >> + sset_clone(&group->selection_fields, >> + &er->route->ecmp_selection_fields); >> + group->source = er->route->source; >> + first = false; >> + } else { >> + sset_intersect(&group->selection_fields, >> + &er->route->ecmp_selection_fields); >> + } >> } >> - group->route_count = i; >> +} >> + >> +/* Assigns the group and member ids of all ecmp groups of 'gn'. Must be >> + * called after the ecmp groups of 'gn' changed, both on full recompute and >> on >> + * incremental processing, so that both yield the same ids for the same set >> + * of routes and the generated logical flows do not change on recompute. */ >> +static void >> +group_ecmp_datapath_finalize(struct group_ecmp_datapath *gn) >> +{ >> + size_t n_groups = hmap_count(&gn->ecmp_groups); >> + if (!n_groups) { >> + return; >> + } >> + >> + struct ecmp_groups_node **groups = xmalloc(n_groups * sizeof *groups); >> + uint32_t *hashes = xmalloc(n_groups * sizeof *hashes); >> + uint16_t *ids = xmalloc(n_groups * sizeof *ids); >> + struct ecmp_groups_node *eg; >> + size_t i = 0; >> + >> + HMAP_FOR_EACH (eg, hmap_node, &gn->ecmp_groups) { >> + ecmp_group_finalize(eg); >> + groups[i] = eg; >> + hashes[i++] = ecmp_group_hash(eg); >> + } >> + ecmp_assign_ids(hashes, ids, n_groups); >> + for (i = 0; i < n_groups; i++) { >> + groups[i]->id = ids[i]; >> + } >> + free(groups); >> + free(hashes); >> + free(ids); >> } >> >> static struct ecmp_groups_node * >> @@ -302,7 +562,6 @@ ecmp_groups_add(struct group_ecmp_datapath *gn, >> struct ecmp_groups_node *eg = xzalloc(sizeof *eg); >> hmap_insert(&gn->ecmp_groups, &eg->hmap_node, route->hash); >> >> - eg->id = hmap_count(&gn->ecmp_groups); >> eg->prefix = route->prefix; >> eg->plen = route->plen; >> eg->is_src_route = route->is_src_route; >> @@ -321,6 +580,7 @@ ecmp_groups_find(struct group_ecmp_datapath *gn, >> { >> struct ecmp_groups_node *eg; >> HMAP_FOR_EACH_WITH_HASH (eg, hmap_node, route->hash, &gn->ecmp_groups) { >> + /* Keep in sync with ecmp_group_hash(). */ >> if (ipv6_addr_equals(&eg->prefix, &route->prefix) && >> eg->plen == route->plen && >> eg->is_src_route == route->is_src_route && >> @@ -396,6 +656,10 @@ group_ecmp_route(struct group_ecmp_route_data *data, >> gn = group_ecmp_datapath_lookup_or_add(data, pr->od); >> add_route(gn, pr); >> } >> + >> + HMAP_FOR_EACH (gn, hmap_node, &data->datapaths) { >> + group_ecmp_datapath_finalize(gn); >> + } >> } >> >> enum engine_node_state >> @@ -472,9 +736,7 @@ handle_deleted_route(struct group_ecmp_route_data *data, >> * unique route. Otherwise it stays an ecmp group with just one >> * member. */ >> ecmp_groups_remove_route(eg, pr); >> - if (ecmp_group_has_symmetric_reply(eg)) { >> - ecmp_group_update_ids(eg); >> - } else { >> + if (!ecmp_group_has_symmetric_reply(eg)) { >> const struct ecmp_route_list_node *er = >> vector_get_ptr(&eg->route_list, 0); >> unique_routes_add(node, er->route); >> @@ -482,11 +744,8 @@ handle_deleted_route(struct group_ecmp_route_data *data, >> ecmp_groups_node_free(eg); >> } >> } else { >> - /* We can just remove the member from the group. We need to >> update >> - * the indices of all routes so that future insertions directly >> - * have a new index. */ >> + /* We can just remove the member from the group. */ >> ecmp_groups_remove_route(eg, pr); >> - ecmp_group_update_ids(eg); >> } >> } >> >> @@ -537,6 +796,7 @@ group_ecmp_route_learned_route_change_handler(struct >> engine_node *eng_node, >> hmapx_add(&data->trk_data.deleted_datapath_routes, node); >> hmap_remove(&data->datapaths, &node->hmap_node); >> } else { >> + group_ecmp_datapath_finalize(node); >> hmapx_add(&data->trk_data.crupdated_datapath_routes, node); >> } >> } >> @@ -589,6 +849,7 @@ group_ecmp_route_routes_change_handler(struct >> engine_node *eng_node, >> hmapx_add(&data->trk_data.deleted_datapath_routes, node); >> hmap_remove(&data->datapaths, &node->hmap_node); >> } else { >> + group_ecmp_datapath_finalize(node); >> hmapx_add(&data->trk_data.crupdated_datapath_routes, node); >> } >> } >> @@ -645,6 +906,7 @@ group_ecmp_route_dynamic_routes_change_handler(struct >> engine_node *eng_node, >> hmapx_add(&gdata->trk_data.deleted_datapath_routes, node); >> hmap_remove(&gdata->datapaths, &node->hmap_node); >> } else { >> + group_ecmp_datapath_finalize(node); >> hmapx_add(&gdata->trk_data.crupdated_datapath_routes, node); >> } >> } >> diff --git a/northd/en-group-ecmp-route.h b/northd/en-group-ecmp-route.h >> index b40935be2..13f84a036 100644 >> --- a/northd/en-group-ecmp-route.h >> +++ b/northd/en-group-ecmp-route.h >> @@ -37,7 +37,6 @@ struct ecmp_groups_node { >> bool has_discard_route; >> enum route_source source; >> uint32_t route_table_id; >> - uint16_t route_count; >> struct vector route_list; /* Contains ecmp_route_list_node */ >> struct sset selection_fields; >> }; >> diff --git a/tests/ovn-macros.at b/tests/ovn-macros.at >> index e0f70bcde..811b285d5 100644 >> --- a/tests/ovn-macros.at >> +++ b/tests/ovn-macros.at >> @@ -1546,6 +1546,40 @@ ovn_strip_lflows() { >> sed 's/table=[[0-9]]\{1,2\}\s\?/table=??/g' | sort >> } >> >> +# The ECMP group and member ids are derived from hashes of the routes, >> replace >> +# them with <gid> and <mid>. >> +ovn_strip_ecmp_ids() { >> + sed -e 's/\(reg8\[[0\.\.15\]] =\{1,2\} \)[[1-9]][[0-9]]*/\1<gid>/' \ >> + -e 's/\(reg8\[[16\.\.31\]] =\{1,2\} \)[[1-9]][[0-9]]*/\1<mid>/' \ >> + -e ':a' \ >> + -e 's/\(select[[^)]]*[[(, ]]\)[[0-9]][[0-9]]*/\1<mid>/' \ >> + -e 'ta' >> +} >> + >> +# ecmp_ids LR PREFIX: prints the group id and the comma separated member ids >> +# of the ECMP group of PREFIX on LR, and fails if the members listed in its >> +# select() action are not the ones that have a flow in >> lr_in_ip_routing_ecmp. >> +ecmp_ids() { >> + local flows gid sel mids >> + flows=$(ovn-sbctl dump-flows $1) >> + gid=$(echo "$flows" | sed -n "s|.*ip[[46]]\.dst == >> $2).*reg8\[[0\.\.15\]] = \([[0-9]]*\);.*|\1|p") >> + sel=$(echo "$flows" | sed -n "s|.*ip[[46]]\.dst == >> $2).*select(\([[^)]]*\));.*|\1|p" | tr -d ' ') >> + mids=$(echo "$flows" | sed -n "s|.*(reg8\[[0\.\.15\]] == $gid && >> reg8\[[16\.\.31\]] == \([[0-9]]*\)).*|\1|p" | sort -n | paste -sd, -) >> + if test -z "$gid" || test "$sel" != "$mids"; then >> + echo "$2: group '$gid', select($sel) != members($mids)" >&2 >> + return 1 >> + fi >> + echo "$gid $sel" >> +} >> + >> +# ecmp_member_id LR PREFIX NEXTHOP: prints the member id of NEXTHOP in the >> +# ECMP group of PREFIX on LR. >> +ecmp_member_id() { >> + local gid >> + gid=$(ecmp_ids $1 $2 | cut -d' ' -f1) >> + ovn-sbctl dump-flows $1 | sed -n "s|.*(reg8\[[0\.\.15\]] == $gid && >> reg8\[[16\.\.31\]] == \([[0-9]]*\)), action=(reg0 = $3;.*|\1|p" >> +} >> + >> ovn_strip_collector_set() { >> sed 's/collector_set=[[0-9]]*,\?/collector_set=??,/g' >> } >> diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at >> index d9682f78e..02f86878a 100644 >> --- a/tests/ovn-northd.at >> +++ b/tests/ovn-northd.at >> @@ -7912,18 +7912,18 @@ check_row_count ECMP_Nexthop 1 >> >> ovn-sbctl dump-flows lr0 > lr0flows >> >> -AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | ovn_strip_lflows], [0], [dnl >> +AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | ovn_strip_ecmp_ids | >> ovn_strip_lflows], [0], [dnl >> table=??(lr_in_ip_routing ), priority=0 , match=(1), action=(drop;) >> table=??(lr_in_ip_routing ), priority=10300, >> match=(ct_mark.ecmp_reply_port == 1 && reg7 == 0 && ip4.dst == 1.0.0.1/32), >> action=(ip.ttl--; flags.loopback = 1; eth.src = 00:00:20:20:12:13; reg5 = >> 192.168.0.1; outport = "lr0-public"; next;) >> table=??(lr_in_ip_routing ), priority=10550, match=(nd_rs || nd_ra), >> action=(drop;) >> table=??(lr_in_ip_routing ), priority=1842 , match=(ip4.dst == >> 192.168.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg5 = >> 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; >> flags.loopback = 1; reg9[[9]] = 1; next;) >> - table=??(lr_in_ip_routing ), priority=1936 , match=(reg7 == 0 && >> ip4.dst == 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] >> = 1; reg8[[16..31]] = 1; next;) >> + table=??(lr_in_ip_routing ), priority=1936 , match=(reg7 == 0 && >> ip4.dst == 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] >> = <gid>; reg8[[16..31]] = <mid>; next;) >> table=??(lr_in_ip_routing ), priority=2322 , match=(inport == >> "lr0-public" && ip6.dst == fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; >> xxreg0 = ip6.dst; xxreg1 = fe80::200:20ff:fe20:1213; eth.src = >> 00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 1; reg9[[9]] = >> 0; next;) >> ]) >> >> -AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | ovn_strip_lflows], >> [0], [dnl >> +AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | ovn_strip_ecmp_ids | >> ovn_strip_lflows], [0], [dnl >> table=??(lr_in_ip_routing_ecmp), priority=0 , match=(1), action=(drop;) >> - table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> 1 && reg8[[16..31]] == 1), action=(reg0 = 192.168.0.10; reg5 = 192.168.0.1; >> eth.src = 00:00:20:20:12:13; outport = "lr0-public"; reg9[[9]] = 1; >> reg9[[10]] = 1; next;) >> + table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> <gid> && reg8[[16..31]] == <mid>), action=(reg0 = 192.168.0.10; reg5 = >> 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; reg9[[9]] >> = 1; reg9[[10]] = 1; next;) >> table=??(lr_in_ip_routing_ecmp), priority=150 , match=(reg8[[0..15]] == >> 0), action=(next;) >> ]) >> >> @@ -7938,18 +7938,18 @@ check_column 192.168.0.10 ECMP_Nexthop nexthop >> nexthop=192.168.0.10 >> check_column 192.168.0.20 ECMP_Nexthop nexthop nexthop=192.168.0.20 >> >> ovn-sbctl dump-flows lr0 > lr0flows >> -AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | ovn_strip_lflows], [0], [dnl >> +AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | ovn_strip_ecmp_ids | >> ovn_strip_lflows], [0], [dnl >> table=??(lr_in_ip_routing ), priority=0 , match=(1), action=(drop;) >> table=??(lr_in_ip_routing ), priority=10300, >> match=(ct_mark.ecmp_reply_port == 1 && reg7 == 0 && ip4.dst == 1.0.0.1/32), >> action=(ip.ttl--; flags.loopback = 1; eth.src = 00:00:20:20:12:13; reg5 = >> 192.168.0.1; outport = "lr0-public"; next;) >> table=??(lr_in_ip_routing ), priority=10550, match=(nd_rs || nd_ra), >> action=(drop;) >> table=??(lr_in_ip_routing ), priority=1842 , match=(ip4.dst == >> 192.168.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg5 = >> 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; >> flags.loopback = 1; reg9[[9]] = 1; next;) >> - table=??(lr_in_ip_routing ), priority=1936 , match=(reg7 == 0 && >> ip4.dst == 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] >> = 1; reg8[[16..31]] = select(1, 2);) >> + table=??(lr_in_ip_routing ), priority=1936 , match=(reg7 == 0 && >> ip4.dst == 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] >> = <gid>; reg8[[16..31]] = select(<mid>, <mid>);) >> table=??(lr_in_ip_routing ), priority=2322 , match=(inport == >> "lr0-public" && ip6.dst == fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; >> xxreg0 = ip6.dst; xxreg1 = fe80::200:20ff:fe20:1213; eth.src = >> 00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 1; reg9[[9]] = >> 0; next;) >> ]) >> -AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed >> 's/192\.168\.0\..0/192.168.0.??/' | ovn_strip_lflows], [0], [dnl >> +AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed >> 's/192\.168\.0\..0/192.168.0.??/' | ovn_strip_ecmp_ids | ovn_strip_lflows], >> [0], [dnl >> table=??(lr_in_ip_routing_ecmp), priority=0 , match=(1), action=(drop;) >> - table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> 1 && reg8[[16..31]] == 1), action=(reg0 = 192.168.0.??; reg5 = 192.168.0.1; >> eth.src = 00:00:20:20:12:13; outport = "lr0-public"; reg9[[9]] = 1; >> reg9[[10]] = 1; next;) >> - table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> 1 && reg8[[16..31]] == 2), action=(reg0 = 192.168.0.??; reg5 = 192.168.0.1; >> eth.src = 00:00:20:20:12:13; outport = "lr0-public"; reg9[[9]] = 1; >> reg9[[10]] = 1; next;) >> + table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> <gid> && reg8[[16..31]] == <mid>), action=(reg0 = 192.168.0.??; reg5 = >> 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; reg9[[9]] >> = 1; reg9[[10]] = 1; next;) >> + table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> <gid> && reg8[[16..31]] == <mid>), action=(reg0 = 192.168.0.??; reg5 = >> 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; reg9[[9]] >> = 1; reg9[[10]] = 1; next;) >> table=??(lr_in_ip_routing_ecmp), priority=150 , match=(reg8[[0..15]] == >> 0), action=(next;) >> ]) >> >> @@ -7980,18 +7980,18 @@ check_column 192.168.0.10 ECMP_Nexthop nexthop >> nexthop=192.168.0.10 >> check_column 192.168.0.20 ECMP_Nexthop nexthop nexthop=192.168.0.20 >> >> ovn-sbctl dump-flows lr0 > lr0flows >> -AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | ovn_strip_lflows], [0], [dnl >> +AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | ovn_strip_ecmp_ids | >> ovn_strip_lflows], [0], [dnl >> table=??(lr_in_ip_routing ), priority=0 , match=(1), action=(drop;) >> table=??(lr_in_ip_routing ), priority=10300, >> match=(ct_mark.ecmp_reply_port == 1 && reg7 == 0 && ip4.dst == 1.0.0.1/32), >> action=(ip.ttl--; flags.loopback = 1; eth.src = 00:00:20:20:12:13; reg5 = >> 192.168.0.1; outport = "lr0-public"; next;) >> table=??(lr_in_ip_routing ), priority=10550, match=(nd_rs || nd_ra), >> action=(drop;) >> table=??(lr_in_ip_routing ), priority=1842 , match=(ip4.dst == >> 192.168.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg5 = >> 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; >> flags.loopback = 1; reg9[[9]] = 1; next;) >> - table=??(lr_in_ip_routing ), priority=1936 , match=(reg7 == 0 && >> ip4.dst == 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] >> = 1; reg8[[16..31]] = select(1, 2);) >> + table=??(lr_in_ip_routing ), priority=1936 , match=(reg7 == 0 && >> ip4.dst == 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] >> = <gid>; reg8[[16..31]] = select(<mid>, <mid>);) >> table=??(lr_in_ip_routing ), priority=2322 , match=(inport == >> "lr0-public" && ip6.dst == fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; >> xxreg0 = ip6.dst; xxreg1 = fe80::200:20ff:fe20:1213; eth.src = >> 00:00:20:20:12:13; outport = "lr0-public"; flags.loopback = 1; reg9[[9]] = >> 0; next;) >> ]) >> -AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed >> 's/192\.168\.0\..0/192.168.0.??/' | ovn_strip_lflows], [0], [dnl >> +AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed >> 's/192\.168\.0\..0/192.168.0.??/' | ovn_strip_ecmp_ids | ovn_strip_lflows], >> [0], [dnl >> table=??(lr_in_ip_routing_ecmp), priority=0 , match=(1), action=(drop;) >> - table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> 1 && reg8[[16..31]] == 1), action=(reg0 = 192.168.0.??; reg5 = 192.168.0.1; >> eth.src = 00:00:20:20:12:13; outport = "lr0-public"; reg9[[9]] = 1; >> reg9[[10]] = 1; next;) >> - table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> 1 && reg8[[16..31]] == 2), action=(reg0 = 192.168.0.??; reg5 = 192.168.0.1; >> eth.src = 00:00:20:20:12:13; outport = "lr0-public"; reg9[[9]] = 1; >> reg9[[10]] = 1; next;) >> + table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> <gid> && reg8[[16..31]] == <mid>), action=(reg0 = 192.168.0.??; reg5 = >> 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; reg9[[9]] >> = 1; reg9[[10]] = 1; next;) >> + table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> <gid> && reg8[[16..31]] == <mid>), action=(reg0 = 192.168.0.??; reg5 = >> 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public"; reg9[[9]] >> = 1; reg9[[10]] = 1; next;) >> table=??(lr_in_ip_routing_ecmp), priority=150 , match=(reg8[[0..15]] == >> 0), action=(next;) >> ]) >> >> @@ -8029,8 +8029,8 @@ check ovn-nbctl set logical_router_static_route >> $route2_uuid selection_fields="i >> >> check ovn-nbctl --wait=sb sync >> ovn-sbctl dump-flows lr0 > lr0flows >> -AT_CHECK([grep -e "(lr_in_ip_routing ).*3.3.0.0" lr0flows | sed >> 's/table=../table=??/' | sort], [0], [dnl >> - table=??(lr_in_ip_routing ), priority=1744 , match=(reg7 == 0 && >> ip4.dst == 3.3.0.0/16), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] >> = 1; reg8[[16..31]] = select(values=(1, 2); >> hash_fields="ip_dst,ip_proto,ip_src");) >> +AT_CHECK([grep -e "(lr_in_ip_routing ).*3.3.0.0" lr0flows | >> ovn_strip_ecmp_ids | sed 's/table=../table=??/' | sort], [0], [dnl >> + table=??(lr_in_ip_routing ), priority=1744 , match=(reg7 == 0 && >> ip4.dst == 3.3.0.0/16), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] >> = <gid>; reg8[[16..31]] = select(values=(<mid>, <mid>); >> hash_fields="ip_dst,ip_proto,ip_src");) >> ]) >> >> check ovn-nbctl set logical_router_static_route $route1_uuid >> selection_fields="ip_src,ip_dst,tp_src,tp_dst" >> @@ -8038,11 +8038,11 @@ check ovn-nbctl set logical_router_static_route >> $route2_uuid selection_fields="i >> >> check ovn-nbctl --wait=sb sync >> ovn-sbctl dump-flows lr0 > lr0flows >> -AT_CHECK([grep -e "(lr_in_ip_routing ).*3.3.0.0" lr0flows | sed >> 's/table=../table=??/' | sort], [0], [dnl >> - table=??(lr_in_ip_routing ), priority=1744 , match=(reg7 == 0 && >> ip4.dst == 3.3.0.0/16), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] >> = 1; reg8[[16..31]] = select(values=(1, 2); >> hash_fields="ip_dst,ip_proto,ip_src");) >> - table=??(lr_in_ip_routing ), priority=1745 , match=(reg7 == 0 && >> ip4.dst == 3.3.0.0/16 && sctp), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = 1; reg8[[16..31]] = select(values=(1, 2); >> hash_fields="ip_dst,ip_proto,ip_src,sctp_dst,sctp_src");) >> - table=??(lr_in_ip_routing ), priority=1745 , match=(reg7 == 0 && >> ip4.dst == 3.3.0.0/16 && tcp), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = 1; reg8[[16..31]] = select(values=(1, 2); >> hash_fields="ip_dst,ip_proto,ip_src,tcp_dst,tcp_src");) >> - table=??(lr_in_ip_routing ), priority=1745 , match=(reg7 == 0 && >> ip4.dst == 3.3.0.0/16 && udp), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = 1; reg8[[16..31]] = select(values=(1, 2); >> hash_fields="ip_dst,ip_proto,ip_src,udp_dst,udp_src");) >> +AT_CHECK([grep -e "(lr_in_ip_routing ).*3.3.0.0" lr0flows | >> ovn_strip_ecmp_ids | sed 's/table=../table=??/' | sort], [0], [dnl >> + table=??(lr_in_ip_routing ), priority=1744 , match=(reg7 == 0 && >> ip4.dst == 3.3.0.0/16), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] >> = <gid>; reg8[[16..31]] = select(values=(<mid>, <mid>); >> hash_fields="ip_dst,ip_proto,ip_src");) >> + table=??(lr_in_ip_routing ), priority=1745 , match=(reg7 == 0 && >> ip4.dst == 3.3.0.0/16 && sctp), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = <gid>; reg8[[16..31]] = select(values=(<mid>, <mid>); >> hash_fields="ip_dst,ip_proto,ip_src,sctp_dst,sctp_src");) >> + table=??(lr_in_ip_routing ), priority=1745 , match=(reg7 == 0 && >> ip4.dst == 3.3.0.0/16 && tcp), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = <gid>; reg8[[16..31]] = select(values=(<mid>, <mid>); >> hash_fields="ip_dst,ip_proto,ip_src,tcp_dst,tcp_src");) >> + table=??(lr_in_ip_routing ), priority=1745 , match=(reg7 == 0 && >> ip4.dst == 3.3.0.0/16 && udp), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = <gid>; reg8[[16..31]] = select(values=(<mid>, <mid>); >> hash_fields="ip_dst,ip_proto,ip_src,udp_dst,udp_src");) >> ]) >> >> AT_CHECK([grep -e "lr_in_ecmp_stateful_egr" lr0flows | ovn_strip_lflows], >> [0], [dnl >> @@ -17551,19 +17551,19 @@ datapath=$(fetch_column datapath_binding _uuid >> external_ids:name=lr0) >> >> # Validate the routes. >> ovn-sbctl dump-flows lr0 > lr0flows >> -AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | ovn_strip_lflows], [0], [dnl >> +AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | ovn_strip_ecmp_ids | >> ovn_strip_lflows], [0], [dnl >> table=??(lr_in_ip_routing ), priority=0 , match=(1), action=(drop;) >> table=??(lr_in_ip_routing ), priority=10550, match=(nd_rs || nd_ra), >> action=(drop;) >> - table=??(lr_in_ip_routing ), priority=1840 , match=(reg7 == 0 && >> ip4.dst == 192.168.0.0/24), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = 1; reg8[[16..31]] = select(1, 2);) >> + table=??(lr_in_ip_routing ), priority=1840 , match=(reg7 == 0 && >> ip4.dst == 192.168.0.0/24), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, <mid>);) >> table=??(lr_in_ip_routing ), priority=1842 , match=(ip4.dst == >> 10.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg5 = >> 10.0.0.1; eth.src = 00:00:00:00:ff:01; outport = "lr0-sw0"; flags.loopback = >> 1; reg9[[9]] = 1; next;) >> table=??(lr_in_ip_routing ), priority=1842 , match=(ip4.dst == >> 10.0.1.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg5 = >> 10.0.1.1; eth.src = 00:00:00:00:ff:02; outport = "lr0-sw1"; flags.loopback = >> 1; reg9[[9]] = 1; next;) >> table=??(lr_in_ip_routing ), priority=2322 , match=(inport == "lr0-sw0" >> && ip6.dst == fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; xxreg0 = >> ip6.dst; xxreg1 = fe80::200:ff:fe00:ff01; eth.src = 00:00:00:00:ff:01; >> outport = "lr0-sw0"; flags.loopback = 1; reg9[[9]] = 0; next;) >> table=??(lr_in_ip_routing ), priority=2322 , match=(inport == "lr0-sw1" >> && ip6.dst == fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; xxreg0 = >> ip6.dst; xxreg1 = fe80::200:ff:fe00:ff02; eth.src = 00:00:00:00:ff:02; >> outport = "lr0-sw1"; flags.loopback = 1; reg9[[9]] = 0; next;) >> ]) >> -AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed -e >> 's/10\.0\..\./10.0.??./g' -e 's/lr0-sw./lr0-sw??/' -e 's/00:ff:0./00:ff:0?/' >> | ovn_strip_lflows], [0], [dnl >> +AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed -e >> 's/10\.0\..\./10.0.??./g' -e 's/lr0-sw./lr0-sw??/' -e 's/00:ff:0./00:ff:0?/' >> | ovn_strip_ecmp_ids | ovn_strip_lflows], [0], [dnl >> table=??(lr_in_ip_routing_ecmp), priority=0 , match=(1), action=(drop;) >> - table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> 1 && reg8[[16..31]] == 1), action=(reg0 = 10.0.??.10; reg5 = 10.0.??.1; >> eth.src = 00:00:00:00:ff:0?; outport = "lr0-sw??"; reg9[[9]] = 1; next;) >> - table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> 1 && reg8[[16..31]] == 2), action=(reg0 = 10.0.??.10; reg5 = 10.0.??.1; >> eth.src = 00:00:00:00:ff:0?; outport = "lr0-sw??"; reg9[[9]] = 1; next;) >> + table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> <gid> && reg8[[16..31]] == <mid>), action=(reg0 = 10.0.??.10; reg5 = >> 10.0.??.1; eth.src = 00:00:00:00:ff:0?; outport = "lr0-sw??"; reg9[[9]] = 1; >> next;) >> + table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> <gid> && reg8[[16..31]] == <mid>), action=(reg0 = 10.0.??.10; reg5 = >> 10.0.??.1; eth.src = 00:00:00:00:ff:0?; outport = "lr0-sw??"; reg9[[9]] = 1; >> next;) >> table=??(lr_in_ip_routing_ecmp), priority=150 , match=(reg8[[0..15]] == >> 0), action=(next;) >> ]) >> >> @@ -17577,20 +17577,20 @@ check ovn-nbctl --wait=sb sync >> check_row_count Advertised_Route 4 >> check_row_count Learned_Route 1 >> ovn-sbctl dump-flows lr0 > lr0flows >> -AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | ovn_strip_lflows], [0], [dnl >> +AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | ovn_strip_ecmp_ids | >> ovn_strip_lflows], [0], [dnl >> table=??(lr_in_ip_routing ), priority=0 , match=(1), action=(drop;) >> table=??(lr_in_ip_routing ), priority=10550, match=(nd_rs || nd_ra), >> action=(drop;) >> table=??(lr_in_ip_routing ), priority=1838 , match=(reg7 == 0 && >> ip4.dst == 192.168.1.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = >> 10.0.0.20; reg5 = 10.0.0.1; eth.src = 00:00:00:00:ff:01; outport = >> "lr0-sw0"; flags.loopback = 1; reg9[[9]] = 1; next;) >> - table=??(lr_in_ip_routing ), priority=1840 , match=(reg7 == 0 && >> ip4.dst == 192.168.0.0/24), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = 1; reg8[[16..31]] = select(1, 2);) >> + table=??(lr_in_ip_routing ), priority=1840 , match=(reg7 == 0 && >> ip4.dst == 192.168.0.0/24), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, <mid>);) >> table=??(lr_in_ip_routing ), priority=1842 , match=(ip4.dst == >> 10.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg5 = >> 10.0.0.1; eth.src = 00:00:00:00:ff:01; outport = "lr0-sw0"; flags.loopback = >> 1; reg9[[9]] = 1; next;) >> table=??(lr_in_ip_routing ), priority=1842 , match=(ip4.dst == >> 10.0.1.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg5 = >> 10.0.1.1; eth.src = 00:00:00:00:ff:02; outport = "lr0-sw1"; flags.loopback = >> 1; reg9[[9]] = 1; next;) >> table=??(lr_in_ip_routing ), priority=2322 , match=(inport == "lr0-sw0" >> && ip6.dst == fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; xxreg0 = >> ip6.dst; xxreg1 = fe80::200:ff:fe00:ff01; eth.src = 00:00:00:00:ff:01; >> outport = "lr0-sw0"; flags.loopback = 1; reg9[[9]] = 0; next;) >> table=??(lr_in_ip_routing ), priority=2322 , match=(inport == "lr0-sw1" >> && ip6.dst == fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; xxreg0 = >> ip6.dst; xxreg1 = fe80::200:ff:fe00:ff02; eth.src = 00:00:00:00:ff:02; >> outport = "lr0-sw1"; flags.loopback = 1; reg9[[9]] = 0; next;) >> ]) >> -AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed -e >> 's/10\.0\..\./10.0.??./g' -e 's/lr0-sw./lr0-sw??/' -e 's/00:ff:0./00:ff:0?/' >> | ovn_strip_lflows], [0], [dnl >> +AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed -e >> 's/10\.0\..\./10.0.??./g' -e 's/lr0-sw./lr0-sw??/' -e 's/00:ff:0./00:ff:0?/' >> | ovn_strip_ecmp_ids | ovn_strip_lflows], [0], [dnl >> table=??(lr_in_ip_routing_ecmp), priority=0 , match=(1), action=(drop;) >> - table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> 1 && reg8[[16..31]] == 1), action=(reg0 = 10.0.??.10; reg5 = 10.0.??.1; >> eth.src = 00:00:00:00:ff:0?; outport = "lr0-sw??"; reg9[[9]] = 1; next;) >> - table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> 1 && reg8[[16..31]] == 2), action=(reg0 = 10.0.??.10; reg5 = 10.0.??.1; >> eth.src = 00:00:00:00:ff:0?; outport = "lr0-sw??"; reg9[[9]] = 1; next;) >> + table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> <gid> && reg8[[16..31]] == <mid>), action=(reg0 = 10.0.??.10; reg5 = >> 10.0.??.1; eth.src = 00:00:00:00:ff:0?; outport = "lr0-sw??"; reg9[[9]] = 1; >> next;) >> + table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> <gid> && reg8[[16..31]] == <mid>), action=(reg0 = 10.0.??.10; reg5 = >> 10.0.??.1; eth.src = 00:00:00:00:ff:0?; outport = "lr0-sw??"; reg9[[9]] = 1; >> next;) >> table=??(lr_in_ip_routing_ecmp), priority=150 , match=(reg8[[0..15]] == >> 0), action=(next;) >> ]) >> >> @@ -17605,22 +17605,22 @@ check ovn-nbctl --wait=sb sync >> check_row_count Advertised_Route 4 >> check_row_count Learned_Route 2 >> ovn-sbctl dump-flows lr0 > lr0flows >> -AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | ovn_strip_lflows], [0], [dnl >> +AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | ovn_strip_ecmp_ids | >> ovn_strip_lflows], [0], [dnl >> table=??(lr_in_ip_routing ), priority=0 , match=(1), action=(drop;) >> table=??(lr_in_ip_routing ), priority=10550, match=(nd_rs || nd_ra), >> action=(drop;) >> - table=??(lr_in_ip_routing ), priority=1838 , match=(reg7 == 0 && >> ip4.dst == 192.168.1.0/24), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = 2; reg8[[16..31]] = select(1, 2);) >> - table=??(lr_in_ip_routing ), priority=1840 , match=(reg7 == 0 && >> ip4.dst == 192.168.0.0/24), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = 1; reg8[[16..31]] = select(1, 2);) >> + table=??(lr_in_ip_routing ), priority=1838 , match=(reg7 == 0 && >> ip4.dst == 192.168.1.0/24), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, <mid>);) >> + table=??(lr_in_ip_routing ), priority=1840 , match=(reg7 == 0 && >> ip4.dst == 192.168.0.0/24), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, <mid>);) >> table=??(lr_in_ip_routing ), priority=1842 , match=(ip4.dst == >> 10.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg5 = >> 10.0.0.1; eth.src = 00:00:00:00:ff:01; outport = "lr0-sw0"; flags.loopback = >> 1; reg9[[9]] = 1; next;) >> table=??(lr_in_ip_routing ), priority=1842 , match=(ip4.dst == >> 10.0.1.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg5 = >> 10.0.1.1; eth.src = 00:00:00:00:ff:02; outport = "lr0-sw1"; flags.loopback = >> 1; reg9[[9]] = 1; next;) >> table=??(lr_in_ip_routing ), priority=2322 , match=(inport == "lr0-sw0" >> && ip6.dst == fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; xxreg0 = >> ip6.dst; xxreg1 = fe80::200:ff:fe00:ff01; eth.src = 00:00:00:00:ff:01; >> outport = "lr0-sw0"; flags.loopback = 1; reg9[[9]] = 0; next;) >> table=??(lr_in_ip_routing ), priority=2322 , match=(inport == "lr0-sw1" >> && ip6.dst == fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; xxreg0 = >> ip6.dst; xxreg1 = fe80::200:ff:fe00:ff02; eth.src = 00:00:00:00:ff:02; >> outport = "lr0-sw1"; flags.loopback = 1; reg9[[9]] = 0; next;) >> ]) >> -AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed -e >> 's/10\.0\..\./10.0.??./g' -e 's/lr0-sw./lr0-sw??/' -e 's/00:ff:0./00:ff:0?/' >> | ovn_strip_lflows], [0], [dnl >> +AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed -e >> 's/10\.0\..\./10.0.??./g' -e 's/lr0-sw./lr0-sw??/' -e 's/00:ff:0./00:ff:0?/' >> | ovn_strip_ecmp_ids | ovn_strip_lflows], [0], [dnl >> table=??(lr_in_ip_routing_ecmp), priority=0 , match=(1), action=(drop;) >> - table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> 1 && reg8[[16..31]] == 1), action=(reg0 = 10.0.??.10; reg5 = 10.0.??.1; >> eth.src = 00:00:00:00:ff:0?; outport = "lr0-sw??"; reg9[[9]] = 1; next;) >> - table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> 1 && reg8[[16..31]] == 2), action=(reg0 = 10.0.??.10; reg5 = 10.0.??.1; >> eth.src = 00:00:00:00:ff:0?; outport = "lr0-sw??"; reg9[[9]] = 1; next;) >> - table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> 2 && reg8[[16..31]] == 1), action=(reg0 = 10.0.??.20; reg5 = 10.0.??.1; >> eth.src = 00:00:00:00:ff:0?; outport = "lr0-sw??"; reg9[[9]] = 1; next;) >> - table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> 2 && reg8[[16..31]] == 2), action=(reg0 = 10.0.??.20; reg5 = 10.0.??.1; >> eth.src = 00:00:00:00:ff:0?; outport = "lr0-sw??"; reg9[[9]] = 1; next;) >> + table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> <gid> && reg8[[16..31]] == <mid>), action=(reg0 = 10.0.??.10; reg5 = >> 10.0.??.1; eth.src = 00:00:00:00:ff:0?; outport = "lr0-sw??"; reg9[[9]] = 1; >> next;) >> + table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> <gid> && reg8[[16..31]] == <mid>), action=(reg0 = 10.0.??.10; reg5 = >> 10.0.??.1; eth.src = 00:00:00:00:ff:0?; outport = "lr0-sw??"; reg9[[9]] = 1; >> next;) >> + table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> <gid> && reg8[[16..31]] == <mid>), action=(reg0 = 10.0.??.20; reg5 = >> 10.0.??.1; eth.src = 00:00:00:00:ff:0?; outport = "lr0-sw??"; reg9[[9]] = 1; >> next;) >> + table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> <gid> && reg8[[16..31]] == <mid>), action=(reg0 = 10.0.??.20; reg5 = >> 10.0.??.1; eth.src = 00:00:00:00:ff:0?; outport = "lr0-sw??"; reg9[[9]] = 1; >> next;) >> table=??(lr_in_ip_routing_ecmp), priority=150 , match=(reg8[[0..15]] == >> 0), action=(next;) >> ]) >> >> @@ -17647,12 +17647,12 @@ check ovn-nbctl --wait=sb sync >> check_row_count Advertised_Route 4 >> check_row_count Learned_Route 4 >> ovn-sbctl dump-flows lr0 > lr0flows >> -AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | sed -e 's/reg8\[[0..15\]] = >> [[123]]/reg8\[[0..15\]] = ??/' | ovn_strip_lflows], [0], [dnl >> +AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | ovn_strip_ecmp_ids | >> ovn_strip_lflows], [0], [dnl >> table=??(lr_in_ip_routing ), priority=0 , match=(1), action=(drop;) >> table=??(lr_in_ip_routing ), priority=10550, match=(nd_rs || nd_ra), >> action=(drop;) >> - table=??(lr_in_ip_routing ), priority=1838 , match=(reg7 == 0 && >> ip4.dst == 192.168.0.0/24), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = ??; reg8[[16..31]] = select(1, 2);) >> - table=??(lr_in_ip_routing ), priority=1838 , match=(reg7 == 0 && >> ip4.dst == 192.168.1.0/24), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = ??; reg8[[16..31]] = select(1, 2);) >> - table=??(lr_in_ip_routing ), priority=1840 , match=(reg7 == 0 && >> ip4.dst == 192.168.0.0/24), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = ??; reg8[[16..31]] = select(1, 2);) >> + table=??(lr_in_ip_routing ), priority=1838 , match=(reg7 == 0 && >> ip4.dst == 192.168.0.0/24), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, <mid>);) >> + table=??(lr_in_ip_routing ), priority=1838 , match=(reg7 == 0 && >> ip4.dst == 192.168.1.0/24), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, <mid>);) >> + table=??(lr_in_ip_routing ), priority=1840 , match=(reg7 == 0 && >> ip4.dst == 192.168.0.0/24), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, <mid>);) >> table=??(lr_in_ip_routing ), priority=1842 , match=(ip4.dst == >> 10.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg5 = >> 10.0.0.1; eth.src = 00:00:00:00:ff:01; outport = "lr0-sw0"; flags.loopback = >> 1; reg9[[9]] = 1; next;) >> table=??(lr_in_ip_routing ), priority=1842 , match=(ip4.dst == >> 10.0.1.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg5 = >> 10.0.1.1; eth.src = 00:00:00:00:ff:02; outport = "lr0-sw1"; flags.loopback = >> 1; reg9[[9]] = 1; next;) >> table=??(lr_in_ip_routing ), priority=2322 , match=(inport == "lr0-sw0" >> && ip6.dst == fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0; xxreg0 = >> ip6.dst; xxreg1 = fe80::200:ff:fe00:ff01; eth.src = 00:00:00:00:ff:01; >> outport = "lr0-sw0"; flags.loopback = 1; reg9[[9]] = 0; next;) >> @@ -17697,13 +17697,13 @@ check_uuid ovn-sbctl create Learned_Route \ >> check ovn-nbctl --wait=sb sync >> >> ovn-sbctl dump-flows lr0 > lr0flows >> -AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | grep -F "42.42.42.42" | >> ovn_strip_lflows], [0], [dnl >> - table=??(lr_in_ip_routing ), priority=1934 , match=(reg7 == 0 && >> ip4.dst == 42.42.42.42/32), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = 1; reg8[[16..31]] = select(1, 2);) >> +AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | grep -F "42.42.42.42" | >> ovn_strip_ecmp_ids | ovn_strip_lflows], [0], [dnl >> + table=??(lr_in_ip_routing ), priority=1934 , match=(reg7 == 0 && >> ip4.dst == 42.42.42.42/32), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, <mid>);) >> ]) >> >> -AT_CHECK([grep "lr_in_ip_routing_ecmp" lr0flows | grep "42" | sed -e >> 's/reg8\[[16..31\]] == [[12]]/reg8\[[16..31\]] == ??/g' | ovn_strip_lflows], >> [0], [dnl >> - table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> 1 && reg8[[16..31]] == ??), action=(reg0 = 192.168.2.42; eth.src = >> 00:00:00:00:ff:02; outport = "lrp2"; reg9[[9]] = 1; next;) >> - table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> 1 && reg8[[16..31]] == ??), action=(reg0 = 192.168.3.42; eth.src = >> 00:00:00:00:ff:03; outport = "lrp3"; reg9[[9]] = 1; next;) >> +AT_CHECK([grep "lr_in_ip_routing_ecmp" lr0flows | grep "42" | >> ovn_strip_ecmp_ids | ovn_strip_lflows], [0], [dnl >> + table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> <gid> && reg8[[16..31]] == <mid>), action=(reg0 = 192.168.2.42; eth.src = >> 00:00:00:00:ff:02; outport = "lrp2"; reg9[[9]] = 1; next;) >> + table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> <gid> && reg8[[16..31]] == <mid>), action=(reg0 = 192.168.3.42; eth.src = >> 00:00:00:00:ff:03; outport = "lrp3"; reg9[[9]] = 1; next;) >> ]) >> >> OVN_CLEANUP_NORTHD >> @@ -19461,8 +19461,8 @@ check_row_count Advertised_Route 1 \ >> # One ECMP selection flow covers the prefix and two member flows carry the >> # respective neighbor nexthops. >> ovn-sbctl lflow-list lr0 > lr0_flows >> -AT_CHECK([grep 'lr_in_ip_routing.*172.16.1.10/32' lr0_flows | \ >> - grep -c 'select(1, 2)'], [0], [1 >> +AT_CHECK([grep 'lr_in_ip_routing.*172.16.1.10/32' lr0_flows | >> ovn_strip_ecmp_ids | \ >> + grep -c 'select(<mid>, <mid>)'], [0], [1 >> ]) >> AT_CHECK([grep 'lr_in_ip_routing_ecmp' lr0_flows | \ >> grep -E -c 'reg0 = 10\.0\.0\.[[12]];'], [0], [2 >> @@ -19481,6 +19481,85 @@ OVN_CLEANUP_NORTHD >> AT_CLEANUP >> ]) >> >> +OVN_FOR_EACH_NORTHD_NO_HV([ >> +AT_SETUP([dynamic-routing - NAT and LB reachable over several paths use >> ECMP]) >> +AT_KEYWORDS([dynamic-routing ecmp]) >> +ovn_start >> + >> +# lr0 reaches lr1 over the switches "a" and "b" and redistributes NAT and LB >> +# addresses on both ports. The NAT and the LB of lr1 then produce one >> +# forwarding route per path on lr0. The two routes of each ECMP group share >> +# their source row and only differ in the nexthop and output port, they must >> +# still get distinct ids that don't depend on the processing order. >> + >> +check ovn-nbctl lr-add lr0 -- set Logical_Router lr0 \ >> + options:dynamic-routing=true options:chassis=hv1 >> +check ovn-nbctl lr-add lr1 -- set Logical_Router lr1 options:chassis=hv1 >> +i=0 >> +for s in a b; do >> + i=$((i + 1)) >> + check ovn-nbctl ls-add $s >> + check ovn-nbctl lrp-add lr0 lr0-$s 00:00:00:00:0$i:01 10.0.$i.1/24 >> + check ovn-nbctl lrp-set-options lr0-$s >> dynamic-routing-redistribute=lb,nat >> + check ovn-nbctl lsp-add-router-port $s $s-lr0 lr0-$s >> + check ovn-nbctl lrp-add lr1 lr1-$s 00:00:00:00:0$i:02 10.0.$i.2/24 >> + check ovn-nbctl lsp-add-router-port $s $s-lr1 lr1-$s >> +done >> +check ovn-nbctl lr-nat-add lr1 dnat_and_snat 172.16.1.10 192.168.1.10 >> +check ovn-nbctl lb-add lb1 172.16.1.20:80 192.168.1.20:80 >> +check ovn-nbctl --wait=sb lr-lb-add lr1 lb1 >> + >> +ovn-sbctl dump-flows lr0 > lr0flows >> +AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | grep "172\.16\.1\." | >> ovn_strip_ecmp_ids | ovn_strip_lflows], [0], [dnl >> + table=??(lr_in_ip_routing ), priority=1935 , match=(ip4.dst == >> 172.16.1.10/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = >> <gid>; reg8[[16..31]] = select(<mid>, <mid>);) >> + table=??(lr_in_ip_routing ), priority=1935 , match=(ip4.dst == >> 172.16.1.20/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = >> <gid>; reg8[[16..31]] = select(<mid>, <mid>);) >> +]) >> +AT_CHECK([grep "lr_in_ip_routing_ecmp" lr0flows | grep "reg0 = 10\.0\." | >> ovn_strip_ecmp_ids | ovn_strip_lflows], [0], [dnl >> + table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> <gid> && reg8[[16..31]] == <mid>), action=(reg0 = 10.0.1.2; reg5 = 10.0.1.1; >> eth.src = 00:00:00:00:01:01; outport = "lr0-a"; reg9[[9]] = 1; next;) >> + table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> <gid> && reg8[[16..31]] == <mid>), action=(reg0 = 10.0.1.2; reg5 = 10.0.1.1; >> eth.src = 00:00:00:00:01:01; outport = "lr0-a"; reg9[[9]] = 1; next;) >> + table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> <gid> && reg8[[16..31]] == <mid>), action=(reg0 = 10.0.2.2; reg5 = 10.0.2.1; >> eth.src = 00:00:00:00:02:01; outport = "lr0-b"; reg9[[9]] = 1; next;) >> + table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> <gid> && reg8[[16..31]] == <mid>), action=(reg0 = 10.0.2.2; reg5 = 10.0.2.1; >> eth.src = 00:00:00:00:02:01; outport = "lr0-b"; reg9[[9]] = 1; next;) >> +]) >> + >> +dnl Both members of each group come from the same NAT or LB row. >> +nat=$(fetch_column nb:NAT _uuid external_ip=172.16.1.10) >> +lb=$(fetch_column nb:Load_Balancer _uuid name=lb1) >> +check_row_count Logical_Flow 2 >> external_ids:stage-name=lr_in_ip_routing_ecmp \ >> + external_ids:stage-hint=${nat:0:8} >> +check_row_count Logical_Flow 2 >> external_ids:stage-name=lr_in_ip_routing_ecmp \ >> + external_ids:stage-hint=${lb:0:8} >> + >> +dnl ecmp_ids also checks that the members have distinct ids. >> +nat_ids=$(ecmp_ids lr0 172.16.1.10/32) >> +lb_ids=$(ecmp_ids lr0 172.16.1.20/32) >> +AT_CHECK([test -n "$nat_ids" && test -n "$lb_ids"]) >> +CHECK_NO_CHANGE_AFTER_RECOMPUTE >> + >> +dnl A third path adds a member to each group without changing the ids of the >> +dnl existing members. >> +check ovn-nbctl ls-add c >> +check ovn-nbctl lrp-add lr0 lr0-c 00:00:00:00:03:01 10.0.3.1/24 >> +check ovn-nbctl lrp-set-options lr0-c dynamic-routing-redistribute=lb,nat >> +check ovn-nbctl lsp-add-router-port c c-lr0 lr0-c >> +check ovn-nbctl lrp-add lr1 lr1-c 00:00:00:00:03:02 10.0.3.2/24 >> +check ovn-nbctl --wait=sb lsp-add-router-port c c-lr1 lr1-c >> +CHECK_NO_CHANGE_AFTER_RECOMPUTE >> + >> +for old in "$nat_ids 172.16.1.10/32" "$lb_ids 172.16.1.20/32"; do >> + set -- $old >> + new=$(ecmp_ids lr0 $3) >> + AT_CHECK([test "${new%% *}" = "$1"]) >> + AT_CHECK([echo "${new#* }" | tr , '\n' | wc -l], [0], [3 >> +]) >> + for m in $(echo $2 | tr , ' '); do >> + AT_CHECK([echo "${new#* }" | tr , '\n' | grep -qx $m]) >> + done >> +done >> + >> +OVN_CLEANUP_NORTHD >> +AT_CLEANUP >> +]) >> + >> OVN_FOR_EACH_NORTHD_NO_HV([ >> AT_SETUP([dynamic-routing - LB redistribute uses backend LSP as >> tracked_port]) >> AT_KEYWORDS([dynamic-routing]) >> @@ -24203,6 +24282,87 @@ done >> OVN_CLEANUP_NORTHD >> AT_CLEANUP >> >> +OVN_FOR_EACH_NORTHD_NO_HV([ >> +AT_SETUP([Static routes - ECMP ids stable across recompute]) >> +AT_KEYWORDS([ecmp]) >> +ovn_start >> + >> +check ovn-nbctl lr-add lr0 >> +for i in 1 2 3 4; do >> + check ovn-nbctl lrp-add lr0 lr0-sw$i 00:00:00:00:0$i:01 10.0.$i.1/24 >> +done >> +check ovn-nbctl --wait=sb sync >> + >> +dnl Add the routes in "reverse" order, the ids must not depend on it. >> +for p in 192.168.30.0/24 192.168.20.0/24 192.168.10.0/24; do >> + for i in 3 2 1; do >> + check ovn-nbctl --wait=sb --ecmp lr-route-add lr0 $p 10.0.$i.10 >> + done >> +done >> +CHECK_NO_CHANGE_AFTER_RECOMPUTE >> + >> +AT_CHECK([ovn-sbctl dump-flows lr0 | grep -w lr_in_ip_routing | grep select >> | ovn_strip_ecmp_ids | ovn_strip_lflows], [0], [dnl >> + table=??(lr_in_ip_routing ), priority=1840 , match=(reg7 == 0 && >> ip4.dst == 192.168.10.0/24), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, <mid>, <mid>);) >> + table=??(lr_in_ip_routing ), priority=1840 , match=(reg7 == 0 && >> ip4.dst == 192.168.20.0/24), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, <mid>, <mid>);) >> + table=??(lr_in_ip_routing ), priority=1840 , match=(reg7 == 0 && >> ip4.dst == 192.168.30.0/24), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, <mid>, <mid>);) >> +]) >> +g10=$(ecmp_ids lr0 192.168.10.0/24) >> +g20=$(ecmp_ids lr0 192.168.20.0/24) >> +g30=$(ecmp_ids lr0 192.168.30.0/24) >> +AT_CHECK([test -n "$g10" && test -n "$g20" && test -n "$g30"]) >> +AT_CHECK([printf '%s\n' "$g10" "$g20" "$g30" | cut -d' ' -f1 | sort -u | wc >> -l], [0], [3 >> +]) >> + >> +dnl Remove a member from the middle of a group with more than 2 members, the >> +dnl ids of the other members and groups must not change. >> +m2=$(ecmp_member_id lr0 192.168.10.0/24 10.0.2.10) >> +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats >> +check ovn-nbctl --wait=sb lr-route-del lr0 192.168.10.0/24 10.0.2.10 >> +check_engine_compute group_ecmp_route incremental >> +CHECK_NO_CHANGE_AFTER_RECOMPUTE >> +g10_ids=$(echo "${g10#* }" | tr , '\n' | grep -vx "$m2" | paste -sd, -) >> +AT_CHECK_UNQUOTED([ecmp_ids lr0 192.168.10.0/24], [0], [${g10%% *} $g10_ids >> +]) >> +AT_CHECK_UNQUOTED([ecmp_ids lr0 192.168.20.0/24], [0], [$g20 >> +]) >> +AT_CHECK_UNQUOTED([ecmp_ids lr0 192.168.30.0/24], [0], [$g30 >> +]) >> +g10=$(ecmp_ids lr0 192.168.10.0/24) >> + >> +dnl Remove a whole group and then add a new one, the ids of the other groups >> +dnl must not change and the group ids must stay unique. >> +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats >> +check ovn-nbctl --wait=sb lr-route-del lr0 192.168.20.0/24 >> +check ovn-nbctl --wait=sb --ecmp lr-route-add lr0 192.168.40.0/24 10.0.1.10 >> +check ovn-nbctl --wait=sb --ecmp lr-route-add lr0 192.168.40.0/24 10.0.4.10 >> +check_engine_compute group_ecmp_route incremental >> +CHECK_NO_CHANGE_AFTER_RECOMPUTE >> +AT_CHECK_UNQUOTED([ecmp_ids lr0 192.168.10.0/24], [0], [$g10 >> +]) >> +AT_CHECK_UNQUOTED([ecmp_ids lr0 192.168.30.0/24], [0], [$g30 >> +]) >> +g40=$(ecmp_ids lr0 192.168.40.0/24) >> +AT_CHECK([test -n "$g40"]) >> +AT_CHECK([printf '%s\n' "$g10" "$g30" "$g40" | cut -d' ' -f1 | sort -u | wc >> -l], [0], [3 >> +]) >> + >> +dnl Once no member is a discard route anymore the group must stop dropping. >> +route=$(ovn-nbctl --bare --columns _uuid find Logical_Router_Static_Route \ >> + ip_prefix="192.168.30.0/24" nexthop="10.0.3.10") >> +check ovn-nbctl --wait=sb set Logical_Router_Static_Route $route >> nexthop=discard >> +AT_CHECK([ovn-sbctl dump-flows lr0 | grep lr_in_ip_routing_ecmp | grep -c >> "drop;"], [0], [4 >> +]) >> +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats >> +check ovn-nbctl --wait=sb set Logical_Router_Static_Route $route >> nexthop=10.0.3.10 >> +check_engine_compute group_ecmp_route incremental >> +AT_CHECK([ovn-sbctl dump-flows lr0 | grep lr_in_ip_routing_ecmp | grep -c >> "drop;"], [0], [1 >> +]) >> +CHECK_NO_CHANGE_AFTER_RECOMPUTE >> + >> +OVN_CLEANUP_NORTHD >> +AT_CLEANUP >> +]) >> + >> OVN_FOR_EACH_NORTHD_NO_HV([ >> AT_SETUP([Static routes - ECMP with discard]) >> ovn_start >> @@ -24217,14 +24377,14 @@ check ovn-nbctl >> \ >> check ovn-nbctl --wait=sb sync >> >> ovn-sbctl dump-flows lr0 > lr0flows >> -AT_CHECK([grep 'lr_in_ip_routing' lr0flows | grep 'select' | >> ovn_strip_lflows], [0], [dnl >> - table=??(lr_in_ip_routing ), priority=1840 , match=(reg7 == 0 && >> ip4.dst == 192.168.10.0/24), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = 1; reg8[[16..31]] = select(1, 2);) >> +AT_CHECK([grep 'lr_in_ip_routing' lr0flows | grep 'select' | >> ovn_strip_ecmp_ids | ovn_strip_lflows], [0], [dnl >> + table=??(lr_in_ip_routing ), priority=1840 , match=(reg7 == 0 && >> ip4.dst == 192.168.10.0/24), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, <mid>);) >> ]) >> >> -AT_CHECK([grep 'lr_in_ip_routing_ecmp' lr0flows | sed -E 's/== >> [[1-9]]+\)/== \?)/' | ovn_strip_lflows], [0], [dnl >> +AT_CHECK([grep 'lr_in_ip_routing_ecmp' lr0flows | ovn_strip_ecmp_ids | >> ovn_strip_lflows], [0], [dnl >> table=??(lr_in_ip_routing_ecmp), priority=0 , match=(1), action=(drop;) >> - table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> 1 && reg8[[16..31]] == ?), action=(reg0 = 10.0.10.10; reg5 = 10.0.10.1; >> eth.src = 00:00:00:00:10:01; outport = "lr0-sw1"; reg9[[9]] = 1; next;) >> - table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> 1 && reg8[[16..31]] == ?), action=(reg0 = 10.0.20.10; reg5 = 10.0.20.1; >> eth.src = 00:00:00:00:20:01; outport = "lr0-sw2"; reg9[[9]] = 1; next;) >> + table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> <gid> && reg8[[16..31]] == <mid>), action=(reg0 = 10.0.10.10; reg5 = >> 10.0.10.1; eth.src = 00:00:00:00:10:01; outport = "lr0-sw1"; reg9[[9]] = 1; >> next;) >> + table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> <gid> && reg8[[16..31]] == <mid>), action=(reg0 = 10.0.20.10; reg5 = >> 10.0.20.1; eth.src = 00:00:00:00:20:01; outport = "lr0-sw2"; reg9[[9]] = 1; >> next;) >> table=??(lr_in_ip_routing_ecmp), priority=150 , match=(reg8[[0..15]] == >> 0), action=(next;) >> ]) >> >> @@ -24235,14 +24395,14 @@ ecmp2=$(fetch_column >> nb:Logical_Router_Static_Route _uuid nexthop="10.0.20.10") >> check ovn-nbctl --wait=sb set Logical_Router_Static_Route $ecmp1 >> nexthop="discard" >> >> ovn-sbctl dump-flows lr0 > lr0flows >> -AT_CHECK([grep 'lr_in_ip_routing' lr0flows | grep 'select' | >> ovn_strip_lflows], [0], [dnl >> - table=??(lr_in_ip_routing ), priority=1840 , match=(reg7 == 0 && >> ip4.dst == 192.168.10.0/24), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = 1; reg8[[16..31]] = select(1, 2);) >> +AT_CHECK([grep 'lr_in_ip_routing' lr0flows | grep 'select' | >> ovn_strip_ecmp_ids | ovn_strip_lflows], [0], [dnl >> + table=??(lr_in_ip_routing ), priority=1840 , match=(reg7 == 0 && >> ip4.dst == 192.168.10.0/24), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, <mid>);) >> ]) >> >> -AT_CHECK([grep 'lr_in_ip_routing_ecmp' lr0flows | sed -E 's/== >> [[1-9]]+\)/== \?)/' | ovn_strip_lflows], [0], [dnl >> +AT_CHECK([grep 'lr_in_ip_routing_ecmp' lr0flows | ovn_strip_ecmp_ids | >> ovn_strip_lflows], [0], [dnl >> table=??(lr_in_ip_routing_ecmp), priority=0 , match=(1), action=(drop;) >> - table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> 1 && reg8[[16..31]] == ?), action=(drop;) >> - table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> 1 && reg8[[16..31]] == ?), action=(drop;) >> + table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> <gid> && reg8[[16..31]] == <mid>), action=(drop;) >> + table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> <gid> && reg8[[16..31]] == <mid>), action=(drop;) >> table=??(lr_in_ip_routing_ecmp), priority=150 , match=(reg8[[0..15]] == >> 0), action=(next;) >> ]) >> >> @@ -24250,14 +24410,14 @@ AT_CHECK([grep 'lr_in_ip_routing_ecmp' lr0flows | >> sed -E 's/== [[1-9]]+\)/== \?) >> check ovn-nbctl --wait=sb set Logical_Router_Static_Route $ecmp2 >> nexthop="discard" >> >> ovn-sbctl dump-flows lr0 > lr0flows >> -AT_CHECK([grep 'lr_in_ip_routing' lr0flows | grep 'select' | >> ovn_strip_lflows], [0], [dnl >> - table=??(lr_in_ip_routing ), priority=1840 , match=(reg7 == 0 && >> ip4.dst == 192.168.10.0/24), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = 1; reg8[[16..31]] = select(1, 2);) >> +AT_CHECK([grep 'lr_in_ip_routing' lr0flows | grep 'select' | >> ovn_strip_ecmp_ids | ovn_strip_lflows], [0], [dnl >> + table=??(lr_in_ip_routing ), priority=1840 , match=(reg7 == 0 && >> ip4.dst == 192.168.10.0/24), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, <mid>);) >> ]) >> >> -AT_CHECK([grep 'lr_in_ip_routing_ecmp' lr0flows | sed -E 's/== >> [[1-9]]+\)/== \?)/' | ovn_strip_lflows], [0], [dnl >> +AT_CHECK([grep 'lr_in_ip_routing_ecmp' lr0flows | ovn_strip_ecmp_ids | >> ovn_strip_lflows], [0], [dnl >> table=??(lr_in_ip_routing_ecmp), priority=0 , match=(1), action=(drop;) >> - table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> 1 && reg8[[16..31]] == ?), action=(drop;) >> - table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> 1 && reg8[[16..31]] == ?), action=(drop;) >> + table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> <gid> && reg8[[16..31]] == <mid>), action=(drop;) >> + table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> <gid> && reg8[[16..31]] == <mid>), action=(drop;) >> table=??(lr_in_ip_routing_ecmp), priority=150 , match=(reg8[[0..15]] == >> 0), action=(next;) >> ]) >> >> @@ -24266,14 +24426,14 @@ check ovn-nbctl set Logical_Router_Static_Route >> $ecmp1 nexthop="10.0.10.10" >> check ovn-nbctl --wait=sb set Logical_Router_Static_Route $ecmp2 >> nexthop="10.0.20.10" >> >> ovn-sbctl dump-flows lr0 > lr0flows >> -AT_CHECK([grep 'lr_in_ip_routing' lr0flows | grep 'select' | >> ovn_strip_lflows], [0], [dnl >> - table=??(lr_in_ip_routing ), priority=1840 , match=(reg7 == 0 && >> ip4.dst == 192.168.10.0/24), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = 1; reg8[[16..31]] = select(1, 2);) >> +AT_CHECK([grep 'lr_in_ip_routing' lr0flows | grep 'select' | >> ovn_strip_ecmp_ids | ovn_strip_lflows], [0], [dnl >> + table=??(lr_in_ip_routing ), priority=1840 , match=(reg7 == 0 && >> ip4.dst == 192.168.10.0/24), action=(ip.ttl--; flags.loopback = 1; >> reg8[[0..15]] = <gid>; reg8[[16..31]] = select(<mid>, <mid>);) >> ]) >> >> -AT_CHECK([grep 'lr_in_ip_routing_ecmp' lr0flows | sed -E 's/== >> [[1-9]]+\)/== \?)/' | ovn_strip_lflows], [0], [dnl >> +AT_CHECK([grep 'lr_in_ip_routing_ecmp' lr0flows | ovn_strip_ecmp_ids | >> ovn_strip_lflows], [0], [dnl >> table=??(lr_in_ip_routing_ecmp), priority=0 , match=(1), action=(drop;) >> - table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> 1 && reg8[[16..31]] == ?), action=(reg0 = 10.0.10.10; reg5 = 10.0.10.1; >> eth.src = 00:00:00:00:10:01; outport = "lr0-sw1"; reg9[[9]] = 1; next;) >> - table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> 1 && reg8[[16..31]] == ?), action=(reg0 = 10.0.20.10; reg5 = 10.0.20.1; >> eth.src = 00:00:00:00:20:01; outport = "lr0-sw2"; reg9[[9]] = 1; next;) >> + table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> <gid> && reg8[[16..31]] == <mid>), action=(reg0 = 10.0.10.10; reg5 = >> 10.0.10.1; eth.src = 00:00:00:00:10:01; outport = "lr0-sw1"; reg9[[9]] = 1; >> next;) >> + table=??(lr_in_ip_routing_ecmp), priority=100 , match=(reg8[[0..15]] == >> <gid> && reg8[[16..31]] == <mid>), action=(reg0 = 10.0.20.10; reg5 = >> 10.0.20.1; eth.src = 00:00:00:00:20:01; outport = "lr0-sw2"; reg9[[9]] = 1; >> next;) >> table=??(lr_in_ip_routing_ecmp), priority=150 , match=(reg8[[0..15]] == >> 0), action=(next;) >> ]) >> > > _______________________________________________ > dev mailing list > [email protected] > https://mail.openvswitch.org/mailman/listinfo/ovs-dev _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
