On 11/13/26 5:34 PM, Numan Siddique via dev wrote: > On Tue, Nov 10, 2026 at 6:38 AM Takao Yatagai via dev > <[email protected]> wrote: >> >> From: Takao Yatagai <[email protected]> >> >> A load balancer on a distributed router without a distributed gateway >> port gets its lr_in_dnat flows on every chassis, but no un-DNAT: only >> gateway routers and routers with a DGP get one. Replies leave the >> router with the backend as source. >> >> With options:symmetric_return=true, the CMS guarantees that replies >> re-enter the router on the chassis that load balanced the request, for >> example because a downstream router translates the source of the >> requests with a distributed NAT rule (logical_port + external_mac). >> northd then adds, on such a router, the same priority 120 lr_out_undnat >> flow that routers with a DGP get, without the DGP and chassis terms. >> OVN does not verify the guarantee: replies that reach another chassis >> are not un-DNATed. >> >> Keeping the load balancer and the SNAT on different routers also keeps >> the configuration compatible with NB_Global use_common_zone. >> >> The option has no effect on gateway routers and routers with a DGP. >> >> Assisted-by: Claude Opus 5.5 >> Signed-off-by: Takao Yatagai <[email protected] > > > Hi Takao, > > Thanks for the patch. I've a few comments. >
Hi Takao, Numan, > So presently if a distributed logical router with no gateway ports has > a load balancer, > as you pointed in the commit message, it adds the logical flow in lr_in_dnat > to > load balance the traffic destined to the VIP, but it adds no flow in > lr_out_unndat. > > So it breaks the load balancer feature anyway. So I think we can just > drop the option "symmetric_return" > and add the lr_out_undnat flow regardless. > I agree, the fewer config knobs we have, the better. > If there is a distributed gateway port, the present code any way > matches "inport" and "is_chassis_resident". > > So I'd suggest not to match on "inport" and "is_chassis_resident" if a > router has no gateway ports. > > It would still be the responsibility of the CMS to ensure through the > proper logical topology > that the reply packet enters the same chassis. > > @Dumitru Ceara @Ales Musil @Mark Michelson Please let me know if > there are any concerns here. > > @Takao Yatagai Also you have to update the logical flow > documentation here - > https://github.com/ovn-org/ovn/blob/main/Documentation/ref/ovn-logical-flows.7.rst > I think we need to document this behavior properly at a higher level too though. In theory ovn-architecture might be the right place but we don't document load balancers much there. Maybe a section in the ovn-nb man page at least? > > Thanks > Numan > Thanks, Dumitru >> --- >> NEWS | 3 ++ >> northd/lb.c | 3 ++ >> northd/lb.h | 6 ++++ >> northd/northd.c | 9 ++++++ >> ovn-nb.xml | 30 ++++++++++++++++++++ >> tests/ovn-northd.at | 67 +++++++++++++++++++++++++++++++++++++++++++++ >> 6 files changed, 118 insertions(+) >> >> diff --git a/NEWS b/NEWS >> index 4176f8d7f..4e3f932d3 100644 >> --- a/NEWS >> +++ b/NEWS >> @@ -44,6 +44,9 @@ Post v26.09.0 >> table at startup; it defers its kernel route updates until the initial >> dump of the advertised routes of the local route-exchange datapaths is >> complete. >> + - Added the "symmetric_return" option to the Load_Balancer table. With >> + it, a distributed router without a distributed gateway port un-DNATs >> + the replies from the load balancer's backends. See ovn-nb(5). >> >> OVN v26.09.0 - xxx xx xxxx >> -------------------------- >> diff --git a/northd/lb.c b/northd/lb.c >> index 5ff9d1fad..081aa6ded 100644 >> --- a/northd/lb.c >> +++ b/northd/lb.c >> @@ -392,6 +392,9 @@ ovn_northd_lb_init(struct ovn_northd_lb *lb, >> lb->is_distributed = smap_get_bool(&nbrec_lb->options, "distributed", >> false); >> >> + lb->symmetric_return = smap_get_bool(&nbrec_lb->options, >> + "symmetric_return", false); >> + >> sset_init(&lb->ips_v4); >> sset_init(&lb->ips_v6); >> struct smap_node *node; >> diff --git a/northd/lb.h b/northd/lb.h >> index 7a98c2f55..145d13dfa 100644 >> --- a/northd/lb.h >> +++ b/northd/lb.h >> @@ -81,6 +81,12 @@ struct ovn_northd_lb { >> >> bool use_stateless_nat; >> >> + /* Set by options:symmetric_return. The CMS guarantees that replies to >> + * this load balancer re-enter the router on the chassis that load >> + * balanced the request, so a distributed router without a distributed >> + * gateway port un-DNATs them there. */ >> + bool symmetric_return; >> + >> char *hairpin_snat_ip; >> }; >> >> diff --git a/northd/northd.c b/northd/northd.c >> index 77c77f273..f88178301 100644 >> --- a/northd/northd.c >> +++ b/northd/northd.c >> @@ -14170,6 +14170,15 @@ build_lrouter_nat_flows_for_lb( >> >> if (vector_is_empty(&od->l3dgw_ports)) { >> bitmap_set1(gw_dp_bitmap[type], index); >> + /* A distributed router without a DGP load balances on every >> + * chassis. With symmetric_return, the CMS guarantees that the >> + * replies re-enter the router on the chassis that load balanced >> + * the request, so un-DNAT them there. */ >> + if (!od->is_gw_router && lb->symmetric_return >> + && !vector_is_empty(&lb_vip->backends)) { >> + build_lrouter_lb_undnat_flow(&ctx, type, od, NULL, NULL, >> + lb_dps->lflow_ref); >> + } >> } else { >> /* Create stateless LB NAT rules when using multiple DGPs and >> * use_stateless_nat is true. >> diff --git a/ovn-nb.xml b/ovn-nb.xml >> index d8c63567a..2ee7f1dec 100644 >> --- a/ovn-nb.xml >> +++ b/ovn-nb.xml >> @@ -2839,6 +2839,36 @@ or >> router. >> </column> >> >> + <column name="options" key="symmetric_return" >> + type='{"type": "boolean"}'> >> + <p> >> + On a distributed logical router with no distributed gateway port, >> + a load balancer is applied in <code>lr_in_dnat</code> on every >> + chassis, but the replies from its backends are not un-DNATed. >> + If set to <code>true</code>, such a router un-DNATs those replies >> + in its egress pipeline on the chassis where they pass it. The >> + default is <code>false</code>. >> + </p> >> + >> + <p> >> + The CMS must guarantee that replies to the load balanced traffic >> + re-enter the router on the chassis that load balanced the request, >> + for example because the source of the requests is translated by >> + a distributed NAT rule (see the <ref column="logical_port" >> + table="NAT"/> and <ref column="external_mac" table="NAT"/> columns >> + of the <ref table="NAT"/> table) on a downstream router whose >> + external IP is reachable only on that chassis. OVN does not >> verify >> + this. If replies reach another chassis, they are not un-DNATed. >> + Unlike <ref column="options" key="distributed"/>, this option does >> + not restrict the backends to logical ports on the local chassis. >> + </p> >> + >> + <p> >> + This option has no effect on gateway routers and on routers with >> + a distributed gateway port. >> + </p> >> + </column> >> + >> <column name="options" key="event" type='{"type": "boolean"}'> >> If set to <code>true</code> and the load balancer has no >> active backends, whenever a packet matches a VIP on this >> diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at >> index 6c37337ad..a81a12fd3 100644 >> --- a/tests/ovn-northd.at >> +++ b/tests/ovn-northd.at >> @@ -25340,3 +25340,70 @@ OVN_CLEANUP_NORTHD >> AT_CLEANUP >> ]) >> >> +OVN_FOR_EACH_NORTHD_NO_HV([ >> +AT_SETUP([Load balancer with symmetric_return on a router without DGP]) >> +ovn_start >> + >> +check ovn-nbctl ls-add sw0 >> +check ovn-nbctl lsp-add sw0 sw0-port1 \ >> + -- lsp-set-addresses sw0-port1 "50:54:00:00:00:03 10.0.0.3" >> + >> +check ovn-nbctl lr-add lr0 >> +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 >> +check ovn-nbctl lsp-add sw0 sw0-lr0 \ >> + -- lsp-set-type sw0-lr0 router \ >> + -- lsp-set-addresses sw0-lr0 router \ >> + -- lsp-set-options sw0-lr0 router-port=lr0-sw0 >> + >> +check ovn-nbctl lb-add lb0 172.168.0.200:80 10.0.0.4:8080 >> +check ovn-nbctl lr-lb-add lr0 lb0 >> +check ovn-nbctl --wait=sb sync >> + >> +# Default: the router load balances in lr_in_dnat with no chassis binding, >> +# but has no un-DNAT for the backend. >> +ovn-sbctl dump-flows lr0 > lr0flows >> +AT_CAPTURE_FILE([lr0flows]) >> +AT_CHECK([grep "lr_in_dnat" lr0flows | grep "ip4.dst == 172.168.0.200" | \ >> + grep -c "ct_lb_mark(backends=10.0.0.4:8080)"], [0], [1 >> +]) >> +AT_CHECK([grep "lr_in_dnat" lr0flows | grep "ip4.dst == 172.168.0.200" | \ >> + grep -c "is_chassis_resident"], [1], [0 >> +]) >> +AT_CHECK([grep "lr_out_undnat" lr0flows | grep -c "10.0.0.4"], [1], [0 >> +]) >> + >> +# symmetric_return: the backend's replies are un-DNATed; lr_in_dnat is >> +# unchanged. >> +check ovn-nbctl --wait=sb set load_balancer lb0 >> options:symmetric_return=true >> +ovn-sbctl dump-flows lr0 > lr0flows >> +AT_CHECK([grep "lr_in_dnat" lr0flows | grep "ip4.dst == 172.168.0.200" | \ >> + grep -c "ct_lb_mark(backends=10.0.0.4:8080)"], [0], [1 >> +]) >> +AT_CHECK([grep "lr_out_undnat" lr0flows | grep "priority=120" | \ >> + grep "ip4.src == 10.0.0.4 && tcp.src == 8080" | \ >> + grep -c "action=(ct_dnat;)"], [0], [1 >> +]) >> +AT_CHECK([grep "lr_out_undnat" lr0flows | grep "10.0.0.4" | \ >> + grep -c "is_chassis_resident"], [1], [0 >> +]) >> + >> +# With use_common_zone the un-DNAT uses the common zone, like the LB's >> +# own flows on routers with a distributed gateway port. >> +check ovn-nbctl --wait=sb set NB_Global . options:use_common_zone=true >> +ovn-sbctl dump-flows lr0 > lr0flows >> +AT_CHECK([grep "lr_out_undnat" lr0flows | grep "priority=120" | \ >> + grep "ip4.src == 10.0.0.4 && tcp.src == 8080" | \ >> + grep -c "action=(ct_dnat_in_czone;)"], [0], [1 >> +]) >> +check ovn-nbctl --wait=sb remove NB_Global . options use_common_zone >> + >> +# Removing the option removes the un-DNAT flow. >> +check ovn-nbctl --wait=sb remove load_balancer lb0 options symmetric_return >> +ovn-sbctl dump-flows lr0 > lr0flows >> +AT_CHECK([grep "lr_out_undnat" lr0flows | grep -c "10.0.0.4"], [1], [0 >> +]) >> + >> +OVN_CLEANUP_NORTHD >> +AT_CLEANUP >> +]) >> + >> -- >> 2.55.0 >> >> _______________________________________________ >> dev mailing list >> [email protected] >> https://mail.openvswitch.org/mailman/listinfo/ovs-dev > _______________________________________________ > dev mailing list > [email protected] > https://mail.openvswitch.org/mailman/listinfo/ovs-dev _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
