On Mon, Feb 28, 2022 at 11:40 AM Brendan Doyle <[email protected]> wrote: > > > > On 20/02/2022 23:38, Han Zhou wrote: > > > > On Thu, Feb 17, 2022 at 3:23 AM Brendan Doyle <[email protected]> > wrote: > > > > Hi, > > > > So I have a Distributed Gateway Port (DGP) on a Gateway through which > > VMs in the overlay can access > > underlay networks. If the VM is not on the chassis where the DGP is > > scheduled then the traffic takes > > the extra tunneled hop to the chassis where the DGP is and is then sent > > to the underlay via the localnet > > switch there. It would be great if I could avoid that extra hop, whilst > > still having the Gateway do NAT > > and routing. So I'm wondering if 'reside-on-redirect-chassis' or ' > > redirect-type' can be used to > > do this? and if so which one? Also will normal traffic between to VMs in > > the overlay on different chassis > > still be tunneled through the underlay? > > > > I'll do some experimentation later, but a nod in the right direction > > would be appreciated. > > > Hi Brendan, > > Not sure if I understood your question well. If you want to avoid the gateway > hop, you can just configure NAT rules with "dnat_and_snat" to do distributed > NAT. > 'reside-on-redirect-chassis' and 'redirect-type' don't seem to help for your > use case because you mentioned your VMs are in overlay, while those options > are for VLAN based logical networks. > > > I must admit I found the ovn-nb/ovn-architetcure doc quite confusing on this. > I tracked down > the patches and some ovn-discuss exchange and I think I understand it better > now. It > seems that rather then avoiding the extra hop to the gateway chassis, these > redirect options > just let you take that extra hop as a VLAN tagged packet in the underlay > rather than a geneve > encapsulated packet. Which is not what I was hoping they would do. I'm not > sure what you mean > by " you can just configure NAT rules with "dnat_and_snat" to do distributed > NAT" I could > not find any reference to "distributed NAT" in the man pages. We use a > distributed router > port Gateway for HA. Consider this config: > > switch 2ffbd2d8-5421-452a-b776-e32cc55f4789 (ls_vcn3) > port 269089c4-9464-41ec-9f63-6b3804b34b07 > addresses: ["52:54:00:30:38:35 192.16.1.5"] > port ls_vcn3_net1-lr_vcn3_net1 > type: router > addresses: ["40:44:00:00:00:90"] > router-port: lr_vcn3_net1-ls_vcn3_net1 > port 284195d2-9280-4334-900e-571ecd00327a > addresses: ["52:54:00:02:55:96 192.16.1.6"] > > switch af0f96b0-d7af-41ef-b5b7-fa8ae0186c2c (ls_vcn3_backbone) > port lsb_vcn3-lr_vcn3 > type: router > router-port: lr_vcn3-lsb_vcn3 > port lsb_vcn3_net1-lr_vcn3_net1 > type: router > router-port: lr_vcn3_net1-lsb_vcn3_net1 > > switch 29bdf630-8326-4aa4-aee1-d9bca9ea298c (ls_vcn3_external_ugw) > port ls_vcn3_external_ugw-lr_vcn3 > type: router > router-port: lr_vcn3-ls_vcn3_external_ugw > port ln-ls_vcn3_external_ugw > type: localnet > addresses: ["unknown"] > > router 638a1401-4e5f-450b-9d35-fe74defc8769 (lr_vcn3) > port lr_vcn3-ls_vcn3_external_ugw > mac: "40:44:00:00:01:60" > networks: ["253.255.80.4/16"] > gateway chassis: [sca15-rain17 sca15-rain05 sca15-rain06] > port lr_vcn3-lsb_vcn3 > mac: "40:44:00:00:01:50" > networks: ["253.255.25.2/25"] > nat cb880ef9-d51c-4265-abdc-db6accaa4d4a > external ip: "253.255.80.4" > logical ip: "192.16.0.0/16" > type: "snat" > > router f370260a-c90c-4824-91a8-fba787272dc2 (lr_vcn3_net1) > port lr_vcn3_net1-lsb_vcn3_net1 > mac: "40:44:00:00:00:a0" > networks: ["253.255.25.1/25"] > port lr_vcn3_net1-ls_vcn3_net1 > mac: "40:44:00:00:00:90" > networks: ["192.16.1.1/24"] > > ovn-nbctl lr-nat-list lr_vcn3 > TYPE EXTERNAL_IP EXTERNAL_PORT LOGICAL_IP > EXTERNAL_MAC LOGICAL_PORT > snat 253.255.80.4 192.16.0.0/16 > > ovn-sbctl show > > Chassis sca15-rain05 > hostname: sca15-rain05 > Encap geneve > ip: "253.255.2.5" > options: {csum="true"} > Port_Binding "269089c4-9464-41ec-9f63-6b3804b34b07" > Port_Binding cr-lr_vcn3-ls_vcn3_external_ugw > > Chassis sca15-rain06 > hostname: sca15-rain06 > Encap geneve > ip: "253.255.2.6" > options: {csum="true"} > Port_Binding "284195d2-9280-4334-900e-571ecd00327a" > > > So if the overlay VM with IP 192.16.1.5 (hosted on chassis sca15-rain05) is > sending traffic to the > underlay it will use "physical"/localnet bridge on sca15-rain05. But if the > overlay VM with IP 192.16.1.6 > (hosted on chassis sca15-rain06) is sending traffic to the underlay, the > traffic is tunneled to sca15-rain05 > first, because that is where the DR port is, and then it is sent to the > underlay via the physical/localnet > bridge there. It was that extra hop I was trying to avoid. i.e Have the NAT > done > by the flows on chassis sca15-rain06 and be sent to the underlay by the > localnet/physical bridge on that > chassis. But seems like this is not possible?
If you want to avoid this extra hop, you need to create a dnat_and_snat entry on router lr_vcn3 of type "dnat_and_snat" and also set external_mac and logical_port columns. Something like this: ovn-nbctl lr-nat-add lr_vcn3 dnat_and_snat 253.255.80.100 192.16.1.6 284195d2-9280-4334-900e-571ecd00327a 30:54:00:00:00:04 With your topology shared above, you have created a NAT entry in the router lr_vcn3 of type "snat".- And OVN doesn't support distributed snat. And I'm not sure if it is even possible (I could be wrong). If you define dnat_and_snat entry as I showed above in the example, then ovn-controller running on chassis sca15-rain06 will send garps to the underlay network with the IP-MAC - (253.255.80.100 - 30:54:00:00:00:04). And that ovn-controller will also do NATting from 192.169.1.6 to 253.255.80.100. This way, the packet will not be tunneled to sca15-rain05. Hope this makes sense. I think Han also suggested the same. Since your overlay logical switch ls_vcn3 doesn't have a localnet port, 'reside-on-redirect-chassis' option doesn't come into picture at all. This option is used when you create VMs on provider networks (like for example ls_vcn3_external_ugw) and you connect these logical switches to an OVN router. Thanks Numan > > Brendan > > > > > > Thanks, > Han > > > Thanks > > > > > > Brendan > > > > _______________________________________________ > > discuss mailing list > > [email protected] > > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss > > > _______________________________________________ > discuss mailing list > [email protected] > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss _______________________________________________ discuss mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
