Numan,
I have a similar problem to Roberto, (not exact but I believe he will run
into this issue also) in which I need a G-ARP to generate a type 2 mac-ip
route in EVPN. When I add the NAT with the binding to the logical switch
port and external MAC I see a G-ARP immediately , the switch learns this
G-ARP and generates the correct ip route. The problem is garp is NOT sent
periodically so if the ARP cache times out you always need traffic from the
overlay to keep this alive. Should OVN be sending a periodic garp and if
so on what interval?
I'm running Ovn 22.03.0
root@usc01a-lab-ovn04:/home/gmckee# ovn-nbctl list nat
_uuid : 59489984-ac1e-4f4b-8dc8-cace49a64360
allowed_ext_ips : []
exempted_ext_ips : []
external_ids : {}
external_ip : "216.147.118.8"
external_mac : "0a:99:00:32:15:07"
external_port_range : ""
logical_ip : "10.200.1.12"
logical_port : c1-sw1-p2
options : {stateless="false"}
type : dnat_and_snat
root@usc01a-lab-ovn04:/home/gmckee# ovn-nbctl list Logical_Switch_Port
localswitch-lr0
_uuid : 010984e7-102c-4917-a6c9-59f6d554ffea
addresses : [router]
dhcpv4_options : []
dhcpv6_options : []
dynamic_addresses : []
enabled : []
external_ids : {}
ha_chassis_group : []
name : localswitch-lr0
options : {nat-addresses=router, router-port=c1-lr0-localswitch}
parent_name : []
port_security : []
tag : []
tag_request : []
type : router
up : true
On Thu, 1 Dec 2022 at 09:18, Numan Siddique via discuss <
ovs-discuss@openvswitch.org> wrote:
> On Thu, Dec 1, 2022 at 10:15 AM Roberto Bartzen Acosta via discuss
> <ovs-discuss@openvswitch.org> wrote:
> >
> > Hey folks,
> >
> > I would like some help to understand how DVR works in ovn/ovs.
> Currently, DVR support for FIP addresses works perfectly. However, I would
> like to understand if it is possible to extend this behavior to IPv6
> addresses. When I talk about IPv6 addresses, I'm referring to the GUA
> addresses that are allocated to VMs (e.g., considering an openstack
> deployment).
> >
> > I understand that there are other requirements for GUA addresses to be
> routed to external networks by OVN (here comes the work of the
> neutron-dynamic-routing project + FRR on hypervisors/openvswitch). For DVR
> to be enabled properly, the provider networks must be stretched over the
> Underlay Network and each Compute Node would have the bridge for external
> traffic. In an L3 Leaf-Spine Underlay network one option for this
> requirement to be reached is for the Underlay Network be able to stretch an
> L2 domain(VLAN) via VXLAN as dataplane and BGP EVPN as Control Plane. In
> this solution, the Leaf switches would need to work as HW VTEP Gateway to
> initiate and terminate the VXLANs tunnels and use BGP EVPN to learn and
> advertise the MAC Addresses from the Compute Node’s provider network.
> >
> > The common reference architecture is detailed in [picture 1] and the
> design for the DVR+FRR solution (IPv4/IPv6) is detailed in [picture 2].
> >
> > What's the problem? well, inbound traffic to a GUA address goes through
> the chassis where the router's external port resides.
> >
> >
> > Looking at the DVR implementation for IPv4, I see that the solution is
> heavily based on the idea of NAT. From initial support in [1], some fixes
> in [2] and [3], and scalability improvements in [4].
> >
> >
> > With some debugs in the openflow pipeline, I see the entries for snat
> and dnat (FIP = 200.201.0.226) with reference to is_chassis_resident (with
> cr-lrp).
> >
> > _uuid : d37d412b-28ed-4d55-8f97-533571ffb3c1
> > actions : "ct_dnat_in_czone(192.168.0.120);"
> > controller_meter : []
> > external_ids : {source="northd.c:12693", stage-hint="5e41244f",
> stage-name=lr_in_dnat}
> > logical_datapath : c3ea6752-5b60-40ec-9734-dcfecbb59f68
> > logical_dp_group : []
> > match : "ip && ip4.dst == 200.201.0.226 && inport ==
> \"lrp-bcf42b8c-5ca5-44d1-8065-6f36892a1473\" &&
> is_chassis_resident(\"cr-lrp-bcf42b8c-5ca5-44d1-8065-6f36892a1473\")"
> > pipeline : ingress
> > priority : 100
> > table_id : 6
> > tags : {}
> > hash : 0
> >
> >
> > Does distributed router support for IPv6 need some kind of NAT rule to
> work? And about NAT, I saw that there is a test case on system-ovn.at [5]
> that validates NAT operation for a distributed router with IPv6 N/S
> traffic, is that right? maybe for local addresses it might make some sense,
> but for N/S traffic - GUA address (does this test make sense?).
>
> In the case of DVR for FIPS, when a NAT entry is added for the FIP in
> the logical router, CMS has to set external_mac and logical_port
> fields.
> When these are set, ovn-controller claiming the VM of the FIP will
> reply for the ARP requests for the FIP. The same for IPv6 FIPs. Also
> the ovn-controller sends periodic GARPs.
>
> I think for the use case you mentioned, same has to happen. i.e For
> a Neighbor Solicitation request for the GUA of the VM, the reply
> should be sent from the same compute node hosting the VM.
>
> Can you please try adding a NAT entry in the logical router like below ?
>
> # ovn-nbctl --help | grep lr-nat-ad
> lr-nat-add ROUTER TYPE EXTERNAL_IP LOGICAL_IP [LOGICAL_PORT EXTERNAL_MAC]
>
> # ovn-nbctl lr-nat-add <logical_router> dnat_and_snat <VM GUA> <VM
> GUA> <VM logical port> <SOME CHOSEN RANDOM MAC>
>
> I've not tried this. If you notice the EXTERNAL_IP and LOGICAL_IP are
> the same (ie VM GUA). If you add an entry like this,
> ovn should add logical flows to respond to IPv6 NS requests for the VM GUA.
>
> If this doesn't work, then we need to think of a solution in OVN.
>
> Thanks
> Numan
>
> >
> > However, it is clear that when the external port of the router is in
> another chassis, the incoming traffic passes through the external port of
> the router [tcpdump 1]. The traffic is redirected to the chassis resident
> via geneve [dpcl 1] and goes through the resident chassis for the external
> destination GUA directly via vxlan[tcpdump 2] without going back through
> geneve to the router.
> >
> >
> > Any ideas about how to program the ovn so that the chassis with the GUA
> address can receive and send traffic directly? (without redirection to the
> centralized router).
> >
> > Regards,
> > Roberto
> >
> > [picture 1] -
> https://drive.google.com/file/d/1oaGmKbFGHqMwBxKVxsT4I-7rFt2pXSJW/view?usp=sharing
> > [picture 2] -
> https://drive.google.com/file/d/1E-MRe9WJubPz5ZP836bNPFPRMuPCt4_s/view?usp=sharing
> >
> >
> > [1]
> https://github.com/ovn-org/ovn/commit/ceacd9d49316d16b9273151bc1ecae9a2b2beeb8
> > [2]
> https://github.com/ovn-org/ovn/commit/551e3d989557bd2249d5bbe0978b44b775c5e619
> > [3]
> https://github.com/ovn-org/ovn/commit/8244c6b6bd8802a018e4ec3d3665510ebb16a9c7
> > [4]
> https://github.com/ovn-org/ovn/commit/2dc7869436de32205f60128172196b3a207ab265
> > [5] https://github.com/ovn-org/ovn/blob/main/tests/system-ovn.at#L3628
> >
> >
> > [tcpdump 1] - router external port chassis
> >
> > root@compute1:~# tcpdump -nni br-301 not port 22 -ne
> > tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
> > listening on br-301, link-type EN10MB (Ethernet), snapshot length 262144
> bytes
> > 14:54:23.442467 fa:16:3e:7b:d4:cf > a2:8f:16:4c:6e:e9, ethertype IPv4
> (0x0800), length 74: 200.201.0.110.57524 > 10.0.3.1.53: Flags [S], seq
> 2063839790, win 64492, options [mss 1402,sackOK,TS val 3936281288 ecr
> 0,nop,wscale 6], length 0
> > 14:54:24.142076 a2:8f:16:4c:6e:e9 > fa:16:3e:9d:67:1f, ethertype IPv6
> (0x86dd), length 118: 2001:db8:aaa::2 > 2001:db8:1234::140: ICMP6, echo
> request, id 9014, seq 5, length 64
> > 14:54:24.473037 fa:16:3e:7b:d4:cf > a2:8f:16:4c:6e:e9, ethertype IPv4
> (0x0800), length 74: 200.201.0.110.57524 > 10.0.3.1.53: Flags [S], seq
> 2063839790, win 64492, options [mss 1402,sackOK,TS val 3936282318 ecr
> 0,nop,wscale 6], length 0
> > 14:54:25.143193 a2:8f:16:4c:6e:e9 > fa:16:3e:9d:67:1f, ethertype IPv6
> (0x86dd), length 118: 2001:db8:aaa::2 > 2001:db8:1234::140: ICMP6, echo
> request, id 9014, seq 6, length 64
> > 14:54:25.166861 a2:8f:16:4c:6e:e9 > fa:16:3e:9d:67:1f, ethertype IPv6
> (0x86dd), length 86: fe80::a08f:16ff:fe4c:6ee9 > 2001:db8:4321:42::316:
> ICMP6, neighbor solicitation, who has 2001:db8:4321:42::316, length 32
> > 14:54:25.168384 fa:16:3e:9d:67:1f > a2:8f:16:4c:6e:e9, ethertype IPv6
> (0x86dd), length 86: 2001:db8:4321:42::316 > fe80::a08f:16ff:fe4c:6ee9:
> ICMP6, neighbor advertisement, tgt is 2001:db8:4321:42::316, length 32
> > 14:54:26.145280 a2:8f:16:4c:6e:e9 > fa:16:3e:9d:67:1f, ethertype IPv6
> (0x86dd), length 118: 2001:db8:aaa::2 > 2001:db8:1234::140: ICMP6, echo
> request, id 9014, seq 7, length 64
> > 14:54:26.489182 fa:16:3e:7b:d4:cf > a2:8f:16:4c:6e:e9, ethertype IPv4
> (0x0800), length 74: 200.201.0.110.57524 > 10.0.3.1.53: Flags [S], seq
> 2063839790, win 64492, options [mss 1402,sackOK,TS val 3936284334 ecr
> 0,nop,wscale 6], length 0
> > 14:54:27.146833 a2:8f:16:4c:6e:e9 > fa:16:3e:9d:67:1f, ethertype IPv6
> (0x86dd), length 118: 2001:db8:aaa::2 > 2001:db8:1234::140: ICMP6, echo
> request, id 9014, seq 8, length 64
> > ^C
> > 9 packets captured
> > 9 packets received by filter
> > 0 packets dropped by kernel
> >
> >
> >
> > [dpcl 1] - router external port chassis
> >
> > root@compute1:~# ovs-dpctl dump-flows
> >
> recirc_id(0),tunnel(tun_id=0x0,src=192.168.200.30,dst=192.168.200.10,flags(-df+csum+key)),in_port(1),eth(),eth_type(0x0800),ipv4(proto=17,frag=no),udp(dst=3784),
> packets:183319, bytes:12099054, used:0.228s,
> actions:userspace(pid=4294967295,slow_path(bfd))
> >
> recirc_id(0),in_port(5),eth(src=fa:16:3e:cc:79:0a,dst=fa:16:3e:2d:98:92),eth_type(0x0800),ipv4(src=192.168.0.199,dst=
> 0.0.0.0/128.0.0.0,proto=6,frag=no), packets:50, bytes:3700, used:2.976s,
> flags:S, actions:ct(zone=7),recirc(0x4dc)
> >
> recirc_id(0x4fa),in_port(5),eth(src=fa:16:3e:7b:d4:cf),eth_type(0x0800),ipv4(src=192.168.0.199,frag=no),
> packets:64306, bytes:4759173, used:2.976s, flags:S,
> actions:ct(commit,zone=1,nat(src=200.201.0.110)),recirc(0x4e2)
> >
> recirc_id(0x629),in_port(4),ct_state(+new-est-rel-rpl-inv+trk),ct_label(0/0x1),eth(src=a2:8f:16:4c:6e:e9,dst=fa:16:3e:9d:67:1f),eth_type(0x86dd),ipv6(src=2000::/ffc0::,dst=2001:db8:1234::140,proto=58,tclass=0/0x3,hlimit=63,frag=no),
> packets:19, bytes:2242, used:0.284s,
> actions:ct_clear,set(tunnel(tun_id=0x2,dst=192.168.200.11,ttl=64,tp_dst=6081,geneve({class=0x102,type=0x80,len=4,0x60008}),flags(df|csum|key))),set(eth(src=fa:16:3e:26:0b:f7,dst=fa:16:3e:c4:15:40)),set(ipv6(hlimit=62)),1
> >
> recirc_id(0x4e2),in_port(5),ct_state(+new-est-rel-rpl-inv+trk),ct_label(0/0x1),eth(src=fa:16:3e:7b:d4:cf,dst=a2:8f:16:4c:6e:e9),eth_type(0x0800),ipv4(dst=
> 0.0.0.0/128.0.0.0,frag=no), packets:64306, bytes:4759173, used:2.976s,
> flags:S, actions:ct_clear,4
> >
> recirc_id(0x4dc),in_port(5),ct_state(+new-est-rel-rpl-inv+trk),ct_label(0/0x1),eth(src=fa:16:3e:cc:79:0a,dst=fa:16:3e:2d:98:92),eth_type(0x0800),ipv4(src=192.168.0.199,dst=
> 8.0.0.0/248.0.0.0,proto=6,ttl=64,frag=no), packets:50, bytes:3700,
> used:2.976s, flags:S,
> actions:ct(commit,zone=7,label=0/0x1,nat(src)),set(eth(src=fa:16:3e:7b:d4:cf,dst=a2:8f:16:4c:6e:e9)),set(ipv4(ttl=63)),ct(zone=1,nat),recirc(0x4fa)
> >
> recirc_id(0),tunnel(tun_id=0x0,src=192.168.200.11,dst=192.168.200.10,flags(-df+csum+key)),in_port(1),eth(),eth_type(0x0800),ipv4(proto=17,frag=no),udp(dst=3784),
> packets:183257, bytes:12094962, used:0.404s,
> actions:userspace(pid=4294967295,slow_path(bfd))
> >
> recirc_id(0),in_port(4),eth(src=a2:8f:16:4c:6e:e9,dst=fa:16:3e:9d:67:1f),eth_type(0x86dd),ipv6(src=2001:db8:aaa::/ffff:ffff:ffff:ffff::,dst=2001:db8:1234::140,proto=58,hlimit=63,frag=no),
> packets:19, bytes:2242, used:0.284s, actions:ct(zone=1,nat),recirc(0x628)
> >
> recirc_id(0x628),in_port(4),eth(),eth_type(0x86dd),ipv6(dst=2001:db8:1234::140,frag=no),
> packets:19, bytes:2242, used:0.284s,
> actions:ct(commit,zone=1,nat(dst=2001:db8:1234::140)),recirc(0x629)
> >
> >
> > [tcpdump 2] - GUA address chassis
> >
> > root@compute2:~# tcpdump -nni br-301 not port 22 -ne
> > tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
> > listening on br-301, link-type EN10MB (Ethernet), snapshot length 262144
> bytes
> > 14:54:30.146532 fa:16:3e:0f:d9:1a > a2:8f:16:4c:6e:e9, ethertype IPv6
> (0x86dd), length 118: 2001:db8:1234::140 > 2001:db8:aaa::2: ICMP6, echo
> reply, id 9014, seq 11, length 64
> > 14:54:31.149147 fa:16:3e:0f:d9:1a > a2:8f:16:4c:6e:e9, ethertype IPv6
> (0x86dd), length 118: 2001:db8:1234::140 > 2001:db8:aaa::2: ICMP6, echo
> reply, id 9014, seq 12, length 64
> > ^C
> > 2 packets captured
> > 2 packets received by filter
> > 0 packets dropped by kernel
> >
> >
> >
> >
> >
> > ‘Esta mensagem é direcionada apenas para os endereços constantes no
> cabeçalho inicial. Se você não está listado nos endereços constantes no
> cabeçalho, pedimos-lhe que desconsidere completamente o conteúdo dessa
> mensagem e cuja cópia, encaminhamento e/ou execução das ações citadas estão
> imediatamente anuladas e proibidas’.
> >
> > ‘Apesar do Magazine Luiza tomar todas as precauções razoáveis para
> assegurar que nenhum vírus esteja presente nesse e-mail, a empresa não
> poderá aceitar a responsabilidade por quaisquer perdas ou danos causados
> por esse e-mail ou por seus anexos’.
> >
> > _______________________________________________
> > discuss mailing list
> > disc...@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
> _______________________________________________
> discuss mailing list
> disc...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>
_______________________________________________
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
