Hi Dan, In 2.1, we had couple of bugs in supporting switch-over. They are getting fixed in R2.2
https://bugs.launchpad.net/juniperopenstack/+bug/1466328 https://bugs.launchpad.net/juniperopenstack/+bug/1464972 https://bugs.launchpad.net/juniperopenstack/+bug/1465122 Regards, Praveen From: Users <[email protected]<mailto:[email protected]>> on behalf of Dan Houtz <[email protected]<mailto:[email protected]>> Date: Thursday, June 25, 2015 at 11:55 PM To: Nischal Sheth <[email protected]<mailto:[email protected]>> Cc: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: [Users] Redundant L3 gateways using MX80s I should also mention that I am currently running 2.1 release of contrail. On Jun 25, 2015 12:58 PM, "Dan Houtz" <[email protected]<mailto:[email protected]>> wrote: Hi Nischal, I'm taking a look at the virtual-gateway-address knob now. Initial testing shows that I can transit out the virtual gateway but when I lose my first MX my TOR does not fail over to the 2nd MX. Considering both MXs are announcing the MAC/IP in EVPN I would think this must be a result of the TOR agent not sending the proper updates to TORs via OVSDB. Does this sound plausbible? Here are details on how I am currently testing: Config on MXs: ---------------- root@gw1z0> show configuration interfaces irb.5 family inet { address 10.10.210.146/29<http://10.10.210.146/29> { virtual-gateway-address 10.10.210.145; } } root@gw2z0> show configuration interfaces irb.5 family inet { address 10.10.210.147/29<http://10.10.210.147/29> { virtual-gateway-address 10.10.210.145; } } EVPN Database on MXs under normal circumstances: ------------------------------------------------- root@gw1z0> show evpn database Instance: Demo-NetworkA VLAN VNI MAC address Active source Timestamp IP address 4 00:00:5e:00:01:01 05:00:00:ff:84:00:00:00:04:00 Jun 25 19:14:25 10.10.210.145 4 50:c5:8d:d4:af:f0 10.10.214.71 Jun 25 19:14:25 10.10.210.147 4 54:9f:35:05:fa:2e 10.10.214.66 Jun 25 19:14:24 4 f8:c0:01:19:b1:88 irb.5 Jun 25 19:13:46 10.10.210.146 root@gw2z0> show evpn database Instance: Demo-NetworkA VLAN VNI MAC address Active source Timestamp IP address 4 00:00:5e:00:01:01 05:00:00:ff:84:00:00:00:04:00 Jun 25 17:43:27 10.10.210.145 4 50:c5:8d:d4:af:f0 irb.5 Jun 25 17:16:46 10.10.210.147 4 54:9f:35:05:fa:2e 10.10.214.66 Jun 25 17:17:15 4 f8:c0:01:19:b1:88 10.10.214.70 Jun 25 17:43:27 10.10.210.146 EVPN advertisements on MXs: ---------------------------- root@gw1z0> show route advertising-protocol bgp 10.10.210.139 __contrail__default-domain_admin_Network1.inet.0: 1 destinations, 2 routes (1 active, 0 holddown, 1 hidden) Prefix Nexthop MED Lclpref AS path * 192.168.1.0/24<http://192.168.1.0/24> Self 100 I Demo-NetworkA.evpn.0: 13 destinations, 13 routes (13 active, 0 holddown, 0 hidden) Prefix Nexthop MED Lclpref AS path 2:10.10.214.70:2::4::00:00:5e:00:01:01/304 * Self 100 I 2:10.10.214.70:2::4::f8:c0:01:19:b1:88/304 * Self 100 I 2:10.10.214.70:2::4::00:00:5e:00:01:01::10.10.210.145/304<http://10.10.210.145/304> * Self 100 I 2:10.10.214.70:2::4::f8:c0:01:19:b1:88::10.10.210.146/304<http://10.10.210.146/304> * Self 100 I 3:10.10.214.70:2::4::10.10.214.70/304<http://10.10.214.70/304> * Self 100 I __default_evpn__.evpn.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden) Prefix Nexthop MED Lclpref AS path 1:10.10.214.70:0::050000ff840000000400::FFFF:FFFF/304 * Self 100 I root@gw2z0> show route advertising-protocol bgp 10.10.210.139 __contrail__default-domain_admin_Network1.inet.0: 1 destinations, 2 routes (1 active, 0 holddown, 1 hidden) Prefix Nexthop MED Lclpref AS path * 192.168.1.0/24<http://192.168.1.0/24> Self 100 I Demo-NetworkA.evpn.0: 13 destinations, 13 routes (13 active, 0 holddown, 0 hidden) Prefix Nexthop MED Lclpref AS path 2:10.10.214.71:1::4::00:00:5e:00:01:01/304 * Self 100 I 2:10.10.214.71:1::4::50:c5:8d:d4:af:f0/304 * Self 100 I 2:10.10.214.71:1::4::00:00:5e:00:01:01::10.10.210.145/304<http://10.10.210.145/304> * Self 100 I 2:10.10.214.71:1::4::50:c5:8d:d4:af:f0::10.10.210.147/304<http://10.10.210.147/304> * Self 100 I 3:10.10.214.71:1::4::10.10.214.71/304<http://10.10.214.71/304> * Self 100 I __default_evpn__.evpn.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden) Prefix Nexthop MED Lclpref AS path 1:10.10.214.71:0::050000ff840000000400::FFFF:FFFF/304 * Self 100 I With both MXs up, my host can transit the gateway: ------------------------------------------------------ dhoutz@host2:/$ ping 8.8.8.8 PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. 64 bytes from 8.8.8.8<http://8.8.8.8>: icmp_seq=1 ttl=54 time=1.38 ms 64 bytes from 8.8.8.8<http://8.8.8.8>: icmp_seq=2 ttl=54 time=1.34 ms ^C --- 8.8.8.8 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1001ms rtt min/avg/max/mdev = 1.346/1.364/1.382/0.018 ms dhoutz@host2:/$ netstat -rn | grep 0.0.0.0 0.0.0.0 10.10.210.145 0.0.0.0 UG 0 0 0 em1 And my TOR shows a VTEP destination of MX1 (10.10.214.70): ------------------------------------------------------------- root@leaf1z0> show ovsdb mac | grep 00:00:5e:00:01:01 00:00:5e:00:01:01 0.0.0.0 Vxlan over Ipv4 10.10.214.70 If I shutdown MX1 my TOR never updates updates OVSDB and remains the same as above. Thanks! Dan On Thu, Jun 25, 2015 at 11:33 AM, Nischal Sheth <[email protected]<mailto:[email protected]>> wrote: Dan and Alex, A slight variation of the below is the recommended configuration. Instead of configuring the same IP on the MX IRBs, you would configure unique IPs on the IRBs so that connectivity to an individual MX can be tested. A new knob called virtual-gateway-address is added under the irb.xxx IP address. You would set the this to the subnet's gateway address on both MXs. The GW IP will be advertised with the same VRRP MAC by both MXs. Robert, this should work even with local VNI in future because the VNI is determined by doing a lookup of the MAC address on the TOR switch. Same if the encapsulation is MPLS instead of VXLAN. -Nischal On Jun 25, 2015, at 9:23 AM, Dan Houtz <[email protected]<mailto:[email protected]>> wrote: Hi Alex, I agree that anycastting the gateway would be ideal. I have tested this a bit but haven't had any luck thus fa as my TOR's only ever show one path. Even if I power down the MX that my TOR's have as the VTEP destination for the hard coded MAC, the entry is never removed from the OVSDB / mac table. Going to test this a bit more and see if I can find anything. I'll provide some results shortly. Thanks! Dan On Thu, Jun 25, 2015 at 3:56 AM, Alex Walker <[email protected]<mailto:[email protected]>> wrote: Hi Dan, Had you considered not using VRRP, and configuring both of your MX IRBs to use the same IP address and MAC? You’ll need to think about it for your own situation, but this could achieve the redundancy you require? Each MX will announce the same MAC via EVPN BGP. If either next-hop is used, either should forward traffic OK for you. Should one gateway fail, standard BGP path selection will then simply select the other gateway. You could probably even play with Local Preference to prefer one gateway over the other? Hope this helps – good luck with it! Alex. From: Users [mailto:[email protected]<mailto:[email protected]>] On Behalf Of Dan Houtz Sent: 25 June 2015 05:31 To: [email protected]<mailto:[email protected]> Subject: [Users] Redundant L3 gateways using MX80s Hi, I've been working hard to get BMS / TOR switch VXLAN overlays working and, with the gracious help of Praveen, I believe everything is mostly working as expected now. With that functionality in a working state I'm now looking at adding in L3 gateway support using a pair of MX80s. This seems pretty straight forward with one MX but I haven't quite got a redundant setup working properly. Are there any documented 'best practices' for this? In an attempt to add redundancy I am trying to configure VRRP (I'm not sure EVPN anycast gateways are viable in this setup. Hopefully someone can confirm). At present VRRP does converge across the overlay, my hosts can ping the physical IP addresses configured on each MX IRB, and my hosts succeed in getting ARP for the VRRP VIPs. They can not, however, complete a ping (I do have accept-data configured for VRRP so MX should reply to pings). After a bit of digging I believe this is because the virtual MAC address of the VIP is not being announced via BGP so my TOR switches never learn the MAC via OVSDB. root@gw1z0> show configuration interfaces irb.5 family inet { address 10.10.10.146/29<http://10.10.10.146/29> { vrrp-group 13 { virtual-address 10.10.10.145; priority 105; accept-data; } } } root@gw1z0> show route advertising-protocol bgp 10.10.10.139 Demo-NetworkA.evpn.0: 13 destinations, 13 routes (13 active, 0 holddown, 0 hidden) Prefix Nexthop MED Lclpref AS path 2:10.10.20.70:2::4::f8:c0:01:19:b1:88/304 * Self 100 I 2:10.10.20.70:2::4::f8:c0:01:19:b1:88::10.10.10.145/304<http://10.10.10.145/304> * Self 100 I 2:10.10.20.70:2::4::f8:c0:01:19:b1:88::10.10.10.146/304<http://10.10.10.146/304> * Self 100 I 3:10.10.20.70:2::4::10.10.20.70/304<http://10.10.20.70/304> * Self 100 I __default_evpn__.evpn.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden) Prefix Nexthop MED Lclpref AS path 1:10.10.20.70:0::050000ff840000000400::FFFF:FFFF/304 * Self 100 I As you can see in the advertised BGP routes above the MX is announcing .145 with the actual MAC of the IRB: root@gw1z0> show interfaces irb | grep hardware Current address: f8:c0:01:19:b1:88, Hardware address: f8:c0:01:19:b1:88 not the virtual MAC that has been assigned for VRRP: root@gw1z0> show vrrp extensive | grep mac Virtual Mac: 00:00:5e:00:01:0d I suspect that because the MX never announces an EVPN mac of 00:00:5e:00:01:0d, my TOR never learns where that MAC lives: oot@leaf1z0> show ovsdb mac Logical Switch Name: Contrail-12905c4b-60db-4d71-8fab-972051db1386 Mac IP Encapsulation Vtep Address Address Address ff:ff:ff:ff:ff:ff 0.0.0.0 Vxlan over Ipv4 10.10.20.65 d4:ae:52:c6:c9:e1 0.0.0.0 Vxlan over Ipv4 10.10.20.65 50:c5:8d:d4:af:f0 0.0.0.0 Vxlan over Ipv4 10.10.20.71 54:9f:35:05:fa:2e 0.0.0.0 Vxlan over Ipv4 10.10.20.66 f8:c0:01:19:b1:88 0.0.0.0 Vxlan over Ipv4 10.10.20.70 ff:ff:ff:ff:ff:ff 0.0.0.0 Vxlan over Ipv4 10.10.10.139 Is there perhaps a knob I'm missing? Is there something other then VRRP that is preferred for redundant gateways in Contrail? I have the same question out to a few other resources but figured someone on the mailing list might have some good pointers as well. Thanks! Dan _______________________________________________ Users mailing list [email protected]<mailto:[email protected]> http://lists.opencontrail.org/mailman/listinfo/users_lists.opencontrail.org
_______________________________________________ Users mailing list [email protected] http://lists.opencontrail.org/mailman/listinfo/users_lists.opencontrail.org
