Praveen, Thanks! Only one of those three links works ( https://bugs.launchpad.net/juniperopenstack/+bug/1464972). I've read through it and quite determine if it matches what I'm seeing 100% - I don't have any VMs/vrouters on compute nodes.
Is there anything I can do in 2.1 to workaround or can I possibly swap needed components if I compile the 2.2 versions? I've tried to get a full 2.2 system going from contrail-installer but havent quite been able to get it t work. Are 2.2 packages still expected to be released in the next couple days? I'd really like to get this working ASAP but if it's a lot less work to wait for the 2.2 official release I can just go that route as well. -Dan On Thu, Jun 25, 2015 at 1:54 PM, Praveen K V <[email protected]> wrote: > Hi Dan, > > In 2.1, we had couple of bugs in supporting switch-over. They are > getting fixed in R2.2 > > https://bugs.launchpad.net/juniperopenstack/+bug/1466328 > https://bugs.launchpad.net/juniperopenstack/+bug/1464972 > https://bugs.launchpad.net/juniperopenstack/+bug/1465122 > > Regards, > Praveen > > From: Users <[email protected]> on behalf of Dan > Houtz <[email protected]> > Date: Thursday, June 25, 2015 at 11:55 PM > To: Nischal Sheth <[email protected]> > Cc: "[email protected]" <[email protected]> > Subject: Re: [Users] Redundant L3 gateways using MX80s > > I should also mention that I am currently running 2.1 release of > contrail. > On Jun 25, 2015 12:58 PM, "Dan Houtz" <[email protected]> wrote: > >> Hi Nischal, >> >> I'm taking a look at the virtual-gateway-address knob now. Initial >> testing shows that I can transit out the virtual gateway but when I lose my >> first MX my TOR does not fail over to the 2nd MX. Considering both MXs are >> announcing the MAC/IP in EVPN I would think this must be a result of the >> TOR agent not sending the proper updates to TORs via OVSDB. Does this sound >> plausbible? >> >> Here are details on how I am currently testing: >> >> Config on MXs: >> ---------------- >> >> root@gw1z0> show configuration interfaces irb.5 >> family inet { >> address 10.10.210.146/29 { >> virtual-gateway-address 10.10.210.145; >> } >> } >> >> root@gw2z0> show configuration interfaces irb.5 >> family inet { >> address 10.10.210.147/29 { >> virtual-gateway-address 10.10.210.145; >> } >> } >> >> >> EVPN Database on MXs under normal circumstances: >> ------------------------------------------------- >> >> root@gw1z0> show evpn database >> Instance: Demo-NetworkA >> VLAN VNI MAC address Active source >> Timestamp IP address >> 4 00:00:5e:00:01:01 05:00:00:ff:84:00:00:00:04:00 Jun 25 >> 19:14:25 10.10.210.145 >> 4 50:c5:8d:d4:af:f0 10.10.214.71 Jun 25 >> 19:14:25 10.10.210.147 >> 4 54:9f:35:05:fa:2e 10.10.214.66 Jun 25 >> 19:14:24 >> 4 f8:c0:01:19:b1:88 irb.5 Jun 25 >> 19:13:46 10.10.210.146 >> >> >> root@gw2z0> show evpn database >> Instance: Demo-NetworkA >> VLAN VNI MAC address Active source >> Timestamp IP address >> 4 00:00:5e:00:01:01 05:00:00:ff:84:00:00:00:04:00 Jun 25 >> 17:43:27 10.10.210.145 >> 4 50:c5:8d:d4:af:f0 irb.5 Jun 25 >> 17:16:46 10.10.210.147 >> 4 54:9f:35:05:fa:2e 10.10.214.66 Jun 25 >> 17:17:15 >> 4 f8:c0:01:19:b1:88 10.10.214.70 Jun 25 >> 17:43:27 10.10.210.146 >> >> >> EVPN advertisements on MXs: >> ---------------------------- >> >> root@gw1z0> show route advertising-protocol bgp 10.10.210.139 >> >> __contrail__default-domain_admin_Network1.inet.0: 1 destinations, 2 >> routes (1 active, 0 holddown, 1 hidden) >> Prefix Nexthop MED Lclpref AS path >> * 192.168.1.0/24 Self 100 I >> >> Demo-NetworkA.evpn.0: 13 destinations, 13 routes (13 active, 0 holddown, >> 0 hidden) >> Prefix Nexthop MED Lclpref AS path >> 2:10.10.214.70:2::4::00:00:5e:00:01:01/304 >> * Self 100 I >> 2:10.10.214.70:2::4::f8:c0:01:19:b1:88/304 >> * Self 100 I >> 2:10.10.214.70:2::4::00:00:5e:00:01:01::10.10.210.145/304 >> * Self 100 I >> 2:10.10.214.70:2::4::f8:c0:01:19:b1:88::10.10.210.146/304 >> * Self 100 I >> 3:10.10.214.70:2::4::10.10.214.70/304 >> * Self 100 I >> >> __default_evpn__.evpn.0: 1 destinations, 1 routes (1 active, 0 holddown, >> 0 hidden) >> Prefix Nexthop MED Lclpref AS path >> 1:10.10.214.70:0::050000ff840000000400::FFFF:FFFF/304 >> * Self 100 I >> >> >> root@gw2z0> show route advertising-protocol bgp 10.10.210.139 >> >> __contrail__default-domain_admin_Network1.inet.0: 1 destinations, 2 >> routes (1 active, 0 holddown, 1 hidden) >> Prefix Nexthop MED Lclpref AS path >> * 192.168.1.0/24 Self 100 I >> >> Demo-NetworkA.evpn.0: 13 destinations, 13 routes (13 active, 0 holddown, >> 0 hidden) >> Prefix Nexthop MED Lclpref AS path >> 2:10.10.214.71:1::4::00:00:5e:00:01:01/304 >> * Self 100 I >> 2:10.10.214.71:1::4::50:c5:8d:d4:af:f0/304 >> * Self 100 I >> 2:10.10.214.71:1::4::00:00:5e:00:01:01::10.10.210.145/304 >> * Self 100 I >> 2:10.10.214.71:1::4::50:c5:8d:d4:af:f0::10.10.210.147/304 >> * Self 100 I >> 3:10.10.214.71:1::4::10.10.214.71/304 >> * Self 100 I >> >> __default_evpn__.evpn.0: 1 destinations, 1 routes (1 active, 0 holddown, >> 0 hidden) >> Prefix Nexthop MED Lclpref AS path >> 1:10.10.214.71:0::050000ff840000000400::FFFF:FFFF/304 >> * Self 100 I >> >> >> With both MXs up, my host can transit the gateway: >> ------------------------------------------------------ >> >> dhoutz@host2:/$ ping 8.8.8.8 >> PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. >> 64 bytes from 8.8.8.8: icmp_seq=1 ttl=54 time=1.38 ms >> 64 bytes from 8.8.8.8: icmp_seq=2 ttl=54 time=1.34 ms >> ^C >> --- 8.8.8.8 ping statistics --- >> 2 packets transmitted, 2 received, 0% packet loss, time 1001ms >> rtt min/avg/max/mdev = 1.346/1.364/1.382/0.018 ms >> >> >> dhoutz@host2:/$ netstat -rn | grep 0.0.0.0 >> 0.0.0.0 10.10.210.145 0.0.0.0 UG 0 0 0 em1 >> >> >> And my TOR shows a VTEP destination of MX1 (10.10.214.70): >> ------------------------------------------------------------- >> root@leaf1z0> show ovsdb mac | grep 00:00:5e:00:01:01 >> 00:00:5e:00:01:01 0.0.0.0 Vxlan over Ipv4 >> 10.10.214.70 >> >> >> If I shutdown MX1 my TOR never updates updates OVSDB and remains the same >> as above. >> >> Thanks! >> Dan >> >> On Thu, Jun 25, 2015 at 11:33 AM, Nischal Sheth <[email protected]> >> wrote: >> >>> >>> Dan and Alex, >>> >>> A slight variation of the below is the recommended configuration. >>> Instead of >>> configuring the same IP on the MX IRBs, you would configure unique IPs on >>> the IRBs so that connectivity to an individual MX can be tested. A new >>> knob >>> called virtual-gateway-address is added under the irb.xxx IP address. You >>> would set the this to the subnet's gateway address on both MXs. The GW IP >>> will be advertised with the same VRRP MAC by both MXs. >>> >>> Robert, this should work even with local VNI in future because the VNI >>> is >>> determined by doing a lookup of the MAC address on the TOR switch. >>> Same if the encapsulation is MPLS instead of VXLAN. >>> >>> -Nischal >>> >>> On Jun 25, 2015, at 9:23 AM, Dan Houtz <[email protected]> wrote: >>> >>> Hi Alex, >>> >>> I agree that anycastting the gateway would be ideal. I have tested this >>> a bit but haven't had any luck thus fa as my TOR's only ever show one path. >>> Even if I power down the MX that my TOR's have as the VTEP destination for >>> the hard coded MAC, the entry is never removed from the OVSDB / mac table. >>> >>> Going to test this a bit more and see if I can find anything. I'll >>> provide some results shortly. >>> >>> Thanks! >>> Dan >>> >>> >>> >>> >>> On Thu, Jun 25, 2015 at 3:56 AM, Alex Walker < >>> [email protected]> wrote: >>> >>>> Hi Dan, >>>> >>>> >>>> >>>> Had you considered not using VRRP, and configuring both of your MX IRBs >>>> to use the same IP address and MAC? You’ll need to think about it for your >>>> own situation, but this could achieve the redundancy you require? >>>> >>>> >>>> >>>> Each MX will announce the same MAC via EVPN BGP. If either next-hop is >>>> used, either should forward traffic OK for you. Should one gateway fail, >>>> standard BGP path selection will then simply select the other gateway. You >>>> could probably even play with Local Preference to prefer one gateway over >>>> the other? >>>> >>>> >>>> >>>> Hope this helps – good luck with it! >>>> >>>> >>>> >>>> Alex. >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> *From:* Users [mailto:[email protected]] *On Behalf >>>> Of *Dan Houtz >>>> *Sent:* 25 June 2015 05:31 >>>> *To:* [email protected] >>>> *Subject:* [Users] Redundant L3 gateways using MX80s >>>> >>>> >>>> >>>> Hi, >>>> >>>> >>>> >>>> I've been working hard to get BMS / TOR switch VXLAN overlays working >>>> and, with the gracious help of Praveen, I believe everything is mostly >>>> working as expected now. With that functionality in a working state I'm now >>>> looking at adding in L3 gateway support using a pair of MX80s. This seems >>>> pretty straight forward with one MX but I haven't quite got a redundant >>>> setup working properly. Are there any documented 'best practices' for >>>> this? In an attempt to add redundancy I am trying to configure VRRP (I'm >>>> not sure EVPN anycast gateways are viable in this setup. Hopefully someone >>>> can confirm). >>>> >>>> >>>> >>>> At present VRRP does converge across the overlay, my hosts can ping the >>>> physical IP addresses configured on each MX IRB, and my hosts succeed in >>>> getting ARP for the VRRP VIPs. They can not, however, complete a ping (I do >>>> have accept-data configured for VRRP so MX should reply to pings). After a >>>> bit of digging I believe this is because the virtual MAC address of the VIP >>>> is not being announced via BGP so my TOR switches never learn the MAC via >>>> OVSDB. >>>> >>>> >>>> >>>> root@gw1z0> show configuration interfaces irb.5 >>>> >>>> family inet { >>>> >>>> address 10.10.10.146/29 { >>>> >>>> vrrp-group 13 { >>>> >>>> virtual-address 10.10.10.145; >>>> >>>> priority 105; >>>> >>>> accept-data; >>>> >>>> } >>>> >>>> } >>>> >>>> } >>>> >>>> >>>> >>>> root@gw1z0> show route advertising-protocol bgp 10.10.10.139 >>>> >>>> >>>> >>>> Demo-NetworkA.evpn.0: 13 destinations, 13 routes (13 active, 0 >>>> holddown, 0 hidden) >>>> >>>> Prefix Nexthop MED Lclpref AS >>>> path >>>> >>>> 2:10.10.20.70:2::4::f8:c0:01:19:b1:88/304 >>>> >>>> * Self 100 I >>>> >>>> 2:10.10.20.70:2::4::f8:c0:01:19:b1:88::10.10.10.145/304 >>>> >>>> * Self 100 I >>>> >>>> 2:10.10.20.70:2::4::f8:c0:01:19:b1:88::10.10.10.146/304 >>>> >>>> * Self 100 I >>>> >>>> 3:10.10.20.70:2::4::10.10.20.70/304 >>>> >>>> * Self 100 I >>>> >>>> >>>> >>>> __default_evpn__.evpn.0: 1 destinations, 1 routes (1 active, 0 >>>> holddown, 0 hidden) >>>> >>>> Prefix Nexthop MED Lclpref AS >>>> path >>>> >>>> 1:10.10.20.70:0::050000ff840000000400::FFFF:FFFF/304 >>>> >>>> * Self 100 I >>>> >>>> >>>> >>>> As you can see in the advertised BGP routes above the MX is announcing >>>> .145 with the actual MAC of the IRB: >>>> >>>> >>>> >>>> root@gw1z0> show interfaces irb | grep hardware >>>> >>>> Current address: f8:c0:01:19:b1:88, Hardware address: >>>> f8:c0:01:19:b1:88 >>>> >>>> >>>> >>>> not the virtual MAC that has been assigned for VRRP: >>>> >>>> >>>> >>>> root@gw1z0> show vrrp extensive | grep mac >>>> >>>> Virtual Mac: 00:00:5e:00:01:0d >>>> >>>> >>>> >>>> I suspect that because the MX never announces an EVPN mac of >>>> 00:00:5e:00:01:0d, my TOR never learns where that MAC lives: >>>> >>>> >>>> >>>> oot@leaf1z0> show ovsdb mac >>>> >>>> Logical Switch Name: Contrail-12905c4b-60db-4d71-8fab-972051db1386 >>>> >>>> Mac IP Encapsulation Vtep >>>> >>>> Address Address Address >>>> >>>> ff:ff:ff:ff:ff:ff 0.0.0.0 Vxlan over Ipv4 >>>> 10.10.20.65 >>>> >>>> d4:ae:52:c6:c9:e1 0.0.0.0 Vxlan over Ipv4 >>>> 10.10.20.65 >>>> >>>> 50:c5:8d:d4:af:f0 0.0.0.0 Vxlan over Ipv4 >>>> 10.10.20.71 >>>> >>>> 54:9f:35:05:fa:2e 0.0.0.0 Vxlan over Ipv4 >>>> 10.10.20.66 >>>> >>>> f8:c0:01:19:b1:88 0.0.0.0 Vxlan over Ipv4 >>>> 10.10.20.70 >>>> >>>> ff:ff:ff:ff:ff:ff 0.0.0.0 Vxlan over Ipv4 >>>> 10.10.10.139 >>>> >>>> >>>> >>>> >>>> >>>> Is there perhaps a knob I'm missing? Is there something other then VRRP >>>> that is preferred for redundant gateways in Contrail? >>>> >>>> >>>> >>>> I have the same question out to a few other resources but figured >>>> someone on the mailing list might have some good pointers as well. >>>> >>>> >>>> >>>> Thanks! >>>> >>>> Dan >>>> >>> >>> _______________________________________________ >>> Users mailing list >>> [email protected] >>> >>> http://lists.opencontrail.org/mailman/listinfo/users_lists.opencontrail.org >>> >>> >>> >>
_______________________________________________ Users mailing list [email protected] http://lists.opencontrail.org/mailman/listinfo/users_lists.opencontrail.org
