Active/Active gateways appear to be working now with 2.2. I am curious if there is a way to make the virtual IP address pingable. In the VRRP there exists an 'accept-data' knob. Is there something similar for virtual-gateway-address?
Thanks! Dan On Thu, Jun 25, 2015 at 7:19 PM, Nischal Sheth <[email protected]> wrote: > Hi Dan, > > Please check https://bugs.launchpad.net/juniperopenstack/+bug/1465122 > again. > Have marked it public - sounds like it's the one you need. > > -Nischal > > > On Jun 25, 2015, at 12:21 PM, Dan Houtz <[email protected]> wrote: > > Praveen, > > Thanks! Only one of those three links works ( > https://bugs.launchpad.net/juniperopenstack/+bug/1464972). I've read > through it and quite determine if it matches what I'm seeing 100% - I don't > have any VMs/vrouters on compute nodes. > > Is there anything I can do in 2.1 to workaround or can I possibly swap > needed components if I compile the 2.2 versions? I've tried to get a full > 2.2 system going from contrail-installer but havent quite been able to get > it t work. > > Are 2.2 packages still expected to be released in the next couple days? > I'd really like to get this working ASAP but if it's a lot less work to > wait for the 2.2 official release I can just go that route as well. > > -Dan > > On Thu, Jun 25, 2015 at 1:54 PM, Praveen K V <[email protected]> > wrote: > >> Hi Dan, >> >> In 2.1, we had couple of bugs in supporting switch-over. They are >> getting fixed in R2.2 >> >> https://bugs.launchpad.net/juniperopenstack/+bug/1466328 >> https://bugs.launchpad.net/juniperopenstack/+bug/1464972 >> https://bugs.launchpad.net/juniperopenstack/+bug/1465122 >> >> Regards, >> Praveen >> >> From: Users <[email protected]> on behalf of Dan >> Houtz <[email protected]> >> Date: Thursday, June 25, 2015 at 11:55 PM >> To: Nischal Sheth <[email protected]> >> Cc: "[email protected]" <[email protected]> >> Subject: Re: [Users] Redundant L3 gateways using MX80s >> >> I should also mention that I am currently running 2.1 release of >> contrail. >> On Jun 25, 2015 12:58 PM, "Dan Houtz" <[email protected]> wrote: >> >>> Hi Nischal, >>> >>> I'm taking a look at the virtual-gateway-address knob now. Initial >>> testing shows that I can transit out the virtual gateway but when I lose my >>> first MX my TOR does not fail over to the 2nd MX. Considering both MXs are >>> announcing the MAC/IP in EVPN I would think this must be a result of the >>> TOR agent not sending the proper updates to TORs via OVSDB. Does this sound >>> plausbible? >>> >>> Here are details on how I am currently testing: >>> >>> Config on MXs: >>> ---------------- >>> >>> root@gw1z0> show configuration interfaces irb.5 >>> family inet { >>> address 10.10.210.146/29 { >>> virtual-gateway-address 10.10.210.145; >>> } >>> } >>> >>> root@gw2z0> show configuration interfaces irb.5 >>> family inet { >>> address 10.10.210.147/29 { >>> virtual-gateway-address 10.10.210.145; >>> } >>> } >>> >>> >>> EVPN Database on MXs under normal circumstances: >>> ------------------------------------------------- >>> >>> root@gw1z0> show evpn database >>> Instance: Demo-NetworkA >>> VLAN VNI MAC address Active source >>> Timestamp IP address >>> 4 00:00:5e:00:01:01 05:00:00:ff:84:00:00:00:04:00 Jun 25 >>> 19:14:25 10.10.210.145 >>> 4 50:c5:8d:d4:af:f0 10.10.214.71 Jun 25 >>> 19:14:25 10.10.210.147 >>> 4 54:9f:35:05:fa:2e 10.10.214.66 Jun 25 >>> 19:14:24 >>> 4 f8:c0:01:19:b1:88 irb.5 Jun 25 >>> 19:13:46 10.10.210.146 >>> >>> >>> root@gw2z0> show evpn database >>> Instance: Demo-NetworkA >>> VLAN VNI MAC address Active source >>> Timestamp IP address >>> 4 00:00:5e:00:01:01 05:00:00:ff:84:00:00:00:04:00 Jun 25 >>> 17:43:27 10.10.210.145 >>> 4 50:c5:8d:d4:af:f0 irb.5 Jun 25 >>> 17:16:46 10.10.210.147 >>> 4 54:9f:35:05:fa:2e 10.10.214.66 Jun 25 >>> 17:17:15 >>> 4 f8:c0:01:19:b1:88 10.10.214.70 Jun 25 >>> 17:43:27 10.10.210.146 >>> >>> >>> EVPN advertisements on MXs: >>> ---------------------------- >>> >>> root@gw1z0> show route advertising-protocol bgp 10.10.210.139 >>> >>> __contrail__default-domain_admin_Network1.inet.0: 1 destinations, 2 >>> routes (1 active, 0 holddown, 1 hidden) >>> Prefix Nexthop MED Lclpref AS path >>> * 192.168.1.0/24 Self 100 I >>> >>> Demo-NetworkA.evpn.0: 13 destinations, 13 routes (13 active, 0 holddown, >>> 0 hidden) >>> Prefix Nexthop MED Lclpref AS path >>> 2:10.10.214.70:2::4::00:00:5e:00:01:01/304 >>> * Self 100 I >>> 2:10.10.214.70:2::4::f8:c0:01:19:b1:88/304 >>> * Self 100 I >>> 2:10.10.214.70:2::4::00:00:5e:00:01:01::10.10.210.145/304 >>> * Self 100 I >>> 2:10.10.214.70:2::4::f8:c0:01:19:b1:88::10.10.210.146/304 >>> * Self 100 I >>> 3:10.10.214.70:2::4::10.10.214.70/304 >>> * Self 100 I >>> >>> __default_evpn__.evpn.0: 1 destinations, 1 routes (1 active, 0 holddown, >>> 0 hidden) >>> Prefix Nexthop MED Lclpref AS path >>> 1:10.10.214.70:0::050000ff840000000400::FFFF:FFFF/304 >>> * Self 100 I >>> >>> >>> root@gw2z0> show route advertising-protocol bgp 10.10.210.139 >>> >>> __contrail__default-domain_admin_Network1.inet.0: 1 destinations, 2 >>> routes (1 active, 0 holddown, 1 hidden) >>> Prefix Nexthop MED Lclpref AS path >>> * 192.168.1.0/24 Self 100 I >>> >>> Demo-NetworkA.evpn.0: 13 destinations, 13 routes (13 active, 0 holddown, >>> 0 hidden) >>> Prefix Nexthop MED Lclpref AS path >>> 2:10.10.214.71:1::4::00:00:5e:00:01:01/304 >>> * Self 100 I >>> 2:10.10.214.71:1::4::50:c5:8d:d4:af:f0/304 >>> * Self 100 I >>> 2:10.10.214.71:1::4::00:00:5e:00:01:01::10.10.210.145/304 >>> * Self 100 I >>> 2:10.10.214.71:1::4::50:c5:8d:d4:af:f0::10.10.210.147/304 >>> * Self 100 I >>> 3:10.10.214.71:1::4::10.10.214.71/304 >>> * Self 100 I >>> >>> __default_evpn__.evpn.0: 1 destinations, 1 routes (1 active, 0 holddown, >>> 0 hidden) >>> Prefix Nexthop MED Lclpref AS path >>> 1:10.10.214.71:0::050000ff840000000400::FFFF:FFFF/304 >>> * Self 100 I >>> >>> >>> With both MXs up, my host can transit the gateway: >>> ------------------------------------------------------ >>> >>> dhoutz@host2:/$ ping 8.8.8.8 >>> PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. >>> 64 bytes from 8.8.8.8: icmp_seq=1 ttl=54 time=1.38 ms >>> 64 bytes from 8.8.8.8: icmp_seq=2 ttl=54 time=1.34 ms >>> ^C >>> --- 8.8.8.8 ping statistics --- >>> 2 packets transmitted, 2 received, 0% packet loss, time 1001ms >>> rtt min/avg/max/mdev = 1.346/1.364/1.382/0.018 ms >>> >>> >>> dhoutz@host2:/$ netstat -rn | grep 0.0.0.0 >>> 0.0.0.0 10.10.210.145 0.0.0.0 UG 0 0 0 >>> em1 >>> >>> >>> And my TOR shows a VTEP destination of MX1 (10.10.214.70): >>> ------------------------------------------------------------- >>> root@leaf1z0> show ovsdb mac | grep 00:00:5e:00:01:01 >>> 00:00:5e:00:01:01 0.0.0.0 Vxlan over Ipv4 >>> 10.10.214.70 >>> >>> >>> If I shutdown MX1 my TOR never updates updates OVSDB and remains the >>> same as above. >>> >>> Thanks! >>> Dan >>> >>> On Thu, Jun 25, 2015 at 11:33 AM, Nischal Sheth <[email protected]> >>> wrote: >>> >>>> >>>> Dan and Alex, >>>> >>>> A slight variation of the below is the recommended configuration. >>>> Instead of >>>> configuring the same IP on the MX IRBs, you would configure unique IPs >>>> on >>>> the IRBs so that connectivity to an individual MX can be tested. A new >>>> knob >>>> called virtual-gateway-address is added under the irb.xxx IP address. >>>> You >>>> would set the this to the subnet's gateway address on both MXs. The GW >>>> IP >>>> will be advertised with the same VRRP MAC by both MXs. >>>> >>>> Robert, this should work even with local VNI in future because the >>>> VNI is >>>> determined by doing a lookup of the MAC address on the TOR switch. >>>> Same if the encapsulation is MPLS instead of VXLAN. >>>> >>>> -Nischal >>>> >>>> On Jun 25, 2015, at 9:23 AM, Dan Houtz <[email protected]> wrote: >>>> >>>> Hi Alex, >>>> >>>> I agree that anycastting the gateway would be ideal. I have tested this >>>> a bit but haven't had any luck thus fa as my TOR's only ever show one path. >>>> Even if I power down the MX that my TOR's have as the VTEP destination for >>>> the hard coded MAC, the entry is never removed from the OVSDB / mac table. >>>> >>>> Going to test this a bit more and see if I can find anything. I'll >>>> provide some results shortly. >>>> >>>> Thanks! >>>> Dan >>>> >>>> >>>> >>>> >>>> On Thu, Jun 25, 2015 at 3:56 AM, Alex Walker < >>>> [email protected]> wrote: >>>> >>>>> Hi Dan, >>>>> >>>>> >>>>> >>>>> Had you considered not using VRRP, and configuring both of your MX >>>>> IRBs to use the same IP address and MAC? You’ll need to think about it for >>>>> your own situation, but this could achieve the redundancy you require? >>>>> >>>>> >>>>> >>>>> Each MX will announce the same MAC via EVPN BGP. If either next-hop is >>>>> used, either should forward traffic OK for you. Should one gateway fail, >>>>> standard BGP path selection will then simply select the other gateway. You >>>>> could probably even play with Local Preference to prefer one gateway over >>>>> the other? >>>>> >>>>> >>>>> >>>>> Hope this helps – good luck with it! >>>>> >>>>> >>>>> >>>>> Alex. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> *From:* Users [mailto:[email protected]] *On >>>>> Behalf Of *Dan Houtz >>>>> *Sent:* 25 June 2015 05:31 >>>>> *To:* [email protected] >>>>> *Subject:* [Users] Redundant L3 gateways using MX80s >>>>> >>>>> >>>>> >>>>> Hi, >>>>> >>>>> >>>>> >>>>> I've been working hard to get BMS / TOR switch VXLAN overlays working >>>>> and, with the gracious help of Praveen, I believe everything is mostly >>>>> working as expected now. With that functionality in a working state I'm >>>>> now >>>>> looking at adding in L3 gateway support using a pair of MX80s. This seems >>>>> pretty straight forward with one MX but I haven't quite got a redundant >>>>> setup working properly. Are there any documented 'best practices' for >>>>> this? In an attempt to add redundancy I am trying to configure VRRP (I'm >>>>> not sure EVPN anycast gateways are viable in this setup. Hopefully someone >>>>> can confirm). >>>>> >>>>> >>>>> >>>>> At present VRRP does converge across the overlay, my hosts can ping >>>>> the physical IP addresses configured on each MX IRB, and my hosts succeed >>>>> in getting ARP for the VRRP VIPs. They can not, however, complete a ping >>>>> (I >>>>> do have accept-data configured for VRRP so MX should reply to pings). >>>>> After >>>>> a bit of digging I believe this is because the virtual MAC address of the >>>>> VIP is not being announced via BGP so my TOR switches never learn the MAC >>>>> via OVSDB. >>>>> >>>>> >>>>> >>>>> root@gw1z0> show configuration interfaces irb.5 >>>>> >>>>> family inet { >>>>> >>>>> address 10.10.10.146/29 { >>>>> >>>>> vrrp-group 13 { >>>>> >>>>> virtual-address 10.10.10.145; >>>>> >>>>> priority 105; >>>>> >>>>> accept-data; >>>>> >>>>> } >>>>> >>>>> } >>>>> >>>>> } >>>>> >>>>> >>>>> >>>>> root@gw1z0> show route advertising-protocol bgp 10.10.10.139 >>>>> >>>>> >>>>> >>>>> Demo-NetworkA.evpn.0: 13 destinations, 13 routes (13 active, 0 >>>>> holddown, 0 hidden) >>>>> >>>>> Prefix Nexthop MED Lclpref AS >>>>> path >>>>> >>>>> 2:10.10.20.70:2::4::f8:c0:01:19:b1:88/304 >>>>> >>>>> * Self 100 I >>>>> >>>>> 2:10.10.20.70:2::4::f8:c0:01:19:b1:88::10.10.10.145/304 >>>>> >>>>> * Self 100 I >>>>> >>>>> 2:10.10.20.70:2::4::f8:c0:01:19:b1:88::10.10.10.146/304 >>>>> >>>>> * Self 100 I >>>>> >>>>> 3:10.10.20.70:2::4::10.10.20.70/304 >>>>> >>>>> * Self 100 I >>>>> >>>>> >>>>> >>>>> __default_evpn__.evpn.0: 1 destinations, 1 routes (1 active, 0 >>>>> holddown, 0 hidden) >>>>> >>>>> Prefix Nexthop MED Lclpref AS >>>>> path >>>>> >>>>> 1:10.10.20.70:0::050000ff840000000400::FFFF:FFFF/304 >>>>> >>>>> * Self 100 I >>>>> >>>>> >>>>> >>>>> As you can see in the advertised BGP routes above the MX is announcing >>>>> .145 with the actual MAC of the IRB: >>>>> >>>>> >>>>> >>>>> root@gw1z0> show interfaces irb | grep hardware >>>>> >>>>> Current address: f8:c0:01:19:b1:88, Hardware address: >>>>> f8:c0:01:19:b1:88 >>>>> >>>>> >>>>> >>>>> not the virtual MAC that has been assigned for VRRP: >>>>> >>>>> >>>>> >>>>> root@gw1z0> show vrrp extensive | grep mac >>>>> >>>>> Virtual Mac: 00:00:5e:00:01:0d >>>>> >>>>> >>>>> >>>>> I suspect that because the MX never announces an EVPN mac of >>>>> 00:00:5e:00:01:0d, my TOR never learns where that MAC lives: >>>>> >>>>> >>>>> >>>>> oot@leaf1z0> show ovsdb mac >>>>> >>>>> Logical Switch Name: Contrail-12905c4b-60db-4d71-8fab-972051db1386 >>>>> >>>>> Mac IP Encapsulation Vtep >>>>> >>>>> Address Address Address >>>>> >>>>> ff:ff:ff:ff:ff:ff 0.0.0.0 Vxlan over Ipv4 >>>>> 10.10.20.65 >>>>> >>>>> d4:ae:52:c6:c9:e1 0.0.0.0 Vxlan over Ipv4 >>>>> 10.10.20.65 >>>>> >>>>> 50:c5:8d:d4:af:f0 0.0.0.0 Vxlan over Ipv4 >>>>> 10.10.20.71 >>>>> >>>>> 54:9f:35:05:fa:2e 0.0.0.0 Vxlan over Ipv4 >>>>> 10.10.20.66 >>>>> >>>>> f8:c0:01:19:b1:88 0.0.0.0 Vxlan over Ipv4 >>>>> 10.10.20.70 >>>>> >>>>> ff:ff:ff:ff:ff:ff 0.0.0.0 Vxlan over Ipv4 >>>>> 10.10.10.139 >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Is there perhaps a knob I'm missing? Is there something other then >>>>> VRRP that is preferred for redundant gateways in Contrail? >>>>> >>>>> >>>>> >>>>> I have the same question out to a few other resources but figured >>>>> someone on the mailing list might have some good pointers as well. >>>>> >>>>> >>>>> >>>>> Thanks! >>>>> >>>>> Dan >>>>> >>>> >>>> _______________________________________________ >>>> Users mailing list >>>> [email protected] >>>> >>>> http://lists.opencontrail.org/mailman/listinfo/users_lists.opencontrail.org >>>> >>>> >>>> >>> > >
_______________________________________________ Users mailing list [email protected] http://lists.opencontrail.org/mailman/listinfo/users_lists.opencontrail.org
