Re: 1 out of 2 BGP sessions down, all routes become unreachable

[Dez C]

Hi,

all external routes were unreachable (apart from the local loopback 
which was imported via direct protocol). master6 consisted solely of 
entries like:

2a02:xxxx:d00d::/45 unreachable [bgp_v6_a 19:38:23.222 from 
2a02:xxxx:yyyy:53::a] * (100) [ASxxxxx1i]

No entries whatsoever from the other bgp session (bgp_v6_b), like in 
normal operation:

2a02:xxxx:d00d::/45   unicast [bgp_v6_a 2021-04-07] * (100) [ASxxxxx1i]
        via 2a02:xxxx:yyyy:53::a on eth2
                     unicast [bgp_v6_b 2021-04-07] (100) [ASxxxxx1i]
        via 2a02:xxxx:yyyy:53::b on eth2

 

regards,
Yannis


On 4/9/21 11:22 PM, Alexander Zubkov wrote:
Hi,

And what do you see in your bird's console? "show route" etc.

On Fri, Apr 9, 2021 at 7:10 PM Dez C <d...@otenet.gr> wrote:
Hello,

First message here, although a long-time bird user (not an expert in any
way). We're using bird as a bgp speaker for all our VMs that either need
to announce routes or import routes from VRFs etc.

The setup is pretty straight forward I'd guess. The VM has 2 IPv6 BGP
sessions with 2 L3 switches (one per switch). The VM announces one /128
route and imports all routes learned from the BGP sessions. The same
routes are learned from both sessions, so whenever I do a 'ip -6 r' I
see 2 entries for each route (with different next hops) like:
2a02:xxxx:d00d::/45 via 2a02:xxxx:yyyy:zz::a dev eth2 proto bird metric
32 pref medium
2a02:xxxx:d00d::/45 via 2a02:xxxx:yyyy:zz::b dev eth2 proto bird metric
32 pref medium

At some point, both BGP sessions went down and one of them did not
recover for an hour. Normally, this should not be a problem but in this
specific case, I could no longer see 2 routes in the kernel table, one
reachable and another unreachable, just an unreachable one, so even
though the anycast route was being announced and traffic was reaching
the VM, it couldn't send the packets to their destination as all routes
were unreachable in the kernel table:
unreachable 2a02:xxxx:d00d::/45 dev lo proto bird metric 32 error -113
pref medium .

I'm running 2.0.7 (upgrading soon) on centos7 and here's the relevant
part of my config:

protocol device {
}
protocol direct direct_v6_lo {
          interface "lo*";  # announce loopback
          ipv6;
}
protocol direct direct_v6_eth {
          interface "eth*";
          ipv6;
}
protocol bgp bgp_v6_a {
          local 2a02:xxxx:yyyy:53::37 as 66666
          neighbor 2a02:xxxx:yyyy:53::a as 66666;  # switch A
          ipv6 {
                 export where proto = "direct_v6_lo";
          };
}
protocol bgp bgp_v6_b {
          local 2a02:1388:4000:53::37 as 66666;
          neighbor 2a02:1388:4000:53::b as 66666;  # switch B
          ipv6 {
                  export where proto = "direct_v6_lo";
          };
}
protocol kernel {
      ipv6 {
          export filter {
              if proto = "direct_v6_eth" then reject;
              accept;
          };
      };
     merge paths limit 2;
}

any ideas?
thanks in advance and regards,
Yannis