Re: post-6.5-upgrade bgpd(8) problem
On Thu, May 09, 2019 at 10:58:54AM -0500, Adam Thompson wrote:
> I've upgraded my looking glass from 6.4 to 6.5, and an experiencing an
> unexpected problem - routes learned from one (iBGP) peer are not being
> automatically exported to other (eBGP) peers.
>
> I did not change /etc/bgpd.conf, but behaviour seems to have changed
> nonetheless. The upgrade from 6.4 to 6.5 appeared smooth otherwise, nothing
> to suggest subtle breakage under the hood.
>
> As you can see below, this bgpd.conf is almost so simple it *can't* have
> problems. Apparently "almost" being the operative word.
>
> Under 6.4, this behaved as though "export none" only applied to the first
> group. Under 6.5, it behaves as though "export none" is a global setting.
>
> Under 6.5, bgpctl show produces:
> root@bgpmirror:~# bgpctl sh
> Neighbor ASMsgRcvdMsgSent OutQ Up/Down
> State/PrfRcvd
> Hermes IPv4 16796 128773 85 0 00:41:40
> 753748
> Hermes IPv6 16796 29727 85 0 00:41:40
> 68228
> MBNOG IPv4 65204 97 85 0 00:41:40
> 0
> MBNOG IPv6 65204 97 85 0 00:41:40
> 0
> BGPMon.io IPv4 6447 0 21 0 Never
> Active
> isolario.it IPv465517 86 85 0 00:41:39
> 0
> isolario.it IPv665517 86 85 0 00:41:39
> 0
> and the operator of the MBNOG route collector confirms that I stopped
> sending him a full routing table at the same time I did the OS upgrade.
>
> Any ideas? What other information would help diagnose this problem?
>
> Thanks,
> -Adam
>
>
>
> Dmesg & bgpd.conf:
> https://gist.github.com/athompso/e334d8621ce458925e25bb44b8068341
>
>
> bgpd.conf, duplicated here for convenience:
>
> ===BOF===
> route-collector yes
You have route-collector turned on and so you disable the decision process
and so no prefix will be selected and sent out. This is the way it is
supposed to work. Your setup is not a route-collector.
In 6.4 route-collector mode was broken (as in you could not turn it on)
and I fixed this. That is why your noticed the behaviour change.
> socket "/var/www/run/bgpd.rsock" restricted # for bgplg
>
> # settings
> nexthop qualify via default
> fib-update no
> dump table-v2 "/var/www/htdocs/bgplg/mrt/rib-dump.mrt" 3600
> dump updates in "/var/www/htdocs/bgplg/mrt/updates-in-%H%M" 300
> dump all in "/var/www/htdocs/bgplg/mrt/all-in-%H%M" 300
>
> # myself
> AS X
> router-id X.X.X.X
>
> # neighbors
>
> group hermes {
> enforce local-as no
> enforce neighbor-as no
> export none
>
> neighbor X.X.X.X {
> remote-as X
> descr "Hermes IPv4"
> }
> neighbor X:X:X:X::X {
> remote-as X
> descr "Hermes IPv6"
> }
> }
>
> group bgpresearch {
> multihop 32
> enforce local-as no
> enforce neighbor-as no
>
> neighbor 192.160.102.196 {
> remote-as 65204
> descr "MBNOG IPv4"
> }
> neighbor 2620:132:3003:300::196 {
> remote-as 65204
> descr "MBNOG IPv6"
> }
> neighbor 129.82.138.6 {
> remote-as 6447
> descr "BGPMon.io IPv4"
> }
> neighbor 146.48.78.12 {
> remote-as 65517
> descr "isolario.it IPv4"
> }
> neighbor 2a00:1620:c0:4e:146:48:78:12 {
> remote-as 65517
> descr "isolario.it IPv6"
> }
> }
>
> # policies
> allow quick from group hermes
> allow quick to group bgpresearch
> ===EOF===
>
> (if you want to see the unredacted version of bgpd.conf, ask and I'll email
> it directly to you, I just don't want internal addresses in the public
> archive.)
>
--
:wq Claudio
Re: post-6.5-upgrade bgpd(8) problem
On 2019-05-09 13:53, Sebastian Benoit wrote: bgpctl sh rib neigh out for all neighbors. All empty. Also look at bgpctl sh rib best Completely empty. if any routes are actually selected - maybe the "nexthop qualify via default" isnt working. I see two things... 1) when run as "bgpd -dv" I get repeated notifications about the same next-hops, dunno if that's normal or not. And 2) from bgpd.conf(5): route-collector (yes|no) If set to yes, the route selection process is turned off. The default is no. and I have it set to yes, since this is supposed to behave like a route collector. Granted, with a single source of routes at the moment, I could turn that off... let's see what happens when I do: Yup. That behaviour has definitely changed, somehow - taking a not-so-wild guess, it's likely _somehow_ related to claudio@'s introduction of Adj-Rib-Out but I'm not enough of a C programmer (or kernel routing expert) to say for sure. For now, commenting out the "route-collector yes" line in bgpd.conf will work, but there's a (for me) minor regression. I have no idea which way the code is supposed to behave, so can't tell if this is a bug or a bugfix! Based on CVS logs, it's probably a change introduced in rde.c after v1.442. -Adam
Re: post-6.5-upgrade bgpd(8) problem
Hi,
Adam Thompson(athom...@athompso.net) on 2019.05.09 10:58:54 -0500:
> I've upgraded my looking glass from 6.4 to 6.5, and an experiencing an
> unexpected problem - routes learned from one (iBGP) peer are not being
> automatically exported to other (eBGP) peers.
>
> I did not change /etc/bgpd.conf, but behaviour seems to have changed
> nonetheless. The upgrade from 6.4 to 6.5 appeared smooth otherwise,
> nothing to suggest subtle breakage under the hood.
>
> As you can see below, this bgpd.conf is almost so simple it *can't* have
> problems. Apparently "almost" being the operative word.
>
> Under 6.4, this behaved as though "export none" only applied to the
> first group. Under 6.5, it behaves as though "export none" is a global
> setting.
>
> Under 6.5, bgpctl show produces:
> root@bgpmirror:~# bgpctl sh
> Neighbor ASMsgRcvdMsgSent OutQ
> Up/Down State/PrfRcvd
> Hermes IPv4 16796 128773 85 0
> 00:41:40 753748
> Hermes IPv6 16796 29727 85 0
> 00:41:40 68228
> MBNOG IPv4 65204 97 85 0
> 00:41:40 0
> MBNOG IPv6 65204 97 85 0
> 00:41:40 0
> BGPMon.io IPv4 6447 0 21 0 Never
> Active
> isolario.it IPv465517 86 85 0
> 00:41:39 0
> isolario.it IPv665517 86 85 0
> 00:41:39 0
> and the operator of the MBNOG route collector confirms that I stopped
> sending him a full routing table at the same time I did the OS upgrade.
>
> Any ideas? What other information would help diagnose this problem?
>
> Thanks,
> -Adam
>
>
>
> Dmesg & bgpd.conf:
> https://gist.github.com/athompso/e334d8621ce458925e25bb44b8068341
>
>
> bgpd.conf, duplicated here for convenience:
>
> ===BOF===
> route-collector yes
> socket "/var/www/run/bgpd.rsock" restricted # for bgplg
>
> # settings
> nexthop qualify via default
> fib-update no
> dump table-v2 "/var/www/htdocs/bgplg/mrt/rib-dump.mrt" 3600
> dump updates in "/var/www/htdocs/bgplg/mrt/updates-in-%H%M" 300
> dump all in "/var/www/htdocs/bgplg/mrt/all-in-%H%M" 300
>
> # myself
> AS X
> router-id X.X.X.X
>
> # neighbors
>
> group hermes {
> enforce local-as no
> enforce neighbor-as no
> export none
>
> neighbor X.X.X.X {
> remote-as X
> descr "Hermes IPv4"
> }
> neighbor X:X:X:X::X {
> remote-as X
> descr "Hermes IPv6"
> }
> }
>
> group bgpresearch {
> multihop 32
> enforce local-as no
> enforce neighbor-as no
>
> neighbor 192.160.102.196 {
> remote-as 65204
> descr "MBNOG IPv4"
> }
> neighbor 2620:132:3003:300::196 {
> remote-as 65204
> descr "MBNOG IPv6"
> }
> neighbor 129.82.138.6 {
> remote-as 6447
> descr "BGPMon.io IPv4"
> }
> neighbor 146.48.78.12 {
> remote-as 65517
> descr "isolario.it IPv4"
> }
> neighbor 2a00:1620:c0:4e:146:48:78:12 {
> remote-as 65517
> descr "isolario.it IPv6"
> }
> }
>
> # policies
> allow quick from group hermes
> allow quick to group bgpresearch
> ===EOF===
Please check
bgpctl sh rib neigh out
for all neighbors.
Also look at
bgpctl sh rib best
if any routes are actually selected - maybe the "nexthop qualify via
default" isnt working.
/Benno
post-6.5-upgrade bgpd(8) problem
I've upgraded my looking glass from 6.4 to 6.5, and an experiencing an
unexpected problem - routes learned from one (iBGP) peer are not being
automatically exported to other (eBGP) peers.
I did not change /etc/bgpd.conf, but behaviour seems to have changed
nonetheless. The upgrade from 6.4 to 6.5 appeared smooth otherwise,
nothing to suggest subtle breakage under the hood.
As you can see below, this bgpd.conf is almost so simple it *can't* have
problems. Apparently "almost" being the operative word.
Under 6.4, this behaved as though "export none" only applied to the
first group. Under 6.5, it behaves as though "export none" is a global
setting.
Under 6.5, bgpctl show produces:
root@bgpmirror:~# bgpctl sh
Neighbor ASMsgRcvdMsgSent OutQ
Up/Down State/PrfRcvd
Hermes IPv4 16796 128773 85 0
00:41:40 753748
Hermes IPv6 16796 29727 85 0
00:41:40 68228
MBNOG IPv4 65204 97 85 0
00:41:40 0
MBNOG IPv6 65204 97 85 0
00:41:40 0
BGPMon.io IPv4 6447 0 21 0 Never
Active
isolario.it IPv465517 86 85 0
00:41:39 0
isolario.it IPv665517 86 85 0
00:41:39 0
and the operator of the MBNOG route collector confirms that I stopped
sending him a full routing table at the same time I did the OS upgrade.
Any ideas? What other information would help diagnose this problem?
Thanks,
-Adam
Dmesg & bgpd.conf:
https://gist.github.com/athompso/e334d8621ce458925e25bb44b8068341
bgpd.conf, duplicated here for convenience:
===BOF===
route-collector yes
socket "/var/www/run/bgpd.rsock" restricted # for bgplg
# settings
nexthop qualify via default
fib-update no
dump table-v2 "/var/www/htdocs/bgplg/mrt/rib-dump.mrt" 3600
dump updates in "/var/www/htdocs/bgplg/mrt/updates-in-%H%M" 300
dump all in "/var/www/htdocs/bgplg/mrt/all-in-%H%M" 300
# myself
AS X
router-id X.X.X.X
# neighbors
group hermes {
enforce local-as no
enforce neighbor-as no
export none
neighbor X.X.X.X {
remote-as X
descr "Hermes IPv4"
}
neighbor X:X:X:X::X {
remote-as X
descr "Hermes IPv6"
}
}
group bgpresearch {
multihop 32
enforce local-as no
enforce neighbor-as no
neighbor 192.160.102.196 {
remote-as 65204
descr "MBNOG IPv4"
}
neighbor 2620:132:3003:300::196 {
remote-as 65204
descr "MBNOG IPv6"
}
neighbor 129.82.138.6 {
remote-as 6447
descr "BGPMon.io IPv4"
}
neighbor 146.48.78.12 {
remote-as 65517
descr "isolario.it IPv4"
}
neighbor 2a00:1620:c0:4e:146:48:78:12 {
remote-as 65517
descr "isolario.it IPv6"
}
}
# policies
allow quick from group hermes
allow quick to group bgpresearch
===EOF===
(if you want to see the unredacted version of bgpd.conf, ask and I'll
email it directly to you, I just don't want internal addresses in the
public archive.)
