Hello.
We found the following problem: in case of unavailability (problem with the
communication channel) between routers (on which OpenBSD 6.6 is installed),
after the BGP hold timer expires, the bgp neighborhood drops and then the bgpd
daemon stops on all routers. The situation also repeats when two other routers
that are located on the same site are unavailable.
If I execute the command to check the BGP neighborship - I get a response that
the bgpd daemon is not running:
bgpctl show summary
bgpctl: connect: /var/run/bgpd.sock.0: Connection refused
After i start the bgpd daemon the neighborhood rises: rcctl start bgpd.
We have the following topology. Routers are indicated on the topology as
1,2,3,4,5,6ю On all routers the OpenBSD 6.6 operating system is running.
1---------------3
| \ /|
| 2-----------4 |
| \ / |
| ----6---- |
| | |
+-------5------+
Routers are connected by GRE interfaces: 1 and 3, 1 and 5, 3 and 5, 2 and 4, 2
and 6, 4 and 6.
Routers are connected by vlan interfaces: 1 and 2, 3 and 4, 5 and 6.
Routers that are installed on the one site: 1 and 2, 3 and 4, 5 and 6.
On all routers the bird daemon was run with OSPF on the gre, vlan, and loopback
interfaces to organize the availability of loopback interfaces.
Also, the ldpd daemon was run on interfaces GRE and vlan to generate MPLS
labels.
Also, the bgpd daemon was run on the loopback interfaces for MPLS L3 VPN.
Routers 2, 4, 6 are configured as bgp route reflectors and organize one cluster
with the same cluster id.
In the logs, we observe the following messages (its logs from router 6):
Oct 24 13:53:02 router6 bgpd[22410]: peer closed imsg connection
Oct 24 13:53:02 router6 bgpd[22410]: main: Lost connection to RDE
Oct 24 13:53:02 router6 bgpd[68713]: peer closed imsg connection
Oct 24 13:53:02 router6 bgpd[22410]: kernel routing table 99 (VRF_99) decoupled
Oct 24 13:53:02 router6 bgpd[68713]: SE: Lost connection to RDE
Oct 24 13:53:02 router6 bgpd[22410]: kernel routing table 0 (Loc-RIB) decoupled
Oct 24 13:53:02 router6 bgpd[68713]: peer closed imsg connection
Oct 24 13:53:02 router6 bgpd[68713]: SE: Lost connection to RDE control
Oct 24 13:53:02 router6 bgpd[47521]: Rib Loc-RIB: neighbor 10.0.0.6 (LOCAL)
AS64500: withdraw announce 192.168.51.0/24
Oct 24 13:53:02 router6 bgpd[68713]: peer closed imsg connection
Oct 24 13:53:02 router6 bgpd[68713]: SE: Lost connection to parent
Oct 24 13:53:02 router6 bgpd[47521]: Rib Loc-RIB: neighbor 10.0.0.6 (LOCAL)
AS64500: withdraw announce 172.16.17.0/24
Oct 24 13:53:02 router6 bgpd[68713]: neighbor 10.0.0.2 : sending notification:
Cease, administratively down
Oct 24 13:53:02 router6 bgpd[22410]: route decision engine terminated; signal 10
Oct 24 13:53:02 router6 bgpd[68713]: neighbor 10.0.0.2 : state change
Established -> Idle, reason: Stop
Oct 24 13:53:02 router6 bgpd[68713]: neighbor 10.0.0.1 : sending notification:
Cease, administratively down
Oct 24 13:53:02 router6 bgpd[68713]: neighbor 10.0.0.1 : state change
Established -> Idle, reason: Stop
Oct 24 13:53:02 router6 bgpd[68713]: neighbor 10.0.0.3 : sending notification:
Cease, administratively down
Oct 24 13:53:02 router6 bgpd[68713]: neighbor 10.0.0.3 : state change
Established -> Idle, reason: Stop
Oct 24 13:53:02 router6 bgpd[68713]: neighbor 10.0.0.5 : sending notification:
Cease, administratively down
Oct 24 13:53:02 router6 bgpd[68713]: neighbor 10.0.0.5 : state change
Established -> Idle, reason: Stop
Oct 24 13:53:02 router6 bgpd[68713]: session engine exiting
Oct 24 13:53:02 router6 bgpd[22410]: terminating
Example bgp configuration of router 1. The configuration of BGP routers 3 and 5
is similar to the configuration of router 1.
AS 64500
router-id 10.0.0.1
holdtime 60
connect-retry 1
log updates
listen on 10.0.0.1
vpn "VRF_99" on mpe99 {
rd 64500:99
export-target rt 64500:99
import-target rt 64500:99
network inet priority 25 set metric 30
}
group "RR-servers" {
remote-as 64500
local-address 10.0.0.1
announce IPv4 unicast
announce IPv4 vpn
neighbor 10.0.0.2 {
}
neighbor 10.0.0.4 {
}
neighbor 10.0.0.6 {
}
}
allow from any
allow to any
Example bgp configuration of router 2. The configuration of BGP routers 4 and 6
is similar to the configuration of router 2.
AS 64500
router-id 10.0.0.2
holdtime 60
connect-retry 1
log updates
listen on 10.0.0.2
vpn "VRF_99" on mpe99 {
rd 64500:99
export-target rt 64500:99
import-target rt 64500:99
network inet priority 25 set metric 50
}
group "RR-servers" {
remote-as 64500
local-address 10.0.0.2
announce IPv4 unicast
announce IPv4 vpn
neighbor 10.0.0.4 {
}
neighbor 10.0.0.6 {
}
}
group "RR-clients" {
remote-as 64500
local-address 10.0.0.2
announce IPv4 unicast
announce IPv4 vpn
route-reflector 10.0.0.0
neighbor 10.0.0.1 {
}
neighbor 10.0.0.3 {
}
neighbor 10.0.0.5 {
}
}
allow from any
allow to any
On the previous version of OpenBSD 6.5, there was no such problem.
Can you tell me how to fix the problem?
Email me if you need more information.
