Hi Henning,
Hope the hackathon went well.
Just wanted to update you to let you know that we have asked a C
developer to help us with this and he is going to look into coding this
enhancement for us. We would like to do it so that it meets your coding
standards and can make it into the OpenBSD code base.
It would be great if you could provide any comments at all on what
would be the best way to add this enhancement if you were doing it
yourself? :)
bgpd.conf snippet;
group "ISP1 peering AS1111" {
remote-as 1111
announce self
neighbor 170.16.3.1 {
descr "ISP1-RT"
}
}
match to 170.16.3.1 set nexthop 170.16.3.4
Setup overview;
OpenBSD1;
vlan1: 170.16.3.2
carp1: 170.16.3.4
OpenBSD1;
vlan1: 170.16.3.3
carp1: 170.16.3.4
Cisco ISP1-RT;
170.16.3.1
Summary;
When the OpenBSD box is a CARP backup there is *not* a route for
170.16.3.4/32 in 'netstat -rn'
Destination Gateway Flags Refs Use Mtu Prio
Iface
170.16.3/24 170.16.3.1 UG 0 0 - 48
vlan1
Running 'bgpd -d' we see the logging;
nexthop 170.16.3.4 now valid: directly connected
Because 170.16.3.4 is within the subnet 170.16.3/24 ;)
And the Cisco learns our announcement with the nexthop correctly set to
170.16.3.4 (our CARP IP :)
When the 'other' OpenBSD box is a CARP master there *is* a route for
170.16.3.4/32 in 'netstat -rn'
Destination Gateway Flags Refs Use Mtu Prio
Iface
170.16.3/24 170.16.3.1 UG 0 0 - 48
vlan1
170.16.3.4 170.16.3.4 UH 0 0 - 4
carp1
Running 'bgpd -d' we see the logging;
nexthop 170.16.3.4 now valid: via 170.16.3.4
According to the rules of eBGP peers, the nexthop *must* be in the
directly attached layer 2 network connecting the OpenBSD firewall and
the Cisco Router. It cannot be 'via' a hop.
Thus due to the presence of the route for the CARP IP (which states the
desired nexthop is on the carp1 and not the vlan1 interface), the
nexthop validation check interprets this is a hop and so rejects the
declaration;
match to 170.16.3.1 set nexthop 170.16.3.4
And the Cisco does *not* learn our announcement with the nexthop set to
170.16.3.4 as desired, but instead the nexthop is set to the IP on
vlan1 :(
Omar mentioned that after only a very quick look he could see in the
code that OpenBGP reads the routing table (kroute) and it also records
the interface for the BGP peer and the interface for the desired
nexthop, and see's them as being different.
We want to ask;
1) Would it be appropriate to change the code so that the nexthop
validation check instead checks if the desired nexthop is in the same
layer 3 subnet/domain as the BGP peer?
2) Or would to change the code so that the nexthop validation check
instead checks that, if the found nexthop interface is a CARP
interface, that it should then check if that CARP interface is a
member/child of the physical interface connected to the BGP peer?
My feeling is that the first option would be easier to code and more
robust, instead of having to go off and poll for CARP and PHY interface
associations?
Thanks for your time. We will get a solution eventually :)
This of course all means that a pair of OpenBSD firewalls in an
active-backup CARP pair configuration can be used as both statefull
firewalls and stateless BGP routers :)
I have attached an image to show the idea using two upstream providers.
Cheers, Andy Lemin.
On Wed 22 Jan 2014 15:14:55 GMT, Andy wrote:
> Hi, Cool!
>
> When you have some time I'll shoot something over which is maybe a bit
> more clear.
> I agree that it is not a bug as the nexthop validation is working and
> correctly detects the nexthop I'm trying to set is on a different
> interface (due to the /32 carp route) to the BGP peer, the problem is
> that the nexthop IP is on the carp interface on-top of the same
> interface to which the BGP peer is connected.. Guess it would be an
> enhancement to accommodate validating the carp interface ;)
>
> Have fun at the hackathon :)
>
> PS; Enjoyed your new queuing subsystem talk at EuroBSDcon.
>
> Thanks again, Andy.
>
>
> -------- Original Message --------
> Subject: Re: BGP changes to support CARP better
> Date: Wed, 22 Jan 2014 01:39:47 +0100
> From: Henning Brauer <[email protected]>
> To: Andy <[email protected]>
>
>
>
> I'm at a hackathon and can't analyse in detail, but from glancing over
> I don't see no bug.
> feel free to poke me later, 2nd week of feb is realisticly the
> earliest i might have some spare cycles again. use my @openbsd addrss
> tho, as you noticed i'm FAR behind on misc etc (and this address ends
> up in the misc folder)
>
> * Andy<[email protected]> [2014-01-21 11:32]:
> > Hi Henning,
> >
> > Thank you for your reply. I have poured through the manpage and I am not
> > sure what you are referring to? What have I missed?
> >
> > I'm really sorry if I'm being stupid here, I am still learning and BGP is a
> > steep learning curve.. I am reading and studying lots of Cisco sources at
> > the moment to improve my general BGP knowledge, and as ever when I learn
> > something I try to help others in the community as best I can to echo the
> > knowledge. In this case however I am really stuck :(
> >
> > I have the 'set nexthop' working, but only on the CARP backup.
> >
> > 'set nexthop' doesn't work on the carp master as the nexthop validation
> > fails, because on the master a /32 route for the CARP IP (which I am trying
> > to set the nexthop too) exists pointing to the CARP interface (not the
> > physical interface, thus reports 'via' instead of 'directly connected').
> >
> > I have read the manpage closely regarding the nexthop validation, and tried
> > the 'nexthop qualify' options.
> >
> > This really seems like a bug?
> >
> > Thank you for your time, I do greatly appreciate your work on the queuing
> > subsystem and OpenBGPd (and I am sure their are many other great things you
> > have done that I am not aware of).
> >
> > Humbly needing help, Andy.
> >
> >
> >
> > On Mon 20 Jan 2014 09:02:43 GMT, Henning Brauer wrote:
> > >andy,
> > >
> > >you really need to improve your manpage reading. almost everything
> > >you're asking is in the manpages. I dunno how to put it otherwise.
> > >the manages do not explain concepts, that is not their purpose and
> > >completely out of scope.
> > >
> > >you might need some consulting for a kickstart.
> > >
> > >and fwiw, bugs go to, well, bugs@
> > >
> > >ciao
> > >
> > >henning
> > >(obviously far behind on misc@)
> > >
> > >* Andy<[email protected]> [2013-12-07 21:20]:
> > >>Hi guys, what is the best way to raise a bug report for OpenBGPd on
> > >>OpenBSD?
> > >>
> > >>I know this may take a while to get fixed and I do humbly appreciative the
> > >>great work you guys do. I hope I don't seem pushy, I would have a go at
> > >>writing a code diff myself but I'm well out of my depth to try.
> > >>
> > >>If you have to run traffic through only the CARP master with PF running,
> > >>as
> > >>you cannot run PF with 'sloppy states', when a CARP fail-over happens it
> > >>takes a very long time to converge as it requires the whole BGP table to
> > >>be
> > >>transfered and to reflect the new IP of the secondary firewall (using
> > >>'depend on carpX').
> > >>
> > >>If you could just set the CARP IP as the eBGP nexthop the fail-over time
> > >>would be instant.
> > >>
> > >>As you may know the 'set nexthop' bgpd.conf directive doesn't work on the
> > >>CARP master host;
> > >>E.g. 'match to 170.16.3.1 set nexthop 170.16.3.4' where 170.16.3.4 is the
> > >>CARP IP and 170.16.3.1 is the ISP eBGP neighbor.
> > >>
> > >>
> > >>After some more testing, trying different workarounds and examining the
> > >>routing tables, the reason why the eBGP 'nexthop' is not being correctly
> > >>set
> > >>to the CARP IP address is because the nexthop validation check is not
> > >>taking
> > >>into account that the carp interface is in-fact the same as the physical
> > >>interface.
> > >>
> > >>Only when a host is the CARP master does the second route as shown below
> > >>exist (with Iface carp901);
> > >>Destination Gateway Flags Refs Use Mtu Prio Iface
> > >>170.16.3/24 link#12 UC 2 0 - 4 vlan901
> > >>170.16.3.4 170.16.3.4 UH 0 0 - 4 carp901
> > >>
> > >>This causes the nexthop validation check to result in;
> > >>nexthop 170.16.3.4 now valid: via 170.16.3.4
> > >>instead of;
> > >>nexthop 170.16.3.4 now valid: directly connected
> > >>
> > >>
> > >>As per the bgpd.conf(5) man page, by default bgpd will use the static
> > >>routes
> > >>to verify the nexthop validity before applying it. So I have tried setting
> > >>'nexthop qualify via bgp' with 'network 170.16.3.4/32', and 'nexthop
> > >>qualify
> > >>via default' but without success.
> > >>
> > >>Thanks for your time and for reading this.
> > >>When you guys are next looking at the OpenBGPd code I would be greatly
> > >>indebted to you if you could look at adding the code to ignore /32 static
> > >>routes on carp interfaces.
> > >>
> > >>Kind regards, Andy.
> > >>
> > >>
> > >>On Thu 05 Dec 2013 17:30:14 GMT, Andy wrote:
> > >>>Hi,
> > >>>
> > >>>Does anyone have an idea how I can get this nexthop qualify to work?
> > >>>
> > >>>from the man page it says;
> > >>> nexthop qualify via (bgp|default)
> > >>> If set to bgp, bgpd(8) may use BGP routes to verify
> > >>>nexthops. If
> > >>> set to default, bgpd may use the default route to verify
> > >>> nexthops. By default bgpd will only use static routes or
> > >>>routes
> > >>> added by other routing daemons like ospfd(8).
> > >>>
> > >>>I've tried various things but nothing works..
> > >>>
> > >>>The carp IP is on the 'carp' interface and not the phys interface and
> > >>>so I think thats why the nexthop is not being applied correctly on the
> > >>>master?
> > >>>
> > >>>I've tried 'multihop' too.
> > >>>
> > >>>Thanks, Andy.
> > >>>
> > >>>
> > >>>On Tue 03 Dec 2013 17:06:40 GMT, Andy wrote:
> > >>>>Hi, I've got something really interesting to show, which shows this
> > >>>>clearly and should help point to the root cause.
> > >>>>
> > >>>>In short, it seems that the desired nexthop is not applied by the CARP
> > >>>>master when it is in state 'nexthop 180.25.32.20 now valid: via
> > >>>>180.25.32.20'. I.e. when it is 'via' even though it is a local IP..
> > >>>>
> > >>>>From the perspective of the 'backup' the CARP IP is a directly
> > >>>>connected IP which it can reach 'nexthop 180.25.32.20 now valid:
> > >>>>directly connected'.
> > >>>>
> > >>>>NB; I haven't had a chance to test IPv6 or iBGP but from this
> > >>>>observation it looks like the same problem will be seen, unless there
> > >>>>is a way of telling OpenBGPd to use nexthops which are 'via' something..
> > >>>>
> > >>>>
> > >>>>THE SETUP;
> > >>>>
> > >>>>- Two OpenBSD boxes with CARP on their BGP and LAN Interfaces.
> > >>>>- One or two upstream Cisco routers on BGP interface via switch (both
> > >>>>show same problem).
> > >>>>- PF disabled (just for this testing).
> > >>>>- 180.25.32.1 = iBGP Cisco Router
> > >>>>- 180.25.32.20 = CARP IP
> > >>>>- 180.25.32.21 = OBSD1
> > >>>>- 180.25.32.22 = OBSD2
> > >>>>- Neighbors are eBGP
> > >>>>
> > >>>>OpenBSD Host 1 (master) /etc/bgpd.conf;
> > >>>>AS 66868
> > >>>>router-id 180.25.32.21
> > >>>>log updates
> > >>>>network 180.25.32.0/22
> > >>>>network 2a00:7ee0::/32
> > >>>>neighbor 180.25.32.1 {
> > >>>> remote-as 66868
> > >>>> announce self
> > >>>> local-address 180.25.32.21
> > >>>> tcp md5sig password secret
> > >>>> descr "THN"
> > >>>>}
> > >>>>match to 180.25.32.1 set nexthop 180.25.32.20
> > >>>>allow from any inet prefixlen 8 - 26
> > >>>>allow from any inet6 prefixlen 16 - 48
> > >>>>allow to any
> > >>>>
> > >>>>
> > >>>>OpenBSD Host 1 (backup) /etc/bgpd.conf;
> > >>>>AS 66868
> > >>>>router-id 180.25.32.22
> > >>>>log updates
> > >>>>network 180.25.32.0/22
> > >>>>network 2a00:7ee0::/32
> > >>>>neighbor 180.25.32.1 {
> > >>>> remote-as 66868
> > >>>> announce self
> > >>>> local-address 180.25.32.22
> > >>>> tcp md5sig password secret
> > >>>> descr "THN"
> > >>>>}
> > >>>>match to 180.25.32.1 set nexthop 180.25.32.20
> > >>>>allow from any inet prefixlen 8 - 26
> > >>>>allow from any inet6 prefixlen 16 - 48
> > >>>>allow to any
> > >>>>
> > >>>>
> > >>>>Cisco Host;
> > >>>>router bgp 12345
> > >>>>bgp router-id 180.25.32.1
> > >>>>bgp log-neighbor-changes
> > >>>>neighbor 180.25.32.21 remote-as 66868
> > >>>>neighbor 180.25.32.21 password secret
> > >>>>neighbor 180.25.32.22 remote-as 66868
> > >>>>neighbor 180.25.32.22 password secret
> > >>>>!
> > >>>>address-family ipv4
> > >>>>neighbor 180.25.32.21 activate
> > >>>>neighbor 180.25.32.22 activate
> > >>>>exit-address-family
> > >>>>!
> > >>>>!
> > >>>>
> > >>>>
> > >>>>
> > >>>>TEST 1 - Start BGP on master then backup;
> > >>>>
> > >>>>BGP Process is already running on the Cisco..
> > >>>>THN(config)#do show ip bgp
> > >>>>THN(config)#
> > >>>>
> > >>>>
> > >>>>OpenBSD Host 1 (MASTER) bgpd -dv;
> > >>>>[LIVE]root@OpenBSD1:~# bgpd -dv
> > >>>>startup
> > >>>>rereading config
> > >>>>route decision engine ready
> > >>>>session engine ready
> > >>>>new ktable rdomain_0 for rtableid 0
> > >>>>nexthop 180.25.32.20 now valid: via 180.25.32.20
> > >>>>listening on 0.0.0.0
> > >>>>listening on ::
> > >>>>SE reconfigured
> > >>>>neighbor 180.25.32.1 (THN): state change None -> Idle, reason: None
> > >>>>neighbor 180.25.32.1 (THN): state change Idle -> Connect, reason: Start
> > >>>>RDE reconfigured
> > >>>>neighbor 180.25.32.1 (THN): state change Connect -> OpenSent, reason:
> > >>>>Connection opened
> > >>>>neighbor 180.25.32.1 (THN): state change OpenSent -> OpenConfirm,
> > >>>>reason: OPEN message received
> > >>>>neighbor 180.25.32.1 (THN): state change OpenConfirm -> Established,
> > >>>>reason: KEEPALIVE message received
> > >>>>
> > >>>>
> > >>>>THN(config)#do show ip bgp
> > >>>>BGP table version is 8, local router ID is 180.25.32.1
> > >>>>Status codes: s suppressed, d damped, h history, * valid, > best, i -
> > >>>>internal,
> > >>>> r RIB-failure, S Stale, m multipath, b backup-path, f
> > >>>>RT-Filter,
> > >>>> x best-external, a additional-path, c RIB-compressed,
> > >>>>Origin codes: i - IGP, e - EGP, ? - incomplete
> > >>>>RPKI validation codes: V valid, I invalid, N Not found
> > >>>>
> > >>>> Network Next Hop Metric LocPrf Weight Path
> > >>>>*> 180.25.32.0/22 180.25.32.21 0 66868 i
> > >>>>
> > >>>>
> > >>>>NOTICE THIS IS THE WRONG NEXTHOP! :(
> > >>>>
> > >>>>
> > >>>>OpenBSD Host 2 (BACKUP) bgpd -dv;
> > >>>>[LIVE]root@OpenBSD2:~# bgpd -dv
> > >>>>startup
> > >>>>rereading config
> > >>>>route decision engine ready
> > >>>>session engine ready
> > >>>>new ktable rdomain_0 for rtableid 0
> > >>>>nexthop 180.25.32.20 now valid: directly connected
> > >>>>listening on 0.0.0.0
> > >>>>listening on ::
> > >>>>SE reconfigured
> > >>>>neighbor 180.25.32.1 (THN): state change None -> Idle, reason: None
> > >>>>neighbor 180.25.32.1 (THN): state change Idle -> Connect, reason: Start
> > >>>>RDE reconfigured
> > >>>>neighbor 180.25.32.1 (THN): state change Connect -> OpenSent, reason:
> > >>>>Connection opened
> > >>>>neighbor 180.25.32.1 (THN): state change OpenSent -> OpenConfirm,
> > >>>>reason: OPEN message received
> > >>>>neighbor 180.25.32.1 (THN): state change OpenConfirm -> Established,
> > >>>>reason: KEEPALIVE message received
> > >>>>Rib Loc-RIB: neighbor 180.25.32.1 (THN) AS12345: update 180.25.32.0/22
> > >>>>via 180.25.32.1
> > >>>>nexthop 180.25.32.1 now valid: directly connected
> > >>>>
> > >>>>(^ Why do these last two lines not show on the master?)
> > >>>>
> > >>>>
> > >>>>THN(config)#do show ip bgp
> > >>>>BGP table version is 8, local router ID is 180.25.32.1
> > >>>>Status codes: s suppressed, d damped, h history, * valid, > best, i -
> > >>>>internal,
> > >>>> r RIB-failure, S Stale, m multipath, b backup-path, f
> > >>>>RT-Filter,
> > >>>> x best-external, a additional-path, c RIB-compressed,
> > >>>>Origin codes: i - IGP, e - EGP, ? - incomplete
> > >>>>RPKI validation codes: V valid, I invalid, N Not found
> > >>>>
> > >>>> Network Next Hop Metric LocPrf Weight Path
> > >>>>* 180.25.32.0/22 180.25.32.20 0 66868 i
> > >>>>*> 180.25.32.21 0 66868 i
> > >>>>
> > >>>>
> > >>>>THE CORRECT NEXTHOP IS SHOWN (180.25.32.20) BUT IS NOT THE > BEST
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>TEST 2 - Shutdown OpenBGPd on both and restart OpenBGPd on just the
> > >>>>backup;
> > >>>>
> > >>>>OpenBSD Host 2 (BACKUP) bgpd -dv;
> > >>>>[LIVE]root@OpenBSD2:~# bgpd -dv
> > >>>>startup
> > >>>>rereading config
> > >>>>route decision engine ready
> > >>>>session engine ready
> > >>>>new ktable rdomain_0 for rtableid 0
> > >>>>nexthop 180.25.32.20 now valid: directly connected
> > >>>>listening on 0.0.0.0
> > >>>>listening on ::
> > >>>>SE reconfigured
> > >>>>neighbor 180.25.32.1 (THN): state change None -> Idle, reason: None
> > >>>>neighbor 180.25.32.1 (THN): state change Idle -> Connect, reason: Start
> > >>>>RDE reconfigured
> > >>>>neighbor 180.25.32.1 (THN): state change Connect -> OpenSent, reason:
> > >>>>Connection opened
> > >>>>neighbor 180.25.32.1 (THN): state change OpenSent -> OpenConfirm,
> > >>>>reason: OPEN message received
> > >>>>neighbor 180.25.32.1 (THN): state change OpenConfirm -> Established,
> > >>>>reason: KEEPALIVE message received
> > >>>>
> > >>>>
> > >>>>THN(config)#do show ip bgp
> > >>>>BGP table version is 14, local router ID is 180.25.32.1
> > >>>>Status codes: s suppressed, d damped, h history, * valid, > best, i -
> > >>>>internal,
> > >>>> r RIB-failure, S Stale, m multipath, b backup-path, f
> > >>>>RT-Filter,
> > >>>> x best-external, a additional-path, c RIB-compressed,
> > >>>>Origin codes: i - IGP, e - EGP, ? - incomplete
> > >>>>RPKI validation codes: V valid, I invalid, N Not found
> > >>>>
> > >>>> Network Next Hop Metric LocPrf Weight Path
> > >>>>*> 180.25.32.0/22 180.25.32.20 0 66868 i
> > >>>>
> > >>>>
> > >>>>THE CORRECT NEXTHOP IS STILL SHOWN AND OF COURSE IS NOW THE BEST AS
> > >>>>ITS THE ONLY ONE..
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>TEST 3 - Now lets start OpenBGPd on the master;
> > >>>>
> > >>>>[LIVE]root@OpenBSD1:~# bgpd -dv
> > >>>>startup
> > >>>>rereading config
> > >>>>route decision engine ready
> > >>>>session engine ready
> > >>>>new ktable rdomain_0 for rtableid 0
> > >>>>nexthop 180.25.32.20 now valid: via 180.25.32.20
> > >>>>listening on 0.0.0.0
> > >>>>listening on ::
> > >>>>SE reconfigured
> > >>>>neighbor 180.25.32.1 (THN): state change None -> Idle, reason: None
> > >>>>neighbor 180.25.32.1 (THN): state change Idle -> Connect, reason: Start
> > >>>>RDE reconfigured
> > >>>>neighbor 180.25.32.1 (THN): state change Connect -> OpenSent, reason:
> > >>>>Connection opened
> > >>>>neighbor 180.25.32.1 (THN): state change OpenSent -> OpenConfirm,
> > >>>>reason: OPEN message received
> > >>>>neighbor 180.25.32.1 (THN): state change OpenConfirm -> Established,
> > >>>>reason: KEEPALIVE message received
> > >>>>Rib Loc-RIB: neighbor 180.25.32.1 (THN) AS12345: update 180.25.32.0/22
> > >>>>via 180.25.32.1
> > >>>>nexthop 180.25.32.1 now valid: directly connected
> > >>>>
> > >>>>
> > >>>>THN(config)#do show ip bgp
> > >>>>BGP table version is 14, local router ID is 180.25.32.1
> > >>>>Status codes: s suppressed, d damped, h history, * valid, > best, i -
> > >>>>internal,
> > >>>> r RIB-failure, S Stale, m multipath, b backup-path, f
> > >>>>RT-Filter,
> > >>>> x best-external, a additional-path, c RIB-compressed,
> > >>>>Origin codes: i - IGP, e - EGP, ? - incomplete
> > >>>>RPKI validation codes: V valid, I invalid, N Not found
> > >>>>
> > >>>> Network Next Hop Metric LocPrf Weight Path
> > >>>>* 180.25.32.0/22 180.25.32.21 0 66868 i
> > >>>>*> 180.25.32.20 0 66868 i
> > >>>>
> > >>>>THE MASTER IS STILL SENDING A NEXTHOP OF ITS PHYSICAL INTERFACE AND
> > >>>>NOT THE CARP IP, SO THE STARTING ORDER DOESN'T MATTER AND THIS ISN'T
> > >>>>SOME ROUTE REFLECTION WIERDNESS
> > >>>>
> > >>>>
> > >>>>
> > >>>>TEST 4 - Now lets stop OpenBGPd on the master, switch the firewalls to
> > >>>>make the master the backup and restart OpenBGPd;
> > >>>>
> > >>>>
> > >>>>[LIVE]root@OpenBSD1:~# ifconfig -g carp carpdemote 10
> > >>>>
> > >>>>This following message appeared on the OpenBGPd debug on the backup as
> > >>>>I performed the carpdemote on the master;
> > >>>>nexthop 180.25.32.20 now valid: via 180.25.32.20
> > >>>>
> > >>>>
> > >>>>[LIVE]root@OpenBSD1:~# bgpd -dv
> > >>>>startup
> > >>>>rereading config
> > >>>>route decision engine ready
> > >>>>session engine ready
> > >>>>new ktable rdomain_0 for rtableid 0
> > >>>>nexthop 180.25.32.20 now valid: directly connected
> > >>>>listening on 0.0.0.0
> > >>>>listening on ::
> > >>>>SE reconfigured
> > >>>>neighbor 180.25.32.1 (THN): state change None -> Idle, reason: None
> > >>>>neighbor 180.25.32.1 (THN): state change Idle -> Connect, reason: Start
> > >>>>RDE reconfigured
> > >>>>neighbor 180.25.32.1 (THN): state change Connect -> OpenSent, reason:
> > >>>>Connection opened
> > >>>>neighbor 180.25.32.1 (THN): state change OpenSent -> OpenConfirm,
> > >>>>reason: OPEN message received
> > >>>>neighbor 180.25.32.1 (THN): state change OpenConfirm -> Established,
> > >>>>reason: KEEPALIVE message received
> > >>>>Rib Loc-RIB: neighbor 180.25.32.1 (THN) AS12345: update 180.25.32.0/22
> > >>>>via 180.25.32.1
> > >>>>nexthop 180.25.32.1 now valid: directly connected
> > >>>>
> > >>>>
> > >>>>THN#show ip bgp
> > >>>>BGP table version is 14, local router ID is 180.25.32.1
> > >>>>Status codes: s suppressed, d damped, h history, * valid, > best, i -
> > >>>>internal,
> > >>>> r RIB-failure, S Stale, m multipath, b backup-path, f
> > >>>>RT-Filter,
> > >>>> x best-external, a additional-path, c RIB-compressed,
> > >>>>Origin codes: i - IGP, e - EGP, ? - incomplete
> > >>>>RPKI validation codes: V valid, I invalid, N Not found
> > >>>>
> > >>>> Network Next Hop Metric LocPrf Weight Path
> > >>>>* 180.25.32.0/22 180.25.32.20 0 66868 i
> > >>>>*> 180.25.32.20 0 66868 i
> > >>>>
> > >>>>WE NOW HAVE TWO ROUTES IN THE CISCO BGP RIB WITH THE CARP IP AS A
> > >>>>RESULT OF ONLY STARTING OPENBGPD WHEN THE FIREWALL IS A BACKUP.
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>TEST 5 - Without shutting down OpenBGPd switch the firewalls back
> > >>>>
> > >>>>[LIVE]root@OpenBSD1:~# ifconfig -g carp -carpdemote 10
> > >>>>
> > >>>>[LIVE]root@OpenBSD1(debug);
> > >>>>nexthop 180.25.32.1 now valid: directly connected
> > >>>>nexthop 180.25.32.20 now valid: via 180.25.32.20
> > >>>>
> > >>>>[LIVE]root@OpenBSD2(debug);
> > >>>>nexthop 180.25.32.20 now valid: via 180.25.32.20
> > >>>>nexthop 180.25.32.20 now valid: directly connected
> > >>>>
> > >>>>
> > >>>>THN#show ip bgp
> > >>>>BGP table version is 14, local router ID is 180.25.32.1
> > >>>>Status codes: s suppressed, d damped, h history, * valid, > best, i -
> > >>>>internal,
> > >>>> r RIB-failure, S Stale, m multipath, b backup-path, f
> > >>>>RT-Filter,
> > >>>> x best-external, a additional-path, c RIB-compressed,
> > >>>>Origin codes: i - IGP, e - EGP, ? - incomplete
> > >>>>RPKI validation codes: V valid, I invalid, N Not found
> > >>>>
> > >>>> Network Next Hop Metric LocPrf Weight Path
> > >>>>* 180.25.32.0/22 180.25.32.20 0 66868 i
> > >>>>*> 180.25.32.20 0 66868 i
> > >>>>
> > >>>>
> > >>>>THE ROUTES CONTINUE TO BE ANNOUNCED USING THE CARP IP.
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>So we can see that the nexthop is only correctly set when; nexthop
> > >>>>180.25.32.20 now valid: directly connected
> > >>>>
> > >>>>
> > >>>>
> > >>>>On Tue 03 Dec 2013 02:26:30 GMT,[email protected] wrote:
> > >>>>>No, I'm seeing the same thing - the carp master advertises the carp
> > >>>>>IP as next-hop no matter what.
> > >>>>>The carp backup advertises whatever you've told it to advertise via
> > >>>>>"set nexthop".
> > >>>>>-Adam
> > >>>>>
> > >>>>>On Dec 2, 2013 6:43 PM, Chris Cappuccio<[email protected]> wrote:
> > >>>>>>
> > >>>>>>andy [[email protected]] wrote:
> > >>>>>>>Hi,
> > >>>>>>>
> > >>>>>>>Could someone help me with this issue we have found where the
> > >>>>>>>OpenBGPd
> > >>>>>>>rule 'match to bgppeerip set nexthop bgpcarpip' doesn't work if
> > >>>>>>>OpenBGPd is
> > >>>>>>>started whilst the OpenBSD host is a carp master. It only works if
> > >>>>>>>it is a
> > >>>>>>>CARP backup :(
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>Or could someone give me a clue where in the source code to look so
> > >>>>>>>I can
> > >>>>>>>try to comment out the code which is checking the state of carp?
> > >>>>>>>This is
> > >>>>>>>desperately important for us for reasons discussed in this thread and
> > >>>>>>>others.
> > >>>>>>>
> > >>>>>>>Thanks for your time, Andy.
> > >>>>>>>
> > >>>>>>>PS; Thanks to Henning and Claudio for their great work with OpenBGPd.
> > >>>>>>>
> > >>>>>>
> > >>>>>>Can you demonstrate the failure through any bgpd output or some
> > >>>>>>other way?
> > >>>>>>
> > >>>>>>For instance, does bgpd fail to advertise routes via bgp if it's the
> > >>>>>>CARP nexthop master?
> > >>>>>>
> > >>>>>>Or does it all look like it should work, and just fail?
> > >>>
> > >>
> > >>
> > >
> >
> >
>
> --
> Henning Brauer,[email protected],[email protected]
> BS Web Services GmbH,http://bsws.de, Full-Service ISP
> Secure Hosting, Mail and DNS Services. Dedicated Servers, Root to Fully
> Managed
> Henning Brauer Consulting,http://henningbrauer.com/
[demime 1.01d removed an attachment of type image/jpeg which had a name of
bgp_and_carp_nexthops (1).jpg]
