Hi, Could someone help me with this issue we have found where the OpenBGPd rule 'match to bgppeerip set nexthop bgpcarpip' doesn't work if OpenBGPd is started whilst the OpenBSD host is a carp master. It only works if it is a CARP backup :(
Or could someone give me a clue where in the source code to look so I can try to comment out the code which is checking the state of carp? This is desperately important for us for reasons discussed in this thread and others. Thanks for your time, Andy. PS; Thanks to Henning and Claudio for their great work with OpenBGPd. On Thu, 21 Nov 2013 16:44:14 +0000, Andy <[email protected]> wrote: > Ah, so we have a potential bug here then I'm thinking! > > After all, why would the setting of nexthop have anything to do with > CARP? > > > On Thu 21 Nov 2013 16:14:33 GMT, Adam Thompson wrote: >> (Apologies for top-posting) >> >> I've seen the same thing, but I assumed I'd made a mistake somewhere. >> Maybe not. >> >> -Adam >> >> >> Andy <[email protected]> wrote: >> >>> On 15/11/13 16:50, Adam Thompson wrote: >>>> On 13-11-15 04:17 AM, Andy wrote: >>>>> On 12/11/13 05:48, Chris Cappuccio wrote: >>>>>> Two BGP sessions from different IPs (no CARP) >>>>>> BGP next-hop pointing to CARP-protected IP >>>>> >>>>> Hi Chris, >>>>> This sounds good.. Could you clarify further? >>>> >>>> I can clarify for him, see below. (Apologies if he's already done it >>>> - I'm on the daily digest.) >>>> >>>>> Setup eBGP to the Transit router on both OBSD boxes using physical >>>>> IPs, and iBGP between the OBSD routers. Got that working fine without >>>>> 'depends on' (don't want the BGP teardown/setup delay. >>>> >>>> Yup. >>>> >>>>> How are you configuring the BGP next-hop to the CARP IP?? >>>> >>>> "match to x.x.x.x set nexthop x.x.x.x" >>>> "allow from any" >>>> "allow to any" >>>> >>>>> Hi Adam, >>>>> The problem is to do with ensuring inbound packets always go to the >>>>> CARP master. >>>> >>>> That's what "set nexthop" does in BGP - it tells the *other* router >>>> what to use for its nexthop. >>> >>> Hi, I have observed some strangeness with this! :( >>> >>> I have two OpenBSD firewalls running in a CARP pair. Each firewall in >>> the pair has a single eBGP neighbor with the same single Cisco router >>> using its physical IP with no 'depends on' statement. >>> >>> I have added the following line to /etc/bgp.conf on both firewalls; >>> match to 170.16.3.1 set nexthop 170.16.3.4 >>> >>> NB; 170.16.3.1 is the Cisco router and 170.16.3.4 is the CARP IP of the >>> firewall pair. >>> >>> >>> If I start BGP on FW1 (master), the announced network seen in the Cisco >>> has a nexthop = the physical IP and not the CARP IP :( >>> If I start BGP on FW2 (backup), the announced network seen in the Cisco >>> has a nexthop = the CARP IP :) >>> >>> Hmm, strange.. Maybe something is wrong with the master config I >>> thought, but lets just try switching CARP first. >>> >>> So I stopped OpenBGPd on both and swapped the CARP master to be the >>> other firewall etc. >>> >>> If I start BGP on FW1 (backup), the announced network seen in the Cisco >>> has a nexthop = the CARP IP :) >>> If I start BGP on FW2 (master), the announced network seen in the Cisco >>> has a nexthop = the physical IP and not the CARP IP :( >>> >>> >>> This is really strange! It seems that only the CARP backup sets the >>> nexthop properly. >>> >>> Just for kicks, I shut down BGP on both and restarted BGPd on just the >>> backup. Cisco shows one route via the CARP IP as wanted. >>> I then swapped the CARP master again, and started BGP on the other >>> firewall (just made backup). And now the Cisco shows two routes both via >>> the CARP IP... This is what we want all the time. >>> >>> This confirms that if BGP is started when its the backup it works, but >>> if its started when its the master, its the nexthop is the physical IP? >>> >>> Any thoughts as I'm lost.. This is just strange! >>> Cheers, Andy. >>> >>>> >>>>> 'match to X.X.X.161 set nexthop X.X.X.162' Wouldn't this only mean >>>>> that the outbound packets would egress to the transit via the CARP >>>>> IP? Its the inbound control that's needed. >>>> >>>> Nope. It's actually much more difficult to control the egress IP, >>>> AFAIK. >>>> >>>>> I was thinking about using ifstatd to dynamically change the MED / >>>>> path prepending based on the CARP status, rather than trying to force >>>>> which router is master. Experience says that fail-overs happen for >>>>> many reasons (probably once every couple of months), but so far never >>>>> because the master is actually dead, which means BGP will pretty much >>>>> always be left running on the old master (unless ifstatd does >>>>> something to it).. >>>> >>>> With 'set nexthop', it's OK if the old BGP session stays up - packets >>>> will always come inbound to the CARP master. You don't need to do >>>> anything to bgpd or routing tables on the old box. >>>> >>>> What you *might* have to do is use ifstated(8) to ensure that the >>>> "LAN" carp(4) interface always stays in sync with the "WAN" carp(4) >>>> interface. (i.e. router #1 being master for inside-facing while #2 is >>>> master for outside-facing will break pf(4).) >>>> >>>>> I just can't seem to figure out a true clean way of doing this >>>>> without configuring multiple BGP attributes in OpenBGPd based on CARP >>>>> status :( >>>> >>>> I think that's only because you had the wrong end of the stick for the >>>> nexthop attribute. >>>> >>>>> PS; For inbound path control which would you recommend? MED or >>>>> padding the AS path? I.e. is one potentially more responsive than >>>>> another.. >>>> >>>> Neither! Just "set nexthop" appropriately.
