Hi sorry to reply with a stupid question but I get this error;
[LIVE]root@mg1311:~# pkg_add -i symon
symon-2.86p0:libart-2.3.21: ok
symon-2.86p0:png-1.6.2p0: ok
Can't install rrdtool-1.2.30p3 because of libraries
|library freetype.20.0 not found
| not found anywhere
Direct dependencies for rrdtool-1.2.30p3 resolve to png-1.6.2p0
libart-2.3.21
Full dependency tree is png-1.6.2p0 libart-2.3.21
Can't install symon-2.86p0: can't resolve rrdtool-1.2.30p3
I've come across this issue of missing freetype in the OpenBSD sources
before in older versions.
Anyway, I think you could be on the money as every time this has
happened, it always happens just after seeing the message nexthop now
valid.
We don't use nexthop-self on our cisco routers which are connected to
Transits and IXPs.
At the moment the OpenBSD routers only have iBGP neighborships with two
cisco ABR routers (no eBGP transit or IXP connections, just internal
AS).
Thus the BGP nexthop networks (transit and IXP links) are redistributed
to the internal OpenBSD routers by IGP (OSPF) to validate the BGP
nexthops and provide BGP prefix independent convergence.
In most cases as far as I can remember, the penultimate message before
the crash is the CARP partner's nexthop becoming valid.
Just to throw the question out there. iBGP should always be a full mesh
(or use RRs etc), and so we run iBGP from both of the OpenBSD firewalls
to every other router, and I also run iBGP between the CARP pair
themselves too. Is the iBGP between the CARP pair really needed??
I'd imagine not as they are always active-backup and one should not be
handing packets to the other etc.. (we only direct packets to the CARP
addresses as they are running as both stateful firewalls *and*
stateless routers, hence other posts here regarding trying to set the
nexthop on BGP announcements to the CARP IPs).
Thanks for your thought Stuart.
Cheers, Andy.
On Wed 30 Apr 2014 04:14:55 BST, Stuart Henderson wrote:
On 2014-04-28, Andy <[email protected]> wrote:
Yea thats all I see in /var/log/messages ..
/var/log/daemon had;
2014-04-28T01:02:21.238360+01:00 mg1311 ospf6d[25154]: send_rtmsg: action 1,
prefix ::/0: File exists
2014-04-28T01:02:22.048344+01:00 mg1311 ospfd[19386]: desync; scheduling fib
reload
2014-04-28T01:02:32.518186+01:00 mg1311 ospfd[19386]: reloading interface list
and routing table
2014-04-28T01:02:59.077656+01:00 mg1311 bgpd[18490]: nexthop 185.25.32.22 now
valid: directly connected
2014-04-28T01:03:01.762457+01:00 mg1311 bgpd[18490]: dispatch_imsg in main:
pipe closed
2014-04-28T01:03:01.762473+01:00 mg1311 bgpd[18490]: Lost child: route decision
engine exited
2014-04-28T01:03:01.762756+01:00 mg1311 bgpd[18490]: kernel routing table 0
(Loc-RIB) decoupled
2014-04-28T01:03:01.778424+01:00 mg1311 bgpd[18490]: Terminating
It had been up for about a month before this. bgpd on the carp backup
is still up and running fine and never dies.
It might be interesting to monitor memory use (symon is fairly good
for this), one failure mode is that bgpd can't handle nexthop revalidation
churn fast enough and the rde eats all memory (or at least runs into
login.conf datasize limits - make sure these are sufficiently high
if you haven't already done so). I don't remember all of the various ways
this can show up in logs, but what you're seeing here does ring a bell.
I work around this by only feeding a partial table to routers which are
particularly likely to suffer ospf churn ... not ideal but stability
improved a lot after doing that. (It used to happen more often but
there was a change in 5.2 which seriously reduced the situations that
trigger this).
It's always been the master that falls over. I might make the other
firewall the master and see if it still crashes on the same box.
Is there anyway I can increase the session engine logging?
There's bgpd_flags="-v" if you're not already using it - I'm not too
sure if it will actually give you any more information but worth a try.
Make sure syslogd does actually log the messages - I use memory buffer
logs for these,
syslogd_flags="-s /var/run/syslogd.sock"
with this the top of syslog.conf:
!!bgpd
*.* :256:bgpd
daemon.info /var/log/daemon
!*
