OpenBGPd max-prefix
Hi, One funny thing today. One of our customer did announce us too many routes. The max-prefix has been reached (was 5) and the session closed. A few seconds later I saw several peering sessions go down in the logs but did not thought about any links between events. Having had exchange with network managers on the other sides, they told me I reached the max-prefix on their side (was at 10 and I usually announce 2 routes + 5 from my customer). That means I did announce up to 10 routes at some point. I announce 2 for us and should have 5 more from my customer. That's 7 routes max. How could I have reached 10 announced prefixes ? What I imagine is that for a few seconds I did announce all the routes I received from my customer before the max-prefix did cut the session ? Would this be possible that max-prefix is not synchronously checked ? BR, -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
Re: OpenBGPd max-prefix
well, your 2 plus the 5 from your other customers plus the $max-prefix The 5 is the $max_prefix. We have just only one BGP customer. Total is 7. I should never have announced more than 7 routes in any case. -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/ Tel: +33 (0)1 30 42 72 95 Gsm: +33 (0)6 30 79 26 33
OpenBGP nexthop
Hi, OpenBSD-current amd64 from around march, 20th. Next to a reboot, OpenBGP had a problem validating NextHops : Nexthop State x.x.x.105 invalid vlan97 UP, Ethernet, no carrier, 100 MBit/s I had about 30 addresses on different vlans in this case. This resulted in the BGP session being up but routes were not valid, thus not installed. I tried to ifconfig down one vlan and it crashed the whole box. Once rebooted, everything was fine. There is two points about this : - Why is this happening ? What can I do to avoid this and/or get back into business without crashing the system ? Could this have to do with the OpenBGP/kroute.c patch sent yesterday ? - Shouldn't OpenBGP drop the session if the nexthop is not valid ? In our case, we do announce another AS behind us. Sessions were up, so I believe routes were announced to everybody. But as nexthop was declared invalid, routes to this AS were not installed. I think this could have created a loop : our upstream provider was sending packets for this customer's routes. As we hadn't routes installed for customer's prefix, we were sending back the packets to upstream ... BR, -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
Re: OpenBGP nexthop
What was the state of the parent interface and what kind of interface is it? Bge driver. It was up and running : BGP sessions were established through the vlans reported as invalid by OpenBGP. ifconfig down should not crash the box. Panic message and trace would be interesting. It was remote and we did a hard reboot without console access. Log files were empty. No, the session and the nexthop are two different things. I agree. My point is : how to prevent routing loops in such cases ? Whatever triggered the case (a link down for any reason or a bug) is not so important. Announcing routes over the Internet and creating a routing loop for those routes is important. It could be one more setting that, if set to yes, would drop the session if it receives an unreachable nexthop ... just an idea. It could default to yes for eBGP session and no for iBGP sessions. Would that fit most of usual cases ? BR, -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
OpenBGP crashes
Funny, I also have this : Apr 12 16:48:29 x bgpd[10601]: Lost child: session engine terminated; signal 11 Apr 12 16:48:29 x bgpd[31105]: fatal in RDE: rde_dispatch_imsg_session: pipe closed Apr 12 16:48:29 x bgpd[10601]: Lost child: route decision engine exited Once every two or three weeks. Usually where bgpd has some work to do or when the box has more traffic ... Any idea ? BR, -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
[Update] OpenBGP crashes
Funny, I also have this : Apr 12 16:48:29 x bgpd[10601]: Lost child: session engine terminated; signal 11 Apr 12 16:48:29 x bgpd[31105]: fatal in RDE: rde_dispatch_imsg_session: pipe closed Apr 12 16:48:29 x bgpd[10601]: Lost child: route decision engine exited I forgot to see it but just before, there is also : Apr 12 16:48:29 x bgpd[31105]: neighbor x.x.x.x (x): prefix limit reached -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
CPU usage monitoring
Hi all, Is there any way to monitor CPU usage (preference through snmp) on an OpenBSD box ? LoadAverage is reported through netsnmp, but this does not report real CPU usage. As a newbie, I didn't find an easy way to do this. Thanks for tips. BR, -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/ Tel: +33 (0)1 30 42 72 95 Gsm: +33 (0)6 30 79 26 33
pf/carp load balancing on 4 firewalls
Hi gurus, I'm working on a project where carp loadbalancing firewalls could exactly fit our needs. Before that, I wonder how it will work outside of the OpenBSD boxes. First, regarding Carp and STP what happens usually in a manageable L2 switch when the same MAC is announced on two different ports ? I don't remember that STP includes loadsharing, so isn't it possible the switch will only choose one port to forward on ? Please excuse me if it sounds stupid and just explain why ;-) Next, my setup would involve 4 firewalls connected 2 by 2 on two switches, themselves connected together through one port. That setup would connect two or more, but it doesn't matter here) servers : FW1A FW1B FW2A FW2B || || || || SWITCH1-SWITCH2 | | | | SRV1SRV2 Once again, how will spanning tree handle this case with the same MAC announced from the 4 firewalls ? My guess is packets from SRV1 will be dispatched to FW1* because the cost will be lower. Same for SRV2/FW2*. Could some help me understand how this setup could behave in real ... BR, -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
Re: pf/carp load balancing on 4 firewalls
Thanks Jason for the details. I'm quite good from L3 and up, but I still never had to understand so much about L2 ;-)) The problem you will/may encounter will differ based on the vendor of SWITCH1 and SWITCH2. Some vendors will handle it OK if the MAC is a multicast MAC, some will log a warning, some will not allow it and simple accept the first port, some will forward randomly. OK, and it will be a multicast MAC as long as I remember how do carp work. Would it mean the frame could be duplicated ? Is there any good article/tutorial about this you're aware of ? This is a pure vendor-implementation issue of how they forward frames and if their CAM/FDB/Forwarding Database/whatever they call it allows multiple entries and if it expires entries on ports that go down. That's bad news :-( switches, themselves connected together through one port. That setup With all that attention to redundacy, why not make the link between SWITCH1 and SWITCH2 two links or more? I have to be honest : the posted schema is a simplified one. It misses two informations : - There could be two levels of switches involved between servers and firewalls (from two different vendors !), - The link between switches is a metro link. That's why I am interested in having the lowest possible number of frames from one server being forwarded to remote firewalls. Routers will choose the right destination after the firewalls, but I would like to keep server=firewalls traffic as much local as possible. If you have a relationship with the vendor, ask them. Not still sure of the vendor. Should be 3COM. Or simply try it out and report back! I don't have the hardware, I must plan this for the end of the month. BR, -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
Re: OpenBGPd-current memory
that indeed smells like a bug somewhere. Obviously : my bgpd/rde is now eating 21 more MB than a few ours ago (267 MB total). This morning's statistics : bgpd/rde is eating 481 MB (after a bgpctl reload). # bgpctl sh rib memory RDE memory statistics 177310 IPv4 network entries using 10.8M of memory 682 IPv6 network entries using 48.0K of memory 709886 prefix entries using 37.9M of memory 140820 BGP path attribute entries using 16.1M of memory 29625 BGP AS-PATH attribute entries using 1.2M of memory, and holding 140820 references 3670 BGP attributes entries using 143K of memory and holding 167371 references 3669 BGP attributes using 24.1K of memory RIB using 66.2M of memory Any idea about how I should handle this ? ... ;) BR, -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/ Tel: +33 (0)1 30 42 72 95 Gsm: +33 (0)6 30 79 26 33
OpenBGPd-current IPv6
Hi all, I upgraded yesterday to OpenBSD-current and re-ran IPv6 tests. Now, routes are not installed into the kernel. My config : #bgpctl sh int ... vlan97 ok UP Ethernet, active, 100 MBit/s ... #bgpctl sh nexthop ... 2001:xxx:21 valid ... This perhaps shows the problem : for other nexthops, the state of the interface is printed after valid, I'm not sure it should be the case here. And when Updates are received, it turns into : send_rtmsg: action 1, prefix 2001:398::/32: Network is unreachable Currently, no IPv6 routes are installed to the kernel while more than 600 are received. BTW, there is a zebra daemon running. It is not problem for IPv4, could it be for IPv6 ? BR, -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
OpenBGPd-current memory
Hi, I upgraded to OpenBGPd-current (09/2/2006 snapshot) and didn't noticed yesterday about the memory usage. But, checking the rde process memory this morning gave surprising results : Box 1 : OpnBSD-current, 1 IPv4 full mesh eBGP, 1 IPv6 eBGP (681 routes), 1 iBGP to Box 2, and 10-12 peers (2 or 3 routes per peer) : 169 MB. Before the upgrade, I was running at something like 60-80 if I remember it well. Box 2 : OpnBSD 3.8, 1 IPv4 full mesh eBGP, 1 iBGP to Box 1, 1 peer (3 routes) : 57 MB. Stable. On Box 1, I play a little with communities : each route is tagged with 2 to 6 communities attributes ;-) Any guess what cause such a high memory load on my first box ? BR, -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
Re: OpenBGPd-current memory
Softreconfig in. If you modify the path attributes on from rules the will be added twice to the table. You can turn softreconfig in off in -current via the peer directive softreconfig in no. OK. This is just a feature so ;-) Could I try to understand what softreconfig does and not as it is undocumented (or did I search the wrong place ?). As far as I understand it from your mail and CVS comment, if set to on, it will reapply all filters when configuration is reloaded, without the need to restart ? Is that it ? Is it reliable currently ? If I keep it turned on, what will be the memory overhead ? Currently my memory load goes up every hour (between 1 and 2 MB per hour). Will this stop at some point and how could try to approximate the final load ? Thanks for your help. BR, -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
Re: OpenBGPd-current memory
The increase you are seeing might just be fragmentation. I did play a little with my config this afternoon. I ran two reloads and I'm currently eating near 250 MB. I'll continue to monitor in the next hours, but it begins to be a little too much for just one full eBGP and one iBGP ! At some point, I'll reset the sessions or restart the server to try to approx. the amount of memory lost due to fragmentation. BR, -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
Re: OpenBGPd-current memory
that indeed smells like a bug somewhere. How could I try to track this down ? BR, -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
Re: OpenBGPd-current memory
that indeed smells like a bug somewhere. Obviously : my bgpd/rde is now eating 21 more MB than a few ours ago (267 MB total). # bgpctl show rib memory RDE memory statistics 177462 IPv4 network entries using 10.8M of memory 682 IPv6 network entries using 48.0K of memory 355940 prefix entries using 19.0M of memory 70704 BGP path attribute entries using 8.1M of memory 29562 BGP AS-PATH attribute entries using 1.2M of memory, and holding 70704 references 3527 BGP attributes entries using 138K of memory and holding 83977 references 3526 BGP attributes using 21.4K of memory RIB using 39.3M of memory I know deliver a full feed to a downstream customer and it looks each time he resets the session, my memory usage goes up ! It just looks like, I was unable to track this down better. Any idea about how I should handle this ? BR, -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
OpenBGP Communities manipulations
Hello, I'm trying to play around with OpenBGP 3.8 communities and I'd like to define several communities depending on the peers. When I set communities this way : match to any set community x:10 match to any set community x:20 Only x:20 will be set. Each set statement wipes out previous communities. Is there a way to *add* a new community to the path without wiping previous ones ? BTW, does someone have a complex community manipulation conf file to send me (in private) so I can learn tips from it ? BR, -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
Re: OpenBGP Communities manipulations
There is a feature in 3.8 that let you only set one community per AS. This is fixed in -current. OK. BTW, how one could remove community tags ? BR, -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
Re: 3.8/64 bits/snmp
I've seen the same on amd64 (OpenBSD 3.7 and 3.8) running net-snmp 5.x. Yep, that's it ;-) I haven't noticed any issue with interface counters, On our platform, interface counters are sent back using Counter32 while carrying 64 bits values. It works while the counter is less than 4 GB but our monitor rejects larger values ... The problem is with net-snmp. Beyond this I haven't chased it down. But not on all platform. Netsnmp 5 works great with OpenBSD i386 or Ubuntu amd64. BR, -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
Re: OpenBGPd and TTL Security Check (RFC 3682)
Why don't you use IPSec? Or as second best solution TCP MD5? Both are supported by OpenBGPD and give you more protection that playing around with the IP TTL. Hum... some people rather like such options I rather like using TCP MD5 or IPSec... IPsec is not widely supported and md5 causes timeout detection problems. TTL security check is a way to have a small but quite efficient protection. Obviously, everyone will prefer one or the other way, but there are arguments for TTL check as for others. BR, -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
3.8/64 bits/snmp
Hi list, We have problems grabbing statistics through snmp on our amd64 config. Using netsnmp and scripts that work on many other systems (OpenBSD 3.8 i386, Ubuntu Linux amd64, Debian Sarge i386) we are unable to get the CPU usage (always returns 0) and network interfaces return 64 bits counters in 32 bits OIDs. It looks it's long time known problems but we were unable to find a workaround. Any idea ? BR, -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
Re: OpenBGP IPv6
Hi, Try: announce IPv4 unicast announce IPv6 unicast Nothing does :( -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
Re: OpenBGPd filters
Yes they need the session up/down to be applyed I'm not sure, but some of them (I think localpref defined in a group) require the whole daemon to be restarted. Regards, -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
Re: OpenBGP IPv6
Both Cisco based equipmentthere must be something else wrong in the configuration... Any idea what could be else ? ;-) BR, -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
Re: OpenBGP IPv6
so the neighbor refuses our OPEN messgae because we announce some capability it dioesn't like, I traced frames and had a look at BGP's OPEN frames. I think neighbor doesn't like the IPv4 capability !! However, I can't change the configuration (reload, neighbour clear), IPv4 is always the only capability announced. I believe I need to kill bgpd and restart it for the change in the configuration to take effect. I can't for now. BR, -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
Re: OpenBGP IPv6
However, I can't change the configuration (reload, neighbour clear), IPv4 is always the only capability announced. I believe I need to kill bgpd and restart it for the change in the configuration to take effect. I can't for now. hmm, there might have been an issue with changing the announced address families, I vaguely remember fixing something there - tho I don't remember when. Might have been post-3.8 I finally found that deleting the neighbor from the conf, reloading, adding back the neighbor was able to reset (when not changing groups settings) and works. Now I'm facing several (not so) funny ones : 1/ Routes are not installed because bgpd[31578]: send_rtmsg: action 1, prefix 3ffe:800::/24: Network is unreachable. I didn't found what was causing that. I killed bgpd and restarted, nothing does. bgpctl show interfaces shows the interface as ok/UP. bgpctl show nexthop show nothing about the nexthop (others are ..., UP, active, ...). This should be the problem but I don't know how to investigate. 2/ bgpd crashes when the first IPv6 withdraw occurs : Jan 10 23:11:08 r1 bgpd[2945]: neighbor 2001:x: (AS) withdraw 2001:13a8::/48 Jan 10 23:11:08 r1 bgpd[31578]: Lost child: route decision engine terminated; signal 11 [...} Jan 10 23:11:08 r1 bgpd[26296]: session engine exiting Oops :-( 3/ Now bgpctl show nexthop shows a nexthop which does not exist. Its IPv4 address is nothing I know about (and is not the first 32 bits of a v6 address). -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
Re: OpenBGPd filters
I'm not sure, but some of them (I think localpref defined in a group) require the whole daemon to be restarted. certainly not. So what should I do to change the localpref assigned this way ? neighbor clear did not changed the localpref last time I checked ... I didn't try to delete the neighbour, reload and add back. BR, -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
OpenBGP Communities
Hi again, How can one see community tags associated to a route ? BR, -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
Re: OpenBGP IPv6
I was just about to create a new thread when I read :
I have not seen it discussed much on the list, but OpenBGP works *very*
well and is easy to setup using Hurricane Electrics free (ipv6-in-ipv4)
I try to setup a BGP peering with upstream and I have (v3.8) :
Jan 7 10:11:19 r2 bgpd[31645]: neighbor 2001:x:21 (x-v6): state change Idle -
Connect, reason: Start
Jan 7 10:11:19 r2 bgpd[31645]: neighbor 2001:x:21 (x-v6): state change Connect
- OpenSent, reason: Connection opened
Jan 7 10:11:19 r2 bgpd[31645]: neighbor 2001:x:21 (x-v6): state change
OpenSent - OpenConfirm, reason: OPEN message received
Jan 7 10:11:19 r2 bgpd[31645]: neighbor 2001:x:21 (x-v6): received
notification: error in OPEN message, unsupported capabil
ity
Jan 7 10:11:19 r2 bgpd[31645]: neighbor 2001:x:21 (x-v6): parse_notification:
capa_len 16 exceedsremaining msg length
Jan 7 10:11:19 r2 bgpd[31645]: neighbor 2001:x:21 (x-v6): state change
OpenConfirm - Idle, reason: NOTIFICATION received
Upstream told me about capability : We do 'inet6.unicast' only. Upstream
router is a Juniper.
Relevant configuration is :
network 2001:1b58::/32
group Upstream {
set localpref xxx
announceself
neighbor x.x.x.x {
remote-as x
descr x-v4
}
neighbor 2001:x:0021 {
remote-as x
descr x-v6
}
}
Any idea what I've done wrong again ?
BR,
--
Sylvain COUTANT
ADVISEO
http://www.adviseo.fr/
http://www.open-sp.fr/
Re: OpenBGPd filters
Hello Claudio, I just tried a filter like this: match from any AS 29166 set nexthop blackhole and that did work: Yes. I have run more tests since my previous post, and filter apply the blackhole tag to the route. But packet were still forwarded. After having stopped the session and restarted it, they were then dropped. BTW, all tags (localpref, etc.) manipulations require to clear the session and even to restart bgpd itself. Could this be included in the doc so one wouldn't have to search for hours on some settings change ? Thanks for your reply. -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
OpenBGPd filters
Hi and happy new year to all,
I try to apply a nexthop blackhole filter without success on OpenBSD 3.8.
I receive the bogon list from cymru and try to force blackholing of the
routes without success. Here is my configuration :
group BGPBogon {
remote-as 65333
announcenone
multihop255
set localpref 999
neighbor x.x.x.x {
descr BGP-Bogon
local-address y.y.y.y
}
}
Later I apply the filter :
match from group BGPBogon community 65333:888 set nexthop blackhole
I tried several combinations with the reject keyword and without community
filter also, but routes are installed in the fib with a valid nexthop anyway
and the server sends the packets for those routes. I even tried to force the
nexthop at the group level without success ... !
If someone can explain me what I'm missing - any help welcome ;-)
--
Sylvain COUTANT
ADVISEO
http://www.adviseo.fr/
http://www.open-sp.fr/
Re: OpenBGP+CARP : OpenBGP does not see CARP going into master state
* Sylvain Coutant [EMAIL PROTECTED] [2005-12-26 11:29]: OpenBGPd looks fine for eBGP and iBGP links as long as it does not depend on carp. definately works for me Good. I was not very it was used in production somewhere ;-) I don't have any more the resources right now to check as I had to throw the routers to production sooner that I'd wish. According to bgpctl show interfaces all carp interfaces are backup when ifconfig shows them all as master ... I'll post some test results when I'll be able to reproduce and understand a little better what I've done wrong. Regards.
Re: OpenBGP+CARP : OpenBGP does not see CARP going into master state
If you are using bgpd you can add redundancy to your network in a more flexible way than via carp. Terminate your upstreams on multiple boxes run an IBGP mesh and you get failover too. We do. *And* we could also terminate all upstream BGPs (not peerings) on all boxes using carp and have even better failover ... Best wishes ;=) -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
Re: OpenBGP+CARP : OpenBGP does not see CARP going into master state
Hi, I think this depend on is a nice feature - but I would not use for 100% fail save connections. Why not ? It has been coded for this purpose ... You must take into account, that the session will go down if you trigger a failover. Of course, this is the basic of a failover between two routers. This might be acceptable for some kind of sessions (peerings, backup links) but may be undesirable for main (transit) links. This is *highly* desirable in any situation where one router goes down for any reason. Look, we're in a *failover* case. Session going down for 10 seconds is better than session going down until someone brings it back up ... Regards, -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
OpenBGP+CARP : OpenBGP does not see CARP going into master state
Hi all, I'm running some tests using an out of the box OpenBSD 3.8. OpenBGPd looks fine for eBGP and iBGP links as long as it does not depend on carp. When a bgp peer depends on a carp interface, OpenBGP does not see the interface going master and does not trigger connections up. I tried to bgpctl reload manually, but this does nothing. bgpctl show interfaces always show that carp devices never come back master once they entered backup state. I need to kill/restart bgpd in every case. My config does just include depend on carp3 for one eBGP neighbour in this case. Is this kind of a bug or do I miss something ? It's my first round with this configuration, I could have forgot one important thing ... Thanks in advance for any help. Regards and happy Xmas. -- Sylvain COUTANT ADVISEO http://www.adviseo.fr/ http://www.open-sp.fr/
