Re: OpenBGPD prepend-self/neighbor question
Yet my upstream still prefers core2a as correct route to our network. I noticed, that only core2a networks have announced flag, is that right? Any other ideas what could be wrong? If you look at the Loc-Rib aka 'bgpctl show rib 194.143.152.1 all' it will show you that there are two networks for 194.143.152.0/23 on core2b. This comes from the fact that core2a is announcing his network to core2b and the route from core2a is considered better and therefor selected and announced. The A flag is only set on local networks. Actually if I run this command, it doesn't show any network: # bgpctl show rib 194.143.152.1 all flags: * = Valid, = Selected, I = via IBGP, A = Announced origin: i = IGP, e = EGP, ? = Incomplete flags destination gateway lpref med aspath origin (output is identical on both primary and secondary router). Now if the upstreams always selects one route over another then it is a missconfiguration on their side (e.g. there is still a static route somewhere configured or something else). I asked them for all the configuration they have for our network, they claim they don't have any static routes or any other routing protocol involved. They also sent me how my network looks from their routers (I still have prepend-self and prepend-neighbor values 1 on primary, not secondary router): sh ip bgp 194.143.152.1 BGP routing table entry for 194.143.152.0/23, version 73502 Paths: (4 available, best #1, table Default-IP-Routing-Table) Advertised to update-groups: 1 3 4 5 6 12 21178 159.148.214.98 from 159.148.214.99 (159.148.214.99) Origin IGP, localpref 100, valid, external, best Community: 2588:400 2588:500 2588:700 21178, (received-only) 159.148.214.98 from 159.148.214.99 (159.148.214.99) Origin IGP, localpref 100, valid, external 21178 21178 159.148.214.98 from 159.148.214.98 (159.148.214.98) Origin IGP, localpref 100, valid, external Community: 2588:400 2588:500 2588:700 21178 21178, (received-only) 159.148.214.98 from 159.148.214.98 (159.148.214.98) Origin IGP, localpref 100, valid, external It looks like secondary router announces best path to primary router is dirrect connection, except that path it is announcing is shorter than expected. Is that correct behavior? Should it be fixed by filters or my secondary router wasn't supposed to announce such short path in the first place? I can also send configuration options of upstream router if necessary. Thank you, Peter
Re: OpenBGPD prepend-self/neighbor question
A) look at bgpd -nv output and check if the filter rules make sense. They look fine, only filter rules on core2b are affected and they look like this: match from 159.148.214.101 set { prepend-neighbor 3 } match to 159.148.214.101 set { prepend-self 3 } deny from any allow from any inet prefixlen 8 - 24 deny from any prefix 10.0.0.0/8 prefixlen = 8 deny from any prefix 172.16.0.0/12 prefixlen = 12 deny from any prefix 192.168.0.0/16 prefixlen = 16 deny from any prefix 169.254.0.0/16 prefixlen = 16 deny from any prefix 192.0.2.0/24 prefixlen = 24 deny from any prefix 224.0.0.0/4 prefixlen = 4 deny from any prefix 240.0.0.0/4 prefixlen = 4 B) use bgpctl show rib nei latnet out to see what prefixes you are actually sending to the other side. This is actually weird, primary router has only our network, but secondary has all networks, but I'm not sure if it should be like that: # core2a flags: * = Valid, = Selected, I = via IBGP, A = Announced origin: i = IGP, e = EGP, ? = Incomplete flags destination gateway lpref med aspath origin AI* 194.143.152.0/230.0.0.0100 0 i # core2b: flags: * = Valid, = Selected, I = via IBGP, A = Announced origin: i = IGP, e = EGP, ? = Incomplete flags destination gateway lpref med aspath origin I* 31.24.192.0/21 159.148.214.101100 0 21178 21178 21178 2588 42480 8194 i I* 31.170.16.0/21 159.148.214.101100 0 21178 21178 21178 2588 42480 5518 49191 i ... [skip] ... I* 194.143.152.0/23 159.148.214.98 100 0 21178 21178 21178 i ... [skip] ... I* 217.198.224.0/20 159.148.214.101100 0 21178 21178 21178 2588 42480 20910 i I* 217.199.96.0/19 159.148.214.101100 0 21178 21178 21178 2588 42480 20797 20797 20797 20797 i I'm not surprised. You must use filter to limit the networks you announce when using announce all. So at least a deny to any and an allow to any prefix 194.143.152.0/23 rule is needed. Thanks Claudio, I've added these filters to my rules, now both my routers announce only my network to the upstream: # core2a: flags: * = Valid, = Selected, I = via IBGP, A = Announced origin: i = IGP, e = EGP, ? = Incomplete flags destination gateway lpref med aspath origin AI* 194.143.152.0/230.0.0.0100 0 i # core2b: flags: * = Valid, = Selected, I = via IBGP, A = Announced origin: i = IGP, e = EGP, ? = Incomplete flags destination gateway lpref med aspath origin I* 194.143.152.0/23 159.148.214.98 100 0 21178 21178 21178 i Now, to test everything again, I removed any prepend-self and prepend-neighbor settings on secondary router and added them to primary router. After doing that and reloading BGPD, everything seems to be fine: # core2a: flags: * = Valid, = Selected, I = via IBGP, A = Announced origin: i = IGP, e = EGP, ? = Incomplete flags destination gateway lpref med aspath origin AI* 194.143.152.0/230.0.0.0100 0 21178 i # core2b: flags: * = Valid, = Selected, I = via IBGP, A = Announced origin: i = IGP, e = EGP, ? = Incomplete flags destination gateway lpref med aspath origin I* 194.143.152.0/23 159.148.214.98 100 0 i Yet my upstream still prefers core2a as correct route to our network. I noticed, that only core2a networks have announced flag, is that right? Any other ideas what could be wrong? Thanks, Peter
Re: OpenBGPD prepend-self/neighbor question
# core2a: flags: * = Valid, = Selected, I = via IBGP, A = Announced origin: i = IGP, e = EGP, ? = Incomplete flags destination gateway lpref med aspath origin AI* 194.143.152.0/230.0.0.0100 0 21178 i # core2b: flags: * = Valid, = Selected, I = via IBGP, A = Announced origin: i = IGP, e = EGP, ? = Incomplete flags destination gateway lpref med aspath origin I* 194.143.152.0/23 159.148.214.98 100 0 i Yet my upstream still prefers core2a as correct route to our network. I noticed, that only core2a networks have announced flag, is that right? Any other ideas what could be wrong? I've just upgraded both routers to 4.9, updated filters to 4.9 defaults, but the problem still remains. Nothing has changed regarding path to upstream ISP. Filtering rules now look like this: match from 159.148.214.101 set { prepend-neighbor 1 } match to 159.148.214.101 set { prepend-self 1 } deny from any deny to any allow from any inet prefixlen 8 - 24 allow from any inet6 prefixlen 16 - 48 allow to any prefix 194.143.152.0/23 deny from any prefix 0.0.0.0/8 prefixlen = 8 deny from any prefix 10.0.0.0/8 prefixlen = 8 deny from any prefix 127.0.0.0/8 prefixlen = 8 deny from any prefix 169.254.0.0/16 prefixlen = 16 deny from any prefix 172.16.0.0/12 prefixlen = 12 deny from any prefix 192.0.2.0/24 prefixlen = 24 deny from any prefix 192.168.0.0/16 prefixlen = 16 deny from any prefix 198.18.0.0/15 prefixlen = 15 deny from any prefix 198.51.100.0/24 prefixlen = 24 deny from any prefix 203.0.113.0/24 prefixlen = 24 deny from any prefix 224.0.0.0/4 prefixlen = 4 deny from any prefix 240.0.0.0/4 prefixlen = 4 deny from any prefix ::/8 prefixlen = 8 deny from any prefix 2001:2::/48 prefixlen = 48 deny from any prefix 2001:10::/28 prefixlen = 28 deny from any prefix 2001:db8::/32 prefixlen = 32 deny from any prefix 3ffe::/16 prefixlen = 16 Peter
OpenBGPD prepend-self/neighbor question
Hello, I'm having hard time figuring out what's wrong with my or my upstream setup. I have set up 2 BGP routers, each with its own session to the same upstream BGP router. Primary router has pretty much default settings for upstream router and IBGP session with secondary router. Secondary router has prepend-self and prepend-neighbor so that it's used only when primary is down. Primary router had IP address 159.148.214.102 and secondary router has 159.148.214.99, so BGP ID value of secondary router was lower. The problem is that up until I changed primary routers IP address to 159.148.214.98 (lower than secondary), my upstream would always choose secondary router as correct path. According to Stuart's post on this mailing list, aspath is much more important in correct path decision than BGP ID: http://www.mail-archive.com/misc@openbsd.org/msg31183.html Since I've changed IP address, everything seems to work ok, but I'm worried my upstream could be ignoring my BGP settings and some traffic would still be forwarded to wrong router. They claim they've checked everything and configuration on their side is correct. Primary router is running OpenBSD 4.6-stable, secondary router is 4.8-stable. I know I should be running 4.9, but from what I checked nothing has changed regarding OpenBGDP path decision since then. Configuration of primary BGP router is: #macros latnet=159.148.214.101 core2b=159.148.214.99 # global configuration AS 21178 router-id 159.148.214.98 log updates holdtime 30 network 194.143.152.0/23 # neighbors and peers group peering AS21178 { remote-as 2588 neighbor $latnet { descr latnet announce all tcp md5sig password xx demote carp } } group IBGP { remote-as 21178 neighbor $core2b { descr core2b announceall tcp md5sig password } } #(default filter rules follow) Secondary BGP router configuration: # macros latnet=159.148.214.101 core2a=159.148.214.98 # global configuration AS 21178 router-id 159.148.214.99 log updates holdtime 30 network 194.143.152.0/23 # neighbors and peers group peering AS21178 { remote-as 2588 neighbor $latnet { descr latnet announce all tcp md5sig password x set prepend-self 3 set prepend-neighbor 3 demote carp } } group IBGP { remote-as 21178 neighbor $core2a { descr core2a announce all tcp md5sig password yyy } } #(default filter rules follow) I know announce all for EBGP connections is not default, but I've tried changing it to self, result was the same, however. It's set to all because I plan to add couple other AS to this router soon. What am I missing here? Thanks, Peter
Re: OpenBGPD prepend-self/neighbor question
What am I missing here? A) look at bgpd -nv output and check if the filter rules make sense. They look fine, only filter rules on core2b are affected and they look like this: match from 159.148.214.101 set { prepend-neighbor 3 } match to 159.148.214.101 set { prepend-self 3 } deny from any allow from any inet prefixlen 8 - 24 deny from any prefix 10.0.0.0/8 prefixlen = 8 deny from any prefix 172.16.0.0/12 prefixlen = 12 deny from any prefix 192.168.0.0/16 prefixlen = 16 deny from any prefix 169.254.0.0/16 prefixlen = 16 deny from any prefix 192.0.2.0/24 prefixlen = 24 deny from any prefix 224.0.0.0/4 prefixlen = 4 deny from any prefix 240.0.0.0/4 prefixlen = 4 B) use bgpctl show rib nei latnet out to see what prefixes you are actually sending to the other side. This is actually weird, primary router has only our network, but secondary has all networks, but I'm not sure if it should be like that: # core2a flags: * = Valid, = Selected, I = via IBGP, A = Announced origin: i = IGP, e = EGP, ? = Incomplete flags destination gateway lpref med aspath origin AI* 194.143.152.0/230.0.0.0100 0 i # core2b: flags: * = Valid, = Selected, I = via IBGP, A = Announced origin: i = IGP, e = EGP, ? = Incomplete flags destination gateway lpref med aspath origin I* 31.24.192.0/21 159.148.214.101100 0 21178 21178 21178 2588 42480 8194 i I* 31.170.16.0/21 159.148.214.101100 0 21178 21178 21178 2588 42480 5518 49191 i ... [skip] ... I* 194.143.152.0/23 159.148.214.98 100 0 21178 21178 21178 i ... [skip] ... I* 217.198.224.0/20 159.148.214.101100 0 21178 21178 21178 2588 42480 20910 i I* 217.199.96.0/19 159.148.214.101100 0 21178 21178 21178 2588 42480 20797 20797 20797 20797 i I should also note, that our upstream ISP doesn't give us full route table, only Latvian IP addresses. Full output from all commands can be seen here: http://mail2.drosiba.lv/bgpd/ Thank you, Peter
OpenBGP on CARP
Hello, I'm trying to implement CARP on our edge BGP OpenBSD routers. CARP itself seems to be working perfectly but I'm having problems figuring out how to propertly configure BGP. I have couple of external IP's on my network, so limited number of them is not an issue (as it's often with internet exchange points). From claudio's presentation it seems to me doing it by depend on carp0 is wrong [1]. My setup is like this: [ upstream ] 10.1.1.254 | 10.1.1.1 10.1.1.2 carp0 10.1.1.3 [ router#1 ] [ router#2 ] Should I peer both routers using their external IP's to my upstream ISP and keep IBGP session between both of them? Or should I use depend on carp0 and local-address 10.1.1.1 on both of them? In this case, what am I supposed to announce between both of my routers? Thanks, Peter 1. http://www.openbsd.org/papers/linuxtag06-network/mgp00028.html
Re: jails in openbsd
Hello, I'd like to look at some virtualization options for openbsd. The ultimate goal would be to get several isolated Debian systems running inside some kind of enironment for virtualization. Can you point me to an openbsd package, port or source code for the freebsd jail or an equivalent? You should probably go for sysjail: http://sysjail.bsd.lv/ p.
Re: ssh local aliases in known_hosts
i was wondering if it was possible to use known_hosts for some kind of local alias database. That's what ~/.ssh/config file is for, for example: -- Host box HostName 111.222.333.444 Host box2 HostName some.domain.name -- p.
Re: problem with spamd-white table and greylisting
# - # FILTER SETTINGS: set block-policy drop set skip on lo0 set loginterface $ext_if set limit table-entries 25 A guess: the table-entries limits the total table entries 183469+70604 is larger than 25. You're right, limiting table to 500k entries fixed the problem. Thanks a lot, p.
Re: problem with spamd-white table and greylisting
You forgot spamlogd I did not, it starts automatically when spamd_grey is enabled in Oh. Sorry. I didn's see it in your ps grep. I see it with the same ps grep on my system. dumbcat[root]# ps aux | grep spamd _spamd 14450 0.0 0.1 9040 1376 ?? IsTue12AM1:53.41 spamd: (pf spamd-white update) (spamd) _spamd 25485 0.0 0.8 8968 9592 ?? S Tue12AM0:50.54 /usr/libexec/spamd -g _spamd 10132 0.0 0.1 8968 1272 ?? I Tue12AM0:00.60 spamd: (/var/db/spamd update) (spamd) _spamd 14234 0.0 0.1 536 1216 ?? SsTue12AM0:09.53 /usr/libexec/spamlogd root 11990 0.0 0.0 1008 0 p1 R+ 7:36AM0:00.00 grep spamd (ksh) Now this is funny, as my spamlogd runs ar root. # ps aux | grep spam _spamd6455 ... 3:20PM 1:23.34 spamd: (pf spamd-white update) (spamd) _spamd 26548 ... 3:20PM 0:05.24 /usr/libexec/spamd -v -g _spamd 21456 ... 3:20PM 0:00.00 spamd: (/var/db/spamd update) (spamd) root 6596 ... 3:21PM 0:01.31 /usr/libexec/spamlogd This is OpenBSD 4.0-stable (GENERIC) with all 4.0 patches applied. I've just checked on working spamd setup (with greylisting enabled) on 3.9-stable box spamlogd also runs as root. $ cat /etc/rc | grep \$OpenBSD # $OpenBSD: rc,v 1.290 2006/09/01 01:18:02 mpf Exp $ $ cat /etc/rc.conf | grep \$OpenBSD # $OpenBSD: rc.conf,v 1.116 2006/06/02 21:32:50 mcbride Exp $ Though I'm pretty sure that spamlogd doesn't have anything to do with my my problems, whitelisting outgoing connections is next step to do. p.
Re: problem with spamd-white table and greylisting
What could be wrong? What have I been missing? probably a pf rules thing, (not logging port 25 connections, etc.) This is content of my pf.conf: # - # TABLES AND VARIABLES: ext_if=rl0 table spamd persist table spamd-white persist # - # FILTER SETTINGS: set block-policy drop set skip on lo0 set loginterface $ext_if set limit table-entries 25 # - # PACKET NORMALIZATION: scrub in on $ext_if inet all fragment reassemble scrub out on $ext_if inet all random-id # - # NAT/RDR RULES: # spamd: rdr inet proto tcp from spamd to any \ port 25 - 127.0.0.1 port 8025 rdr inet proto tcp from !spamd-white to any \ port 25 - 127.0.0.1 port 8025 # - # FILTER RULES: # default policy: block log all # ssh access: pass in on $ext_if inet proto tcp from any to $ext_if port 22 \ flags S/SA keep state # icmp echo and traceroute: pass in on $ext_if inet proto icmp from any to $ext_if \ icmp-type 8 keep state pass in on $ext_if inet proto udp from any to $ext_if \ port 33435:33524 keep state # allow outgoing traffic: pass out on $ext_if inet proto tcp all flags S/SA keep state pass out on $ext_if inet proto { udp, icmp } all keep state # services: # mail: pass in log on $ext_if inet proto tcp from any to \ $ext_if port 25 flags S/SA keep state pass out log on $ext_if inet proto tcp from $ext_if to \ any port 25 flags S/SA keep state pass in log on $ext_if inet proto tcp from any to \ 127.0.0.1 port 8025 flags S/SA keep state pass in on $ext_if inet proto tcp from any to \ $ext_if port { 465, 587, 110, 143, 993, 995 } flags S/SA keep state # antispoof: block in log on $ext_if inet from $ext_if to any -- turn on debug level syslogging in /etc/syslog.conf i.e. *.debug /var/log/debug and you should see spamlogd yapping in there for every inbound and outbound connection, as well as spamd talking about everything that hits it and everything that it whitelists. I see messages from spamlogd: -- Dec 6 20:10:57 sargon spamlogd[26673]: inbound 193.95.195.234 Dec 6 20:11:03 sargon spamlogd[26673]: inbound 85.74.161.17 Dec 6 20:11:09 sargon spamlogd[26673]: inbound 201.225.200.146 Dec 6 20:11:13 sargon spamlogd[26673]: inbound 65.31.67.90 Dec 6 20:11:15 sargon spamlogd[26673]: inbound 189.13.203.218 Dec 6 20:11:23 sargon spamlogd[26673]: inbound 196.206.22.130 -- (no outbound connections as clients can't connect to the SMTP at the moment) Or theese from spamd: -- Dec 6 20:12:00 sargon spamd[8277]: 89.37.50.15: connected (8/5) Dec 6 20:12:07 sargon spamd[8277]: 195.2.96.10: connected (9/5) Dec 6 20:12:08 sargon spamd[8277]: (GREY) 193.109.211.175: [EMAIL PROTECTED] - [EMAIL PROTECTED] Dec 6 20:12:08 sargon spamd[8277]: 193.109.211.175: disconnected after 11 seconds. Dec 6 20:12:08 sargon spamd[8277]: 193.109.211.175: connected (9/5) Dec 6 20:12:12 sargon spamd[8277]: (GREY) 89.37.50.15: [EMAIL PROTECTED] - [EMAIL PROTECTED] Dec 6 20:12:12 sargon spamd[8277]: 89.37.50.15: disconnected after 12 seconds. -- But that's it. I can see only inbound connections on pflog0 interface as clients are not able to connect to SMTP at this point: -- 20:18:37.430201 196.218.62.68.2125 127.0.0.1.8025: [|tcp] (DF) 20:18:48.992685 201.8.12.152.61364 127.0.0.1.8025: [|tcp] (DF) 20:18:49.933507 81.176.68.176.58866 127.0.0.1.8025: [|tcp] (DF) 20:18:49.960999 81.153.206.207.1946 127.0.0.1.8025: [|tcp] (DF) 20:18:59.710835 62.221.60.166.3082 127.0.0.1.8025: [|tcp] (DF) 20:19:01.826604 89.138.60.65.3639 127.0.0.1.8025: [|tcp] (DF) -- And there is tcpdump process listening to SMTP connections: -- _tcpdump 13159 0.0 0.2 664 1208 ?? S 7:51PM0:00.02 tcpdump -l -n -e -i pflog0 -q -t port 25 and action pass and tcp[13]0x12=0x2 root 107 0.0 0.2 668 836 ?? I 7:51PM0:00.01 tcpdump: [priv] (tcpdump) -- This is how it looks when I'm sending email from other mailserver that I maintain: -- Dec 6 20:13:58 vedjma postfix/smtp[27269]: 66C90A15B: host mail.drosiba.lv[193.109.211.134] said: 451 Temporary failure, please try again later. (in reply to RCPT TO command) -- Which is OK, but then again this host never gets whitelisted for real: -- (on recieving server) Dec 6 20:13:47 sargon spamd[8277]: 159.148.95.112: connected (13/5) Dec 6 20:13:58 sargon spamd[8277]: (GREY) 159.148.95.112: [EMAIL PROTECTED] - [EMAIL PROTECTED] Dec 6 20:13:58 sargon spamd[8277]: 159.148.95.112: disconnected after 11 seconds. Dec 6 20:15:37 sargon spamd[8277]: 159.148.95.112: connected (12/5) Dec 6 20:15:48 sargon spamd[8277]: (GREY) 159.148.95.112: [EMAIL PROTECTED] - [EMAIL PROTECTED] Dec 6 20:15:48
Re: problem with spamd-white table and greylisting
I don't see anything obvious, other than if you have somehow broken your system to pfctl can't add the addreses to the table. try pfctl -tspamd-white -Tadd 111.111.111.111 pfctl -tspamd-white -Tshow Do you see the address addeed to the table when you do it manually? I do: -- # pfctl -tspamd-white -Tadd 111.111.111.111 pfctl -tspamd-white -Tshow 1/1 addresses added. 111.111.111.111 # -- p.
problem with spamd-white table and greylisting
Hello, I'm having some weird problem with spamd in greylisting mode. The problem is that spamd-white table stays empty all the time. I've reread the manpages for a few times now and I have other spamd setup running on other mailserver (3.9 though) like a charm without any problems with pretty much the same pf.conf and spamd.conf. My configuration: # uname -a OpenBSD xxx.xxx.lv 4.0 GENERIC#1 i386 # cat /etc/rc.conf.local | grep spamd spamd_flags=-v spamd_grey=YES # ps aux | grep spamd _spamd 13571 ... 0:05.73 spamd: (pf spamd-white update) (spamd) _spamd 26998 ... 0:00.77 /usr/libexec/spamd -v -g _spamd 3728 ... 0:00.02 spamd: (/var/db/spamd update) (spamd) # pfctl -sT spamd spamd-white # spamdb | grep WHITE | wc -l 183469 # pfctl -t spamd-white -T show | wc -l 0 # pfctl -t spamd -T show | wc -l 70604 # cat /etc/pf.conf | grep table table spamd persist table spamd-white persist # pfctl -sn rdr inet proto tcp from spamd to any port = smtp - 127.0.0.1 port 8025 rdr inet proto tcp from ! spamd-white to any port = smtp - 127.0.0.1 port 8025 I can send all pf.conf or other config files upon demand, but it's pretty much the default. spamd.conf is the same as default except that I added Bob Beck's blaklist and it doesn't have to do anything with greylisting anyway. What could be wrong? What have I been missing? Thanks, p.
splitting one AS between two physical locations
Hello, We're a small ISP working on redesigning our network's topology and the overall architecture before replacing the existing infrastructure, which happens to be different variations of low cost router-boards running Linux. The ultimate goal thus being to run OpenBSD wherever we can. A job on top of our list would be to to migrate the core routers. Such solution would involve carp(4) and pfsync(4) -- hopefully making them fully redundant and a lot more solid network appliances than they currently are. But we're having a tough time figuring out how BGP should operate. In the existing scheme, an AS (autonomous system ) with 2 IP address blocks (193.x.x.0/24 and 194.x.x.0/23) is what holds our networks together. Our two optical lines, namely [upstream0] and [upstream1] are separated geographically (as illustrated in Fig. 1), themselves being interconnected by a wireless link. [upstream0][upstream1] | | | | [core0]-(wireless)-[core1] |193.x.x.0/24 |194.x.x.0/23 | | [clients] [clients] Fig. 1 Question; Is it possible to make two BGP neighbors (one at core0 and second at core1) by using only one AS? We could get a second AS, but as we expect to have more networks around we would really like to set it up by only using one AS, not bothering ripe.net every time we expand the network. And yes, we plan to use OpenBGP. Thanks, Peter