Re: Use an specific ADSL depending on IP
Ok, let's go again. According to Karl (thanks ;)) words I've rebuilded
my pf.conf for to times in two diferrent approaches.
// FIRST TRY
- make the ADSL redirection in bge0 (the internal one, traffic from
LAN to Internet)
- make the queues in bge1/re0/fxp0 (the externals ones, traffic from
Internet to LAN)
##
# --- (bge0/LAN) OpenBSD box (fxp0/adsl1)
# (bge1/adsl2)
# (re0/adsl3)
#
# 1.Macros
# ISP_1
ext_if1=fxp0
ext_gw1=217.126.43.2
# ISP_2
ext_if2=bge1
ext_gw2=192.168.10.1
# ISP_3
ext_if3=re0
ext_gw3=192.168.2.1
# dept. A
workmate_a=192.168.0.121
workmate_b=192.168.0.51
workmate_c=192.168.0.124
workmate_d=192.168.0.49
dept_a = { $workmate_a $workmate_b $workmate_c $workmate_d }
#dept. B
workmate_e=192.168.0.79
wormate_f=192.168.0.78
dept_b = { $workmate_e $workmate_f}
# dept. C
workmate_h=192.168.0.39
workmate_i=192.168.0.43
workmate_j=192.168.0.48
dept_c = { $workmate_h $workmate_i $workmate_j }
# internal gateway
lan_net=192.168.0.0/24
int_if=bge0
int_gw=192.168.0.1
# another macros
cpd={ xxx }
callcenter=xxx
# 2.Tables
# Not used at present
# 3.Options
set skip on lo
set optimization conservative
set limit states 5
# 4. Scrub traffic
scrub all
# 5. Queueing
altq on $int_if cbq bandwidth 30Mb queue { zona1, zona2, zona3 }
queue zona1 bandwidth 10Mb { centraeta, cpd1, ssh1, nocpd1 }
queue centraleta bandwidth 50% priority 7 cbq(default)
queue cpd1 bandwidth 25% priority 6 cbq(borrow)
queue ssh1 bandwidth 5% priority 3 cbq
queue nocpd1 bandwidth 20% priority 0 cbq
queue zona2 bandwidth 10Mb { cpd2, ssh2, nocpd2 }
queue cpd2 bandwidth 40% priority 7 cbq
queue ssh2 bandwidth 40% priority 3 cbq(borrow)
queue nocpd2 bandwidth 20% priority 0 cbq
queue zona3 bandwidth 10Mb { cpd3, ssh3, nocpd3 }
queue cpd3 bandwidth 60% priority 7 cbq
queue ssh3 bandwidth 25% priority 3 cbq(borrow)
queue nocpd3 bandwidth 15% priority 0 cbq
# 6. Translation
nat on $ext_if1 from $lan_net to any - ($ext_if1)
nat on $ext_if2 from $lan_net to any - ($ext_if2)
nat on $ext_if3 from $lan_net to any - ($ext_if3)
# 7. Filer rules
### traffic from LAN to INTERNET
## from LAN to Internet: INBOUND to bge0 ##
# dept_a using their own ADSL
pass out on $int_if route-to \
($ext_if1 $ext_gw1) \
proto { tcp udp } from $dept_a to any keep state
# dept_b using their own ADSL
pass out on $int_if route-to \
($ext_if2 $ext_gw2) \
proto { tcp udp } from $dept_b to any keep state
# dept_c using their own ADSL
pass out on $int_if route-to \
($ext_if3 $ext_gw3) \
proto { tcp udp } from $dept_c to any keep state
## from LAN to internet: OUTBOUND on bge1/re0/fxp0
pass out on $ext_if1
pass out on $ext_if2
pass out on $ext_if3
###
### traffic from INTERNET to LAN
## from Internet to LAN: INBOUND to bge1/re0/fxp0 ##
pass in on $ext_if1
pass in on $ext_if2
pass in on $ext_if3
## from Internet to LAN: OUTBOUND to bge0 ##
### dept_a
pass out quick on $int_if from $callcenter02 \
to $dept_a keep state \
queue centraleta
pass out quick on $int_if proto tcp from $cpd \
to $dept_a port 22 keep state \
queue ssh1
pass out quick on $int_if from $cpd \
to $dept_a keep state \
queue cpd11
pass out on $int_if from any \
to $dept_a keep state \
queue nocpd1
### dept_b
pass out quick on $int_if proto tcp from $cpd \
to $dept_b port 22 keep state \
queue ssh2
pass out quick on $int_if from $cpd \
to $dept_b keep state \
queue cpd2
pass out on $int_if from any \
to $dept_b keep state \
queue nocpd2
### dept_c
pass out quick on $int_if proto tcp from $cpd \
to $dept_c port 22 keep state \
queue ssh3
pass out quick on $int_if from $cpd \
to $dept_a keep state \
queue cpd3
pass out on $int_if from any \
to $dept_c keep state \
queue nocpd3
-
It's clean, it's understable... but it doesn't work. Indeed works the
first part: every LAN client uses the correct ADSL out, but queues
doesn't run.
// SECOND TRY
- make the ADSL redirection and also the queues works in bge0 (the
internal one, traffic from LAN to Internet)
##
# --- (bge0/LAN) OpenBSD box (fxp0/adsl1)
# (bge1/adsl2)
# (re0/adsl3)
#
# 1.Macros
# ISP_1
ext_if1=fxp0
ext_gw1=217.126.43.2
# ISP_2
ext_if2=bge1
ext_gw2=192.168.10.1
# ISP_3
ext_if3=re0
ext_gw3=192.168.2.1
# dept. A
workmate_a=192.168.0.121
workmate_b=192.168.0.51
workmate_c=192.168.0.124
workmate_d=192.168.0.49
dept_a = { $workmate_a $workmate_b $workmate_c $workmate_d }
#dept. B
workmate_e=192.168.0.79
wormate_f=192.168.0.78
Re: Use an specific ADSL depending on IP
I'm not paying much attention to the rest of your rules, but note that traffic going out the internal interface is coming from the Internet and so is _inbound_ traffic not outbound traffic as the comment would indicate. (You have other inbound quick rules in your ruleset so you can't just change out to in here and expect it to work.) Ok Karl, thanks. I think I've a problem of missconception. So, I understand that this schema Internet ---bge1 --- bge0 --- LAN means at least 4 traffic to bge0 ruleset point of view: 1- Traffic from internet (coming from bge1): it's IN 2- Traffic 1 to LAN: it's OUT 3- Traffic from LAN to bge0: it's IN 4- Traffic from bge0 to bge1: it's OUT ¿Am I right? -- I must not fear. Fear is the mind-killer. Fear is the little-death that brings total obliteration. I will face my fear. I will permit it to pass over me and through me. And when it has gone past I will turn the inner eye to see its path. Where the fear has gone there will be nothing. Only I will remain. Bene Gesserit Litany Against Fear.
Use an specific ADSL depending on IP
Hi all,
I've a OpenBSD box with 4 NICs: 3 with ADSLs connection and the one last
for the LAN.
--- (bge0/LAN) OpenBSD box (fxp0/adsl1)
(bge1/adsl2)
(re0/adsl3)
The actual pf.conf looks like:
##
# 1.Macros
# ISP_1
ext_if1=fxp0
ext_gw1=217.126.43.2
# ISP_2
ext_if2=bge1
ext_gw2=192.168.10.1
# ISP_3
ext_if3=re0
ext_gw3=192.168.2.1
# dept. A
workmate_a=192.168.0.121
workmate_b=192.168.0.51
workmate_c=192.168.0.124
workmate_d=192.168.0.49
dept_a = { $workmate_a $workmate_b $workmate_c $workmate_d }
#dept. B
workmate_e=192.168.0.79
wormate_f=192.168.0.78
dept_b = { $workmate_e $workmate_f}
# dept. C
workmate_h=192.168.0.39
workmate_i=192.168.0.43
workmate_j=192.168.0.48
dept_c = { $workmate_h $workmate_i $workmate_j }
# internal gateway
lan_net=192.168.0.0/24
int_if=bge0
int_gw=192.168.0.1
# another macros
cpd={ xxx }
callcenter=xxx
# 2.Tables
# Not used at present
# 3.Options
set skip on lo
set optimization conservative
set limit states 5
# 4. Scrub traffic
scrub all
# 5. Queueing
altq on $int_if cbq bandwidth 30Mb queue { zona1, zona2, zona3 }
queue zona1 bandwidth 10Mb { centraeta, cpd1, ssh1, nocpd1 }
queue centraleta bandwidth 50% priority 7 cbq(default)
queue cpd1 bandwidth 25% priority 6 cbq(borrow)
queue ssh1 bandwidth 5% priority 3 cbq
queue nocpd1 bandwidth 20% priority 0 cbq
queue zona2 bandwidth 10Mb { cpd2, ssh2, nocpd2 }
queue cpd2 bandwidth 40% priority 7 cbq
queue ssh2 bandwidth 40% priority 3 cbq(borrow)
queue nocpd2 bandwidth 20% priority 0 cbq
queue zona3 bandwidth 10Mb { cpd3, ssh3, nocpd3 }
queue cpd3 bandwidth 60% priority 7 cbq
queue ssh3 bandwidth 25% priority 3 cbq(borrow)
queue nocpd3 bandwidth 15% priority 0 cbq
# 6. Translation
nat on $ext_if1 from $lan_net to any - ($ext_if1)
nat on $ext_if2 from $lan_net to any - ($ext_if2)
nat on $ext_if3 from $lan_net to any - ($ext_if3)
# 7. Filer rules
# pass all outgoing packets on internal interface
pass out on $int_if to $lan_net
### dept_a
pass in quick on $int_if from $dept_a \
to $callcenter02 keep state \
queue centraleta
pass in quick on $int_if proto tcp from $dept_a \
to $cpd port 22 keep state \
queue ssh1
pass in quick on $int_if from $dept_a \
to $cpd keep state \
queue cpd11
pass in on $int_if from $dept_a \
to any keep state \
queue nocpd1
### dept_b
pass in quick on $int_if proto tcp from $dept_b \
to $cpd port 22 keep state \
queue ssh2
pass in quick on $int_if from $dept_b \
to $cpd keep state \
queue cpd2
pass in on $int_if from $dept_b \
to any keep state \
queue nocpd2
### dept_c
pass in quick on $int_if proto tcp from $dept_b \
to $cpd port 22 keep state \
queue ssh2
pass in quick on $int_if from $dept_b \
to $cpd keep state \
queue cpd2
pass in on $int_if from $dept_b \
to any keep state \
queue nocpd2
# general pass out rules for external interfaces
pass out on $ext_if1
pass out on $ext_if2
pass out on $ext_if3
# dept_a using their own ADSL to outbound
pass out on $int_if route-to \
($ext_if1 $ext_gw1) \
proto { tcp udp } from $dept_a to any keep state
# dept_b using their own ADSL to outbound
pass out on $int_if route-to \
($ext_if2 $ext_gw2) \
proto { tcp udp } from $dept_b to any keep state
# dept_c using their own ADSL to outbound
pass out on $int_if route-to \
($ext_if3 $ext_gw3) \
proto { tcp udp } from $dept_c to any keep state
# route packets from any IPs on $ext_if1 to $ext_gw1 and the same for
# $ext_if2 and $ext_gw2
pass out on $ext_if1 route-to ($ext_if1 $ext_gw1) from $ext_if1
pass out on $ext_if2 route-to ($ext_if2 $ext_gw2) from $ext_if2
pass out on $ext_if3 route-to ($ext_if3 $ext_gw3) from $ext_if3
As you can see, I've two goals in this business:
1) Control de bandwith in download. It works correctly ;)
2) Use an specific ADSL depending on IP. Doesn't work.
If I check (with a simple service as www.whatismyip.com) a http
connection from dept_a, dept_b or dept_c I see the same IP all the time.
So, the clients always go out using the same ADSL (the first one
NIC/connection indeed).
¿Anyone can point me out where the problem is?
--
I must not fear. Fear is the mind-killer. Fear is the little-death that
brings total obliteration. I will face my fear. I will permit it to pass
over me and through me. And when it has gone past I will turn the inner
eye to see its path. Where the fear has gone there will be nothing. Only
I will remain.
Bene Gesserit Litany Against Fear.
Re: Brutes rules with UDP?
quite clear isn't it? Yes, is it. the tcp one works based on completed 3way handshakes. now think about it. Yep. -- I must not fear. Fear is the mind-killer. Fear is the little-death that brings total obliteration. I will face my fear. I will permit it to pass over me and through me. And when it has gone past I will turn the inner eye to see its path. Where the fear has gone there will be nothing. Only I will remain. Bene Gesserit Litany Against Fear.
Brutes rules with UDP?
Hi all, I use the next rule: # SSH brutes protection pass quick on $bridge inet proto tcp from any to $vlan10 port 22 keep state \ (max-src-conn 20, max-src-conn-rate 3/12, \ overload ssh_brutes flush global) with success. No problem, all works fine. I wonder if I can apply this type of rule to UDP connections (I try to protect some busy DNS servers) -- I must not fear. Fear is the mind-killer. Fear is the little-death that brings total obliteration. I will face my fear. I will permit it to pass over me and through me. And when it has gone past I will turn the inner eye to see its path. Where the fear has gone there will be nothing. Only I will remain. Bene Gesserit Litany Against Fear.
Re: Brutes rules with UDP?
# SSH brutes protection pass quick on $bridge inet proto tcp from any to $vlan10 port 22 keep state \ (max-src-conn 20, max-src-conn-rate 3/12, \ overload ssh_brutes flush global) with success. No problem, all works fine. I wonder if I can apply this type of rule to UDP connections (I try to protect some busy DNS servers) no, there's no way to avoid spoofed requests with UDP. if someone sends a bunch of UDP packets spoofed from $BIG_ISP_RESOLVER's IP address, their legitimate requests will be blocked. I don't understand your response, Stuart. I wonder if the mentioned rule (using max-src-conn and max-src-rate) is also applicable to UDP-oriented connections as DNS is. So, pass quick on $bridge inet proto udp from any to $vlan10 port 53 keep state \ (max-src-conn 30, max-src-conn-rate 2/1, \ overload dns_brutes flush global) ¿it will be work? -- I must not fear. Fear is the mind-killer. Fear is the little-death that brings total obliteration. I will face my fear. I will permit it to pass over me and through me. And when it has gone past I will turn the inner eye to see its path. Where the fear has gone there will be nothing. Only I will remain. Bene Gesserit Litany Against Fear.
A lot of pf: loose state match: TCP
is the problem? -- Thanks, Jordi Espasa Clofent
Re: Failover bridge(4) with RSTP
Jason Dixon escribió: I'm attempting to setup a failover bridge(4) configuration with RSTP for rapid failover. At this point I'm still tweaking the bridges and switches. We're using a Foundry LS648 for this test, so we don't have Cisco's unplinkFast extension at our disposal. We have two VLANs configured on the switch, each with 802.1w enabled and functioning normally. Plugged into each VLAN is a single client and one interface from each firewall. 10.20.0.2 - vlan200 - bridge0 - vlan300 - 10.20.0.3 Regardless of whether I use rstp (default) or stp (+ ifpriority/ifcost) on the bridges, it always takes ~5 minutes to failover. I noticed that with stp enabled on the physical interfaces, the switch would immediately show the correct bridge as the forwarding root. With the default rstp, the switch shows all ports as designated forwarding. I've also tried disabling learning on the internal interfaces and adding static entries for 10.20.0.3, but this has no effect on the recovery time. Any suggestions on getting a rapid failover working? I've got a pair of OpenBSD 4.2 boxes using RSTP perfectly with two DlinK common switches. They've been working fine during last year and the redundancy is completely checked and reliable using RSTP. Maybe the problem is the VLAN; I've not worked with VLAN in this scenario and indeed I've suffering some VLAN/pf/bridge related-troubles . The picture is sw01 --- fw1 sw02 sw01 --- fw2 sw02 and config is $ cat /etc/hostname.em0 up [r...@ares] [~] [18:19:21] $ cat /etc/hostname.em1 up [r...@ares] [~] [18:19:23] $ cat /etc/hostname.em2 inet 192.168.1.253 255.255.0.0 NONE [r...@ares] [~] [18:19:25] $ cat /etc/hostname.em3 inet 10.10.10.1 255.255.255.0 NONE [r...@ares] [~] [18:19:27] $ cat /etc/bridgename.bridge0 add em1 add em0 -learn em1 -learn em0 stp em1 stp em0 up [r...@ares] [~] [18:19:37] $ cat /etc/hostname.pfsync0 up syncdev em3 In both Dlink switches I've enabled RSTP support and simple make the lower cost for fw01 path and more cost for fw02. Even I have a simple last-resort wire with the higer RSTP cost. -- Thanks, Jordi Espasa Clofent
Single VLAN through bridge
: flags=41UP,RUNNING mtu 1500 groups: bridge -- Thanks, Jordi Espasa Clofent
Re: RDR rules based on outcoming traffic type?
Ok. Marcin point me out to http://openbsd.org/faq/pf/pools.html and make
me aware that I mean route-to instead rdr.
The present ruleset is:
# 1.Macros
ext_if1=em0
ext_gw1=xxx.xxx.xxx
ext_if2=em1
ext_gw2=xxx.xxx.xxx
ext_if3=em2
ext_gw3=xxx.xxx.xxx
lan_net=192.168.0.0/24
int_if=em3
# 2.Tables
# Not used at present
# 3.Options
set skip on lo
# 4. Scrub traffic
scrub all
# 5. Queueing
# Not used at present
# 6. Translation
nat on $ext_if1 from $lan_net to any - ($ext_if1)
nat on $ext_if2 from $lan_net to any - ($ext_if2)
nat on $ext_if3 from $lan_net to any - ($ext_if3)
# 7. Filer rules
# pass all outgoing packets on internal interface
pass out on $int_if from any to $lan_net
# pass in quick any packets destined for the gateway itself
pass in quick on $int_if from $lan_net to $int_if
# use concrete DSL uplink for SSH traffic
pass in on $int_if route-to \
($ext_if1 $ext_gw1) \
proto tcp from $lan_net to any port 22 flags S/SA modulate state
# use concrete DSL uplink for VoIP traffic
pass in quick on $int_if route-to \
($ext_if2 $ext_gw2) \
proto tcp from $lan_net to any port 5060 flags S/SA modulate state
pass in quick on $int_if route-to \
($ext_if2 $ext_gw2) \
proto udp from $lan_net to any port 1:2000
# use concrete DSL uplink for www traffic
pass in on $int_if route-to \
($ext_if3 $ext_gw3) \
proto tcp from $lan_net to any port 80 flags S/SA modulate state
# load balance outgoing tcp traffic from internal network.
pass in on $int_if route-to \
{ ($ext_if1 $ext_gw1), ($ext_if2 $ext_gw2) } round-robin \
proto tcp from $lan_net to any flags S/SA modulate state
# load balance outgoing udp and icmp traffic from internal network
pass in on $int_if route-to \
{ ($ext_if1 $ext_gw1), ($ext_if2 $ext_gw2) } round-robin \
proto { udp, icmp } from $lan_net to any keep state
# general pass out rules for external interfaces
pass out on $ext_if1 proto tcp from any to any flags S/SA modulate state
pass out on $ext_if1 proto { udp, icmp } from any to any keep state
pass out on $ext_if2 proto tcp from any to any flags S/SA modulate state
pass out on $ext_if2 proto { udp, icmp } from any to any keep state
pass out on $ext_if3 proto tcp from any to any flags S/SA modulate state
pass out on $ext_if3 proto { udp, icmp } from any to any keep state
It seems correct:
$ pfctl -nvf pf.test before
$ pfctl -o basic -nvf pf.test after
$ diff before after
¿Can I apply queues on it?
--
Thanks,
Jordi Espasa Clofent
RDR rules based on outcoming traffic type?
Hi all, I've the next topology: [LAN] [OpenBSD box) DSL1 [Internet) DSL2 [Internet) DSL3 [Internet) DSL1 is a line intended for VoIP communications. DSL2 is a line intended for www related comunications. DSL3 is a line intended only for ssh comunications. My idea is: * the LAN users always useo the same gateway IP. Only the same gateway for all of them. * according the required protocol, use one or another rdr rule to use DSL1, 2 or 3. For example, Alice has 192.168.0.100 IP and always uses 192.168.0.1 as default gateway. If she wants to use VoIP, her traffic should goes out through DSL1; if she uses her browser, the generated traffic should goes out across DSL2 and, finally, if she wants to use ssh, the traffic should goes out via DSL3. ¿Is it possible? Moreover, if it is possible ¿can I apply queues in redirected traffic? -- Thanks, Jordi Espasa Clofent
ftpsesame 'drop: short capture'
Hi all,
Because of my previous problem I've seen the next output in
/var/log/messages:
-- Sep 22 19:00:01 ares newsyslog[21422]: logfile turned over
Sep 22 19:00:01 ares syslogd: restart
Sep 22 19:00:38 ares ftpsesame[15600]: drop: short capture
Sep 22 19:01:04 ares last message repeated 3 times
Sep 22 19:04:36 ares ftpsesame[15600]: drop: short capture
Sep 22 19:04:47 ares last message repeated 11 times
Sep 22 19:17:07 ares last message repeated 4 times
Sep 22 20:00:01 ares syslogd: restart
Sep 22 20:31:27 ares ftpsesame[15600]: drop: short capture
Sep 22 21:31:43 ares ftpsesame[15600]: drop: short capture
Sep 22 21:31:47 ares last message repeated 3 times
Sep 22 21:34:32 ares last message repeated 6 times
Sep 22 21:53:06 ares last message repeated 8 times
Sep 22 22:00:01 ares syslogd: restart
Sep 22 22:01:34 ares ftpsesame[15600]: drop: short capture
Sep 22 22:02:28 ares ftpsesame[15600]: drop: short capture
Sep 22 22:09:45 ares ftpsesame[15600]: drop: short capture
Sep 22 22:10:00 ares last message repeated 62 times
Sep 23 00:00:01 ares syslogd: restart
Sep 23 00:08:46 ares ftpsesame[15600]: drop: short capture
Sep 23 00:09:02 ares last message repeated 3 times
Sep 23 00:09:28 ares last message repeated 2 times
Sep 23 00:33:06 ares ftpsesame[15600]: drop: short capture
Sep 23 00:34:02 ares last message repeated 4 times
Sep 23 00:47:01 ares ftpsesame[15600]: drop: short capture
Sep 23 01:30:54 ares ftpsesame[15600]: drop: short capture
Sep 23 01:38:49 ares last message repeated 20 times
Sep 23 01:47:12 ares last message repeated 2 times
Sep 23 01:56:28 ares last message repeated 6 times
Sep 23 02:00:01 ares syslogd: restart
Sep 23 02:06:11 ares ftpsesame[15600]: drop: short capture
Sep 23 02:06:20 ares ftpsesame[15600]: drop: short capture
Sep 23 02:08:15 ares last message repeated 4 times
Sep 23 04:00:01 ares syslogd: restart
Sep 23 04:10:20 ares ftpsesame[15600]: drop: short capture
Sep 23 04:10:35 ares last message repeated 2 times
Sep 23 04:12:33 ares last message repeated 3 times
Sep 23 04:13:43 ares ftpsesame[15600]: drop: short capture
Sep 23 04:32:26 ares last message repeated 5 times
Sep 23 04:35:09 ares last message repeated 2 times
Sep 23 06:00:01 ares syslogd: restart
Sep 23 08:00:01 ares syslogd: restart
Sep 23 08:18:10 ares ftpsesame[15600]: drop: short capture
Sep 23 08:39:54 ares ftpsesame[15600]: drop: short capture
Sep 23 08:40:10 ares ftpsesame[15600]: drop: short capture
Sep 23 08:54:00 ares ftpsesame[15600]: drop: short capture
Sep 23 09:07:24 ares ftpsesame[15600]: drop: short capture
Sep 23 09:15:40 ares last message repeated 10 times
Sep 23 09:29:56 ares ftpsesame[15600]: drop: short capture
So, I've donwload the ftpsesame sources and search the concrete string:
$ grep -ir short capture *
ftpsesame.c:logmsg(LOG_WARNING, drop: short capture);
$ vim ftpsesame.c
[...]
if (h-caplen != h-len) {
logmsg(LOG_WARNING, drop: short capture);
return;
[...]
I understand that it's a basic check about the length of the processed
packet. Anyway, I'm not a coder... only a learner.
¿It's really a problem or a simply warning about malformed packets?
--
Thanks,
Jordi Espasa Clofent
Rare problem with HTTPS
Hi all,
The problem:
- I've a web application which uses HTTPS connection. Sometimes, without
any pattern, the connection hangs up.
What I've seen:
- No recognisable pattern. I've made a litlle PHP program that simply
open and next close a HTTPS connextion. Sometimes it hangs up in 10th
try, sometimes in 150th and sometimes in 38. No reason here.
- If I disable completely the pf ('pf -d) the program runs fine until I
stop it (I've seen 2500 consecutive ok connections...)
My corrective actions:
- Check the permissive rule from any to the server that has the app:
alllow quick on $bridge from any to $web_server allow-opts
- Disabling completely the scrubbing options
no scrub in on $bridge
- Pass from 'set optimization aggressive to 'set optimization conservative'
- launch the sampe tcpdump command in PF box and in web-app box. When a
connection hungs up I see that some packets doesn't arrive to web-app
box; PF doesn't pass them for unknown reasons.
¿How can I debug it?
¿Any clue?
--
Thanks,
Jordi Espasa Clofent
Re: Rare problem with HTTPS
Ok; more info: $ pfctl -x misc $ tail -f /var/log/messages | grep 217.130.13.161 Sep 22 16:11:13 ares /bsd: pf: BAD state: TCP 212.36.74.109:443 212.36.74.109:443 217.130.13.161:32796 [lo=4134584134 high=4134650337 win=325 modulator=0 wscale=7] [lo=3328738864 high=3328780464 win=33120 modulator=0 wscale=1] 10:10 S seq=4159168565 (4159168565) ack=3328738864 len=0 ackskew=0 pkts=37:16 dir=in,fwd Sep 22 16:11:14 ares /bsd: pf: BAD state: TCP 212.36.74.109:443 212.36.74.109:443 217.130.13.161:32771 [lo=4155258378 high=4155324581 win=147 modulator=0 wscale=7] [lo=1301018309 high=1301037125 win=33120 modulator=0 wscale=1] 10:10 S seq=4161601276 (4161601276) ack=1301018309 len=0 ackskew=0 pkts=33:15 dir=in,fwd Sep 22 16:11:17 ares /bsd: pf: BAD state: TCP 212.36.74.109:443 212.36.74.109:443 217.130.13.161:32771 [lo=4155258378 high=4155324581 win=147 modulator=0 wscale=7] [lo=1301018309 high=1301037125 win=33120 modulator=0 wscale=1] 10:10 S seq=4161601276 (4161601276) ack=1301018309 len=0 ackskew=0 pkts=33:15 dir=in,fwd IP 217.130.13.161 is from I execute the test program and this is the output I can see EXACTLY when the program hangs up. ¿What it means? -- Thanks, Jordi Espasa Clofent
Re: Rare problem with HTTPS
Stuart Henderson escribió:
You may be recycling port numbers before the state fully expired.
If that's the case you can try reducing the tcp.closed timeout:
keep state (tcp.closed XX).
I've changed
pass quick on $bridge inet proto tcp from any to $mynetwork port { 80,
443 } keep state \
(max-src-conn 800, max-src-conn-rate 100/1, \
overload http_brutes flush global) allow-opts
to
pass quick on $bridge inet proto tcp from any to $mynetwork port { 80,
443 } keep state \
(tcp.closed 15,max-src-conn 800, max-src-conn-rate 100/1, \
overload http_brutes flush global) allow-opts
but I get the same error:
$ tail -f /var/log/messages | grep 217.130.13.161
Sep 22 17:01:27 ares /bsd: pf: BAD state: TCP 217.130.13.161:32798
217.130.13.161:32798 212.36.74.109:443 [lo=971805947 high=971872187
win=96 modulator=0 wscale=7] [lo=2426939406 high=2426951694 win=33120
modulator=0 wscale=1] 4:4 S seq=3055995244 (3055995244) ack=2426939406
len=0 ackskew=0 pkts=16:7 dir=out,fwd
Sep 22 17:01:30 ares /bsd: pf: BAD state: TCP 217.130.13.161:32798
217.130.13.161:32798 212.36.74.109:443 [lo=971805947 high=971872187
win=96 modulator=0 wscale=7] [lo=2426939406 high=2426951694 win=33120
modulator=0 wscale=1] 4:4 S seq=3055995244 (3055995244) ack=2426939406
len=0 ackskew=0 pkts=16:7 dir=out,fwd
Sep 22 17:01:36 ares /bsd: pf: BAD state: TCP 217.130.13.161:32798
217.130.13.161:32798 212.36.74.109:443 [lo=971805947 high=971872187
win=96 modulator=0 wscale=7] [lo=2426939406 high=2426951694 win=33120
modulator=0 wscale=1] 4:4 S seq=3055995244 (3055995244) ack=2426939406
len=0 ackskew=0 pkts=16:7 dir=out,fwd
--
Thanks,
Jordi Espasa Clofent
Re: Rare problem with HTTPS [SOLVED]
Finally, the solution has been the next rule:
pass quick on $bridge inet proto {tcp, udp } from any to
$webserver \ port { 80, 443 } keep state (tcp.finwait 3, tcp.closed
5) \
allow-opts
I tend to think that the real guilty are the routers with Window$ TCP/IP
based stack from this particular ISP (the rare behavior only happens
with one particular ISP connections clients)
Two conclusion about it:
- What amount of shit devices there are out there in common ISPs! It's
the second case I suffer these kinds of problems because of the poor
TCP/IP implementation of the self-called corporate routers.
- What powerful PF is!
--
Thanks,
Jordi Espasa Clofent
Filtering on bridge
Hi all,
I've a pf-based bridged box. The pf.conf file look like:
# 1. Macros
ext_if=em1
int_if=em0
bridge={ $ext_if $int_if }
admin_if=em2
pfsync_if=em3
my_net=xxx.xxx.xxx.xxx/23
# 2. Tables
table unrestricted persist file /etc/pf_files/unrestricted.pf
# brutes tables
table ssh_brutes persist
table mysql_brutes persist
table http_brutes persist
table smtp_brutes persist
# 3. Options
set skip on lo
set skip on em0
set skip on em2
set skip on em3
set fingerprints /etc/pf.os
set block-policy drop
set optimization aggressive
set limit states 70
set loginterface em1
# 4. Scrub traffic
scrub in all
# 5. QUEUEING
# Not used.
# 6. TRANSLATION
# Not used.
# 7. FILTER RULES
# DEFAULT POLICY
block in on $ext_if
# antispoof
antispoof quick for lo
# allow ping/tracert tools
pass in inet proto icmp from any to any
# permit all outbound traffic
pass out quick
# turning away the brutes
block quick from ssh_brutes
block quick from mysql_brutes
block quick from http_brutes
block quick from smtp_brutes
# allow housing without restrictions
pass quick on $bridge inet proto { tcp, udp, icmp } from any to
unrestricted allow-opts
# www with brute control method
pass quick on $bridge inet proto tcp from any to $my_net port { 80, 443
} keep state \
(max-src-conn 650, max-src-conn-rate 80/1, \
overload http_brutes flush global)
# DNS
pass quick on $bridge inet proto { tcp, udp } from any to $my_net port 53
# smtp with brutes
pass quick on $bridge inet proto tcp from any to $my_net port { 25, 578}
keep state \
(max-src-conn 250, max-src-conn-rate 50/1, \
overload smtp_brutes flush global)
# pop3, pop3s, imap4, imap4s
pass quick on $bridge inet proto tcp from any to $my_net port { 110,
143, 993, 995 }
[ other similar rules ...]
As you can see, I always use 'pass quick on $bridge .', but you can
also note that the bridge is formed by $ext_if (em1, externa NIC) and
$int_if (em0, internal NIC) and I DON'T filter anything in em0 (option
'set skip on em0').
So, I think it will be better using 'pass quick on $ext_if';
¿Am I wright?
--
Thanks,
Jordi Espasa Clofent
ftpsesame questions
Hi all, The scenario: bridged-based PF box with ftpsesame. OpenBSD 4.2, production environment. A lot of of FTP concurrents sessions. 1. The last goal is make possible active and passive FTP client connections AND do it with best performance (using symon I see that the ftpsesame processes are slowly sometimes). At present moment, I use the next rules: # FTP passive anchor ftpsesame/* in on $bridge proto tcp from any to ftp_servers anchor ftpsesame/* out on $bridge proto tcp from any to ftp_servers # FTP active anchor ftpsesame/* in on $bridge proto tcp from ftp_servers to any anchor ftpsesame/* out on $bridge proto tcp from ftp_servers to any pass quick on $bridge inet proto tcp from any to ftp_servers port 21 I don't want to control any outbound connection (indeed I've a nice 'pass quick all' rule), so... ¿are these rules the best in relation to performance issues? 2. ftpsesame works fine, great app. I see it's a 0.95 version... and this version was made for OpenBSD 3.6. I suppose the program has not changed because the anchors treatment is the same in 3.6 as 4.2/4.3. ¿Am I right? Currently I use ftpsesame in production environment, so will be very unpleasant to upgrade from 4.2 to 4.3 and discover that ftpsesame not works... ¿Is ftpsesame actively developed/supported nowadays?
Re: ftpsesame questions
It uses a lot of CPU you mean? Do you have a lot of activity on port 21? (lots of small transfers maybe?). Yes, it means a lot of CPU. Indeed I've seen a 90% of 'interrupt' according to symon graphs. Note that ftpsesame doesn't have anything to do with the actual FTP data transfers, it just takes care to insert PF rules that allows those, after that is all kernel. Ok. At present moment, I use the next rules: # FTP passive anchor ftpsesame/* in on $bridge proto tcp from any to ftp_servers anchor ftpsesame/* out on $bridge proto tcp from any to ftp_servers # FTP active anchor ftpsesame/* in on $bridge proto tcp from ftp_servers to any anchor ftpsesame/* out on $bridge proto tcp from ftp_servers to any pass quick on $bridge inet proto tcp from any to ftp_servers port 21 I don't want to control any outbound connection (indeed I've a nice 'pass quick all' rule), so... ¿are these rules the best in relation to performance issues? That's all fine. Ok. Maybe put the 'pass...' rule in top will be increase the performance...¿?¿?¿ pass quick on $bridge inet proto tcp from any to ftp_servers port 21 anchor ftpsesame/* in on $bridge proto tcp from any to ftp_servers anchor ftpsesame/* out on $bridge proto tcp from any to ftp_servers anchor ftpsesame/* in on $bridge proto tcp from ftp_servers to any anchor ftpsesame/* out on $bridge proto tcp from ftp_servers to any Should work, if it doesn't contact me. :-) I can vouch for OpenBSD 4.0. Well, also un 4.1 and 4.2. So, it should be also in 4.3. Great. ¿Is ftpsesame actively developed/supported nowadays? Sure, but it has not been needed the last 3 years... Great to hear. It means that ftpsesame is rock solid and their code and design is reliable. One more time, many thanks Camiel.
Re: match value question
¿anybody knows? -- Thanks, Jordi Espasa Clofent
Allow nmap
} allow-opts But no one of these seems to works as I expect. On the other hand, If I disable de pf, the nmap scans ouput always are: Nmap finished: 1 IP address (1 host up) scanned in 1.019 seconds [EMAIL PROTECTED] [/usr/local/home/jespasac] [19:46] # nmap -p 143 xxx.xxx.xxx.xxx Starting Nmap 4.20 ( http://insecure.org ) at 2008-03-04 19:46 CET Interesting ports on 212.36.75.80: PORTSTATE SERVICE 143/tcp open imap MAC Address: 00:16:3E:33:AA:FD (Xensource) ¿Why the nmap output is randomly open or filtered when pf is enabled? I understand (according to this Henning's message: http://www.monkey.org/openbsd/archive/misc/0204/msg00510.html) that I'm using the correct settings. More info: $ uname -a OpenBSD ares.cdmon.com 4.2 GENERIC#0 amd64 -- Thanks, Jordi Espasa Clofent
Re: Allow nmap [solved]
After a very profitable man nmap(1) reading, it's enough: $ nmap -sT -p 143 xxx.xxx.xxx.xxx I hope it will be useful for someone. -- Thanks, Jordi Espasa Clofent
Re: Allow nmap
I have not done more than skim your config, but you might try
temporarily disabling your block foo_brutes rules and see
if that makes a difference in case your nmap scan is triggering
whatever it is that populates the brutes tables.
It's not the problem Karl.
I've done a lot of tests from $nagios box like this:
$ i=0; [ while i -lt 25 ]; done \
nmap -p 143 imap_server_behind_pf | grep tcp | awk '{print $2}' ; \
sleep 2; \
done
and the output is, randomly, open or filtered as I've said in the
previous post.
I've tried with a lot of nmap's options (scan types, times policies...
etc) but I get always the same results.
The only pattern I've found is when the nmap resolution is quick, it
shows always 'open', and, when the nmap resolution seems to be slow,
the ouput is 'filtered'.
In PF's side I've tried with:
* no scrub from $nagios
* pass in on $bridge inet proto { tcp, udp, icmp } from $nagios allow-opts
--
Thanks,
Jordi Espasa Clofent
Slow SSH connection
Hi all,
The picture is
Internet ---| FW (bridge mode) - My net
I use a OpenBSD box with 4.2 stable in bridge mode with PF enbled as a
FW. Very happy with performance and capabilities of PF. But when I try
ssh connections from outside to my net boxes, they're very very slow.
They work, but work so slowly. On the other hand, when I disabled de PF
un FW box, the ssh connection has the 'normal' speed.
I've tried the verbose flags in ssh connection, but the output is
exactly the same when I use or not use the PF in FW.
My pf.conf is simple:
# 1. MACROS
ext_if=em1
int_if=em0
bridge={ $ext_if $int_if }
admin_if=em2
pfsync_if=em3
oficines=xxx.xxx.xxx.xxx
nagios=xxx.xxx.xxx.xxx
# 2. TABLES
# services tables
table http_servers persist file /etc/pf_files/http_servers.pf
table smtp_servers persist file /etc/pf_files/smtp_servers.pf
table remote_mail_servers persist file
/etc/pf_files/remote_mail_servers.pf
table mysql_servers persist file /etc/pf_files/mysql_servers.pf
table postgresql_servers persist file
/etc/pf_files/postgresql_servers.pf
table ssh_servers persist file /etc/pf_files/ssh_servers.pf
table snmp_servers persist file /etc/pf_files/snmp_servers.pf
table dns_servers persist file /etc/pf_files/dns_servers.pf
table ftp_servers persist file /etc/pf_files/ftp_servers.pf
# brutes tables
table ssh_brutes persist
table mysql_brutes persist
table http_brutes persist
table dns_brutes persist
# 3. OPTIONS
set skip on lo
set skip on em2
set skip on em3
set fingerprints /etc/pf.os
set block-policy drop
set optimization aggressive
# 4. SCRUB
scrub in all
# 5. QUEUEING
# Not used.
# 6. TRANSLATION
# Not used.
# 7. FILTER RULES
# several generic rules
antispoof quick for lo
block in all
pass out quick all
pass inet proto icmp from any to any
# allow always the admin connection from headquarters
pass in quick on $bridge0 inet proto { tcp, udp } from $oficines to any
# blocking quickly the brutes
block quick from ssh_brutes
block quick from mysql_brutes
block quick from http_brutes
block quick from dns_brutes
# httpd
pass in on $bridge0 inet proto tcp from any to http_servers port { 80,
443 } keep state \
(max-src-conn 100, max-src-conn-rate 15/5, \
overload http_brutes flush global)
# smtpd
pass in on $bridge0 inet proto tcp from any to smtp_servers port { 25,
578 }
# pop3(s) and imap4(s)
pass in on $bridge0 inet proto tcp from any to remote_mail_servers
port { 110, 143, 993, 995 }
# MySQL
pass in on $bridge0 inet proto tcp from any to mysql_servers port 3306
keep state \
(max-src-conn 100, max-src-conn-rate 15/5, \
overload mysql_brutes flush global)
# PostgreSQL
pass in on $bridge0 inet proto tcp from any to postgresql_servers port
5432
# SSH
pass inet proto tcp from any to ssh_servers port 22 keep state \
(max-src-conn 100, max-src-conn-rate 15/5, \
overload ssh_brutes flush global)
# SNMP
pass in on $bridge0 inet proto udp from { $oficines $nagios } to
snmp_servers port 161163
# DNS
pass in on $bridge0 inet proto { tcp, udp } from any to dns_servers
port 53
# FTP
# cmd: ftpsesame -i $bridge0
#passive
anchor ftpsesame/* in on $bridge0 proto tcp from any to $bridge0
anchor ftpsesame/* out on $bridge0 proto tcp from any to $bridge0
# active
anchor ftpsesame/* in on $bridge0 proto tcp from $bridge0 to any
anchor ftpsesame/* out on $bridge0 proto tcp from $bridge0 to any
pass in on $bridge0 inet proto tcp from any to $bridge0 port { 20, 21 }
I think the SSH rule is completely fine... so ¿why this low speed?
--
Thanks,
Jordi Espasa Clofent
Re: Slow SSH connection
Stuart Henderson escribió: On 2008/02/24 12:21, Jordi Espasa Clofent wrote: Very happy with performance and capabilities of PF. But when I try ssh connections from outside to my net boxes, they're very very slow. They work, but work so slowly. Describe this in a bit more detail... Yes Stuart, I know my words are vague, but it's exactly what I've said: the ssh connection with pf enable seems a slow process. A few points: * With pf disabled you get the ssh Password prompt in (aprox) 3 secons. * With pf enabled you'll get the ssh Password prompt in (aprox) 15 secons. * The use of ssh verbose flags (-vvv) it's the same with or without pf. Maybe the next step is a bit of work with tcpdump -- Thanks, Jordi Espasa Clofent
Semi-OT: required TAP
Hi all, I want to implement a failover FW structure and a IDS too. So, the picture will be | |Internet. Cat6 to main router | TAP | IDS --- | | | | FW1 FW2 | | | | -- | Main switch | -- | Internal network Obviously, the FW1 and FW2 redudancy is maded by CARP and dedicated NIC/wire. My dumb question is ¿I need a TAP with 2 replication-ports (in relation to main traffic incoming port)? I'm thinking in the case that one FW fails, so I supose I need the main traffic arrives to the both FW. ¿Am I right? If I'm right ¿what TAP provider do you advise to me (netoptics, comfract...) -- Thanks, Jordi Espasa Clofent
PF and multiple CPUs
Hi all, I've read about this tipic in this present list and in the misc@ list. Currently I'm making performance test in a box which has a pair of Quad-Core Intel Xeon processor L5320. If I use bsd kernel ¿only a 1 Core of the first CPU is used? If I use a bsd.mp kernel ¿always pf performance decrease? Sincerely, nowadays is hard to find out a simple-CPU in servers environment. -- Thanks, Jordi Espasa Clofent
Network performance tool with little sized packets
Hi all, I'm testing my FW with OpenBSD 4.2+pf in bridging mode. At present moment I've done test with iperf and netperf tools, using a 32/64/128K packets. But a simple sniffer in my production networks shows me that the packet size is very little: less or equal to 64 bytes (around 35% of total), between 64 and 128 bytes (around 35%) and between 1024 and 1518 bytes (the rest). So, I need a tool as iperf/netperf sytle, but capable to produce/manage a little packets. ¿Do you know someone? ¿Do you another way/method to check the performance capability that OpenBSD+bridge+pf offers with a little packets? -- Thanks, Jordi Espasa Clofent
Re: Network performance tool with little sized packets
Ok Trevor. The real production scenario is: | | CAt6 wire provided by datacenter | - | Switch Gigabit | - | - | Servers | - The idea is put the FW, obviously, between the Cta6 wire and the main gigabit switch. because of that I buildup de FW as a bridge. If I sniffer the traffic which passes through the switch, I notice the size of packets that I've mentioned in previous post: * 64 bytes (35%) * 64-128 bytes (35%) * 1024-1500 (30%) So, I need to benchmark the FW with little size packets. The question is ¿Is there any tool which generates small packets traffic to benchmark the network performance as iperf or netperf does? -- Thanks, Jordi Espasa Clofent
Re: Network performance tool with little sized packets
iperf can do this with a command-line option. Try netblast (from the netrate package) if you want packets sent quickly without controls on the rate (it just sends them as fast as it can). M... I've tried to do it with iperf... but it seems not capable. Maybe I'm not using the correct flag (I sincerely doubt). I'll try netblast. The other parts of netrate allow more control over the packet rates, but don't work too well with normal kernel HZ values. Oks. -- Thanks, Jordi Espasa Clofent
