Jeff Santos wrote:
> Hi there,
>
> I am going to try this list because I think what I
> am facing is somehow related to PF and because I am
> almost desperate to solve this.
>
> I have firewall running OpenBSD 4.1-STABLE and the
> following daemons: spamd, dhcpd, routed and named.
>
> It has 3 interfaces:
>
> 1. an external interface sk1 with IP 200.132.140.3/24
> 2. an internal interface sk0 with IPs 200.132.120.1/24,
> 200.132.120.100/24 and 200.132.135.1/24
> 3. a wireless gateway interface vr0 with IP 172.16.255.1/24
>
> The IP 200.132.120.100 is used for named.
> The IP 200.132.120.1 is the gateway for the network
> 200.132.120/24.
> The IP 200.132.135.1 is the gateway for the network
> 200.132.135/24.
> The IP 172.16.255.1 is the gateway for the network
> 172.16.255/24
>
> This firewall runs "route -s" because there is a need
> to publish RIPv1 routes for these networks.
>
> Now, every once in while I get errors like:
>
> PING 200.132.120.2 (200.132.120.2): 56 data bytes
> ping: sendto: No route to host
> ping: wrote 200.132.120.2 64 chars, ret=-1
> 64 bytes from 200.132.120.2: icmp_seq=1 ttl=64 time=0.231 ms
> 64 bytes from 200.132.120.2: icmp_seq=2 ttl=64 time=0.238 ms
> --- 200.132.120.2 ping statistics ---
> 3 packets transmitted, 2 packets received, 33.3% packet loss
> round-trip min/avg/max/std-dev = 0.231/0.234/0.238/0.015 ms
>
> I know there is an arp entry for the IP address above.
>
> netstat -ni shows no hardware error
>
> Normally, this happens when the load is high. I notice many
> (in my opinion) interrupts on the skc0 and skc1 interfaces,
> using systat vm. I see more than 1000 interrupts for each
> interface.
>
> The number of "punt RTM_ADD without gateway" messages increases
> when this kind of problem appears. (My guess is that they are
> related to spamd redirections)
>
> At night, when the traffic is not intensive, it goes back to
> normal as if nothing had happenned.
>
> If it helps, the output of pfctl -si is:
>
> Interface Stats for sk0 IPv4 IPv6
> Bytes In 47446477242 48781
> Bytes Out 21084047711 64
> Packets In
> Passed 46837573 0
> Blocked 3024000 98
> Packets Out
> Passed 36144287 1
> Blocked 4502 0
>
> State Table Total Rate
> current entries 10000
> searches 168992519 1669.2/s
> inserts 2790142 27.6/s
> removals 2780142 27.5/s
> Counters
> match 6355599 62.8/s
> bad-offset 0 0.0/s
> fragment 22 0.0/s
> short 2 0.0/s
> normalize 6 0.0/s
> memory 3050088 30.1/s
> bad-timestamp 0 0.0/s
> congestion 17968 0.2/s
> ip-option 3 0.0/s
> proto-cksum 1697 0.0/s
> state-mismatch 17276 0.2/s
> state-insert 0 0.0/s
> state-limit 0 0.0/s
> src-limit 0 0.0/s
> synproxy 8130 0.1/s
>
> My questions are:
>
> 1. Could PF be responsible for this kind of misbehaviour?
>
> 2. If it is a hardware problem, could it show up only under
> heavy load, and could it not show up with netstat -ni?
>
> 3. Any tool I could use to help to find out what is going on?
>
>
> I would be really thankful to anyone that can suggest some
> possible reason, test, insight.
>
> In order to avoid this message being too long, I will not
> post other data, but on demand, I could send anything that
> might help.
>
>
> Regards,
>
> Jeff.
>
> ps. below is pf.conf (slightly edited for readability!)
>
> ------------------------
> ------------------------
> # LISTS, MACROS AND TABLES
>
> ext_if="sk1"
> int_if="sk0"
> wl_if="vr0"
>
> lan_n1 = "200.132.120.0/24"
> lan_n2 = "200.132.135.0/24"
> lan_net = $int_if:network
> wlan = $wl_if:network
> default_gw = "200.132.140.22/32"
>
> priv_nets = "{ 127.0.0.0/8, 192.168.0.0/16, 172.16.0.0/12, 10.0.0.0/8 ,
> 169.254.0.0/16 }"
>
> bugrouters = "{ 200.132.140.27 }"
> novowebserver = "200.132.120.253"
> nameservers = "200.132.120.100"
> webservers = "{ 200.132.120.253, 200.132.120.2, 200.132.120.91,
> 200.132.120.86, 200.132.120.29, 200.132.120.206, 200.132.120.26
> }"
> mailservers = "{ 200.132.120.2, 200.132.120.91 }"
> sshservers = "{ 200.132.120.1, 200.132.120.253, 200.132.120.91,
> 200.132.120.157, 200.132.120.98, 200.132.120.58, 200.132.120.57,
> 200.132.120.44, 200.132.120.184, 200.132.120.162,
> 200.132.120.130, 200.132.120.30, 200.132.120.26, 200.132.120.206
> }"
> trustedsshclients = "{ 200.132.120.252 }"
> proxyservers = "{ 200.132.120.2 }"
> routers = "200.132.140.0/24"
> ntpservers = "{ 150.162.34.1, 146.164.53.65, 146.164.48.5,
> 200.218.160.160,
> 200.162.106.162, 200.144.121.33 }"
>
> proxyport = "3128"
> mailports = "{ pop3 pop3s imap imaps }"
> icmp_types = "echoreq"
>
> table <spamd-white> persist
>
> # OPTIONS
>
> set skip on lo0
> set block-policy return
> set loginterface $ext_if
>
> # NORMALIZATION
>
> scrub in
>
> # REDIRECTION
>
> # FTP proxy . Will take care of this later
> # Initially, only ACTIVE mode
>
> nat-anchor "ftp-proxy/*"
> rdr-anchor "ftp-proxy/*"
> rdr pass on $int_if proto tcp to port ftp -> 127.0.0.1 port 8021
>
> # Wireless interface NAT
>
> nat on $ext_if from $wlan to any -> ($ext_if)
> nat on $int_if from $wlan to 200.132.120.100 -> 200.132.120.1
>
> # SPAMD stuff
>
> no rdr on $ext_if proto tcp from <spamd-white> to any port smtp
> rdr pass on $ext_if proto tcp from any to any port smtp \
> -> 127.0.0.1 port spamd
>
> # RULES
>
> pass out quick inet proto udp to port 520 no state
>
> block log all
>
> pass quick on lo0 all no state
>
> pass out quick on $ext_if inet proto icmp keep state
>
> block drop in log quick on $ext_if from $lan_net to any
> block drop out log quick on $ext_if from any to $lan_net
> block drop in log quick on $ext_if from $priv_nets to any
> block drop out log quick on $ext_if from any to $priv_nets
>
> block drop in log quick on $ext_if inet proto udp \
> from $bugrouters to any port 520
>
> # incomming DNS queries
>
> pass in on $ext_if inet proto udp from any to $nameservers \
> port domain keep state
>
> # outgoing DNS queries
>
> pass out on $ext_if inet proto udp from any to any \
> port domain keep state
>
> # Incomming Web traffic
> # **** watch out for the synproxy option ...
> # it may hang web connections
>
> pass in on $ext_if inet proto tcp from any to $webservers \
> port www flags S/SA synproxy state
> # port www flags S/SA keep state
>
> pass in on $ext_if inet proto tcp from any to $webservers \
> port https flags S/SA keep state
>
> # Allow basically everything on the internal interface
> # watch out for wireless traffic
>
> pass in on $int_if from $lan_net to !($int_if) keep state
>
> # DHCP
>
> pass in on $int_if from any to 255.255.255.255/32 keep state
>
> pass in on $int_if inet proto udp from $lan_net to \
> 200.132.120.100 port domain keep state
>
> pass in on $int_if inet proto tcp from $trustedsshclients \
> to 200.132.120.1 port ssh flags S/SA keep state
>
>
> # From Wireless NET to internal NET
>
> pass out on $int_if from !$wlan to $lan_net keep state
>
> pass out on $int_if inet proto udp from $wlan to $nameservers \
> port domain keep state
>
> pass out on $int_if inet proto tcp from $wlan to $webservers \
> port www flags S/SA synproxy state
>
> pass out on $int_if inet proto tcp from $wlan to $webservers \
> port https flags S/SA keep state
>
> pass out on $int_if inet proto tcp from $wlan to $mailservers \
> port smtp flags S/SA keep state
>
> pass out on $int_if inet proto tcp from $wlan to $mailservers \
> port $mailports flags S/SA keep state
>
> # outgoing TCP traffic
>
> pass out on $ext_if inet proto tcp from any to !($ext_if) \
> flags S/SA modulate state
>
> pass out log on $ext_if inet proto tcp from any to any \
> port smtp flags S/SA keep state
>
>
> # MAIL
>
> pass in log on $ext_if inet proto tcp from any to $mailservers \
> port smtp flags S/SA keep state
>
> pass in on $ext_if inet proto tcp from any to $mailservers \
> port $mailports flags S/SA keep state
>
It has to do with the statefull features.
When I load pf with no statefull passing the problem goes away.
-Mike
> # SSH servers
>
> pass in on $ext_if inet proto tcp from any to $sshservers \
> port ssh flags S/SA keep state
>
> # SQUID/PROXY traffic
>
> pass in on $ext_if inet proto tcp from any to $proxyservers \
> port $proxyport flags S/SA keep state
>
> pass out on $int_if inet proto tcp from any to $proxyservers \
> port $proxyport flags S/SA keep state
>
> # ICMP traffic
>
> pass in inet proto icmp all icmp-type $icmp_types keep state
>
> pass in on $ext_if inet proto icmp from $default_gw to \
> ($ext_if) icmp-type "redir" keep state
>
> pass out on $ext_if inet proto icmp all keep state
>
> # Wireless traffic
>
> pass in log on $wl_if from any to any keep state
> pass out log on $wl_if from any to any keep state
>
> # NTP
>
> pass out on $ext_if inet proto udp from any to $ntpservers \
> port ntp keep state
>
> # RIP2 ??
>
> pass out quick to 224.0.0.1 keep state
>
>
