Re: Are there any RDR 1-to-1 Multiple Rule Shortcuts?
On Thu, Aug 14, 2014 at 02:56:45PM -0400, Alan McKay wrote:
internalIPS = { 1 2 3 }
externalIPS = { 4 5 6 }
pass in quick log on $extIf inet proto tcp from any to (externalIPs)
port (some port) rdr-to (internalIPs)
Maybe I'm just hallucinating :-)
There's no such thing with lists or tables, but you might be thinking of
'bitmask'
... to 192.168.0.0/24 rdr-to 10.0.0.0/24 bitmask
which is short for
... to 192.168.0.1 rdr-to 10.0.0.1
... to 192.168.0.2 rdr-to 10.0.0.2
... to 192.168.0.3 rdr-to 10.0.0.3
...
... to 192.168.0.255 rdr-to 10.0.0.255
You can adjust the netmask and network bits on the right-most argument
to shift the range of the internal IPs, but it must always be an
continuous block.
There are more examples on
http://www.openbsd.org/faq/pf/pools.html
HTH,
Daniel
Re: Openbsd Routing Issues
I think you're passing some packets statelessly, because you don't block correctly by default: nat on vr0 from !(vr0) to any - (vr0) round-robin scrub on vr0 all no-df fragment reassemble scrub on vr0 all reassemble tcp block drop in log on vr0 all pass out quick on ath0/rl0 keep state. First, change the block rule to apply to all interfaces and directions: block log all Now you'll see (by tcpdump'ing pflog0) what is getting blocked. Most likely, you also need pass in on ath0/rl0 keep state pass out on vr0 keep state Kind regards, Daniel
Re: PF + gif + ipsec + racoon + routing problems
I rebuilt your setup but can't reproduce the problem.
I picked A.A.A.A=3.3.3.3 and B.B.B.B=4.4.4.4 and used FreeBSD 8.3-STABLE
i386 with GENERIC plus IPSEC, and installed ipsec-tools-0.8.0_3.
-- gatewayA --
/etc/rc.conf
ifconfig_em0=inet 1.1.1.254 netmask 255.255.255.0
ifconfig_em1=inet 3.3.3.3 netmask 255.255.255.0
gif_interfaces=gif0
gifconfig_gif0=3.3.3.3 4.4.4.4
ifconfig_gif0=1.1.1.254 2.2.2.254 netmask 255.255.255.0
defaultrouter=3.3.3.1
static_routes=gif
route_gif=-net 2.2.2.0/24 2.2.2.254
gateway_enable=YES
racoon_enable=YES
pf_enable=YES
# ifconfig gif0
gif0: flags=8051UP,POINTOPOINT,RUNNING,MULTICAST metric 0 mtu 1280
tunnel inet 3.3.3.3 -- 4.4.4.4
inet 1.1.1.254 -- 2.2.2.254 netmask 0xff00
options=1ACCEPT_REV_ETHIP_VER
# netstat -anr
DestinationGatewayFlagsRefs Use Netif Expire
default3.3.3.1UGS 0 1948em1
1.1.1.0/24 link#1 U 0 1270em0
1.1.1.254 link#1 UHS 10lo0
2.2.2.0/24 2.2.2.254 UGS 0 1873 gif0
2.2.2.254 link#5 UH 0 39 gif0
3.3.3.0/24 link#2 U 00em1
3.3.3.3link#2 UHS 00lo0
127.0.0.1 link#4 UH 00lo0
/etc/pf.conf is a simple (and identical on gatewayB)
set state-policy if-bound
set skip on { lo }
scrub in log all fragment reassemble
block log
pass
# pfctl -ss
em0 icmp 2.2.2.2:25352 - 1.1.1.1 0:0
em1 esp 3.3.3.3 - 4.4.4.4 MULTIPLE:MULTIPLE
/usr/local/etc/racoon/psk.txt
4.4.4.4 topsecret
/usr/local/etc/racoon/ipsec.conf
flush;
spdflush;
spdadd 1.1.1.0/24 2.2.2.0/24 any -P out ipsec esp/tunnel/3.3.3.3-4.4.4.4/use;
spdadd 2.2.2.0/24 1.1.1.0/24 any -P in ipsec esp/tunnel/4.4.4.4-3.3.3.3/use;
/usr/local/etc/racoon/racoon.conf
(exact copy of handbook example, only differences:)
listen
isakmp 3.3.3.3 [500];
isakmp_natt 3.3.3.3 [4500];
remote 4.4.4.4 [500]
my_identifier address 3.3.3.3;
peers_identifieraddress 4.4.4.4;
sainfo (address 1.1.1.0/24 any address 2.2.2.0/24 any)
-- gatewayB --
ifconfig_em0=inet 2.2.2.254 netmask 255.255.255.0
ifconfig_em1=inet 4.4.4.4 netmask 255.255.255.0
gif_interfaces=gif0
gifconfig_gif0=4.4.4.4 3.3.3.3
ifconfig_gif0=2.2.2.254 1.1.1.254 netmask 255.255.255.0
defaultrouter=4.4.4.1
static_routes=gif
route_gif=-net 1.1.1.0/24 1.1.1.254
gateway_enable=YES
racoon_enable=NO
pf_enable=YES
# ifconfig gif0
gif0: flags=8051UP,POINTOPOINT,RUNNING,MULTICAST metric 0 mtu 1280
tunnel inet 4.4.4.4 -- 3.3.3.3
inet 2.2.2.254 -- 1.1.1.254 netmask 0xff00
options=1ACCEPT_REV_ETHIP_VER
# netstat -anr
DestinationGatewayFlagsRefs Use Netif Expire
default4.4.4.1UGS 0 2066em1
1.1.1.0/24 1.1.1.254 UGS 0 2023 gif0
1.1.1.254 link#5 UH 00 gif0
2.2.2.0/24 link#1 U 0 1984em0
2.2.2.254 link#1 UHS 10lo0
4.4.4.0/24 link#2 U 00em1
4.4.4.4link#2 UHS 00lo0
127.0.0.1 link#4 UH 00lo0
# pfctl -ss
em1 esp 4.4.4.4 - 3.3.3.3 MULTIPLE:MULTIPLE
em0 icmp 1.1.1.1:25352 - 2.2.2.2 0:0
/usr/local/etc/racoon/psk.txt
3.3.3.3 topsecret
/usr/local/etc/racoon/ipsec.conf
flush;
spdflush;
spdadd 2.2.2.0/24 1.1.1.0/24 any -P out ipsec esp/tunnel/4.4.4.4-3.3.3.3/use;
spdadd 1.1.1.0/24 2.2.2.0/24 any -P in ipsec esp/tunnel/3.3.3.3-4.4.4.4/use;
/usr/local/etc/racoon/racoon.conf
listen
isakmp 4.4.4.4 [500];
isakmp_natt 4.4.4.4 [4500];
remote 3.3.3.3 [500]
my_identifier address 4.4.4.4;
peers_identifieraddress 3.3.3.3;
sainfo (address 2.2.2.0/24 any address 1.1.1.0/24 any)
-- router --
When I ping from gatewayB to 1.1.1.1 (or from 1.1.1.1 to 2.2.2.2),
I see only encrypted packets:
13:23:52.800285 IP (tos 0x0, ttl 63, id 6391, offset 0, flags [none], proto ESP
(50), length 136)
4.4.4.4 3.3.3.3: ESP(spi=0x016bdbe7,seq=0x5e), length 116
13:23:52.801401 IP (tos 0x0, ttl 64, id 5827, offset 0, flags [none], proto ESP
(50), length 136)
3.3.3.3 4.4.4.4: ESP(spi=0x04049e8b,seq=0x5e), length 116
13:23:53.820296 IP (tos 0x0, ttl 63, id 6394, offset 0, flags [none], proto ESP
(50), length 136)
4.4.4.4 3.3.3.3: ESP(spi=0x016bdbe7,seq=0x5f), length 116
13:23:53.821230 IP (tos 0x0, ttl 64, id 5829, offset 0, flags [none], proto ESP
(50), length 136)
Re: Filtering on the basis of the relay host
On Sat, May 11, 2013 at 09:10:09AM -0600, JCA wrote: I would be interested to use milter-regex to filter incoming emails according to the relay host. When an email arrives, sendmail logs a line containing several fields, like 'from', 'size', 'msgtype', etc. and their values. One of those fields is 'relay'. Can milter-regex filter emails depending on the value of this field? That's simply the peer of the SMTP connection, which is matchable with the 'connect' expression. Daniel
Re: I want to filter some/all inbound traffic twice
If you need NAT, you have to do that on the external interface, and it requires (implies, even) creating states. However, you can filter statelessly on the internal interface (the states won't match there (wrong direction, if-bound), dropping outgoing TCP RST, passing everything else. Sounds similar to what was done to ignore the great firewall of China, see http://www.cl.cam.ac.uk/~rnc1/ignoring.pdf :) HTH, Daniel
Re: I want to filter some/all inbound traffic twice
On Fri, Apr 05, 2013 at 07:03:52PM +1100, Cameron Simpson wrote: I was imagining NATing on an internal virtual interface to a private address on some kind of internal virtual interface; this might keep the necessary state without being the outmost layer. And then to do stateless filtering of RSTs on the real physical external interface (because the NAT states are not present there, or even if floating, will not match), and then have some kind of stateless binat or other rewrite of the remaining non-RST packets from the real external address to the private address used for the NAT. I guess you have already considered the obvious solution of adding a second device (stateless filtering bridge) justs to drop the RSTs. While it may seem a waste of parts and electricity, it's much simpler than trying to route a connection through the same device multiple times. Is it a router or a bridge now? If a router, you'd probably have to use rtables to trick the stack, see http://www.packetmischief.ca/2011/09/20/virtualizing-the-openbsd-routing-table/ If you have two spare physical interfaces, a patch cable between them would be simple and produces non-surprising interaction with pf. I don't know which (if any) virtual interface would help. | However, you can filter statelessly on the internal interface (the | states won't match there (wrong direction, if-bound), dropping outgoing | TCP RST, passing everything else. Won't the RST packets shut down the TCP states as they traverse to external interface? The RST packets will cause the state entries to show up in pfctl -ss as TIME_WAIT:TIME_WAIT, but that doesn't make them not match further packets. The only effect is that the entries will time out (when idle!) with tcp.closed (90s by default) instead of tcp.established (24 hours). You can increase tcp.closed, the downside is that the entries will stay around longer, even you'd want them to be purged. Daniel
Re: Best/simplest/fastest approach for creating virtual switch out of
Yes, bridge between em2 and em3.
Assign the IP (used as gateway by the clients) to bridge0.
You'll have to duplicate the MAC filter rules per interface.
The pf rules need to match both interfaces with 'on { em2 em3 }',
and floating state-policy (default) will simply work. No increase in
complexity there.
If you dislike the syntax there, you can use an interface group, but
it's purely cosmetical.
HTH,
Daniel
Re: forwarding loop
On Wed, Jan 16, 2013 at 10:19:45AM +0100, Leslie Jensen wrote: The squid access.log says tcp_miss which should mean that the website has not replied. The browser shows the squid access denied screen. I cannot see any denied packets with tcpdump. Commenting out the rdr rule gives direct access but I would like to get the use of squid back again. I would appreciate further suggestions because at the moment I'm stuck! Standard debugging procedure, then: On the firewall, run the following fw# tcpdump -nvpi bge0 host 172.18.0.1 fw# tcpdump -nvpi xl0 tcp port 80 fw# tcpdump -nvvveeepi pflog0 Then initiate a single connection attempt from the proxy, like with proxy# nc -v www.openbsd.org 80 and watch the output of the tcpdumps. What do you expect to see? 1) TCP SYN from 172.18.0.1 to 129.128.5.194:80 on bge0 2) TCP SYN from $ext_if to 129.128.5.194:80 on xl0 3) TCP SYN+ACK from 129.128.5.194:80 to $ext_if on xl0 4) TCP SYN+ACK from 129.128.5.194:80 to 172.18.0.1 on bge0 5) nothing on pflog0 (except 'pass' messages) What do you see instead? If you see block messages on pflog0, re-post your current ruleset. Daniel
Re: forwarding loop
Wait, the squid server is on a separate host, on the $int_if side of the firewall (the same side the clients are on)? Then transparent proxying would require reflection, and doesn't work, see http://www.openbsd.org/faq/pf/rdr.html#reflect If squid is seeing TCP_MISS errors, that probably means the clients are using the proxy explicitely (not transparently), and the rdr is not being used at all... This worked for the past years how? Daniel
Re: forwarding loop
On Tue, Jan 15, 2013 at 11:50:14AM +0100, Leslie Jensen wrote: 2013-01-15 11:10, Daniel Hartmeier skrev: Wait, the squid server is on a separate host, on the $int_if side of the firewall (the same side the clients are on)? Yes! This machine has been in service since Freebsd 7.2. It's one machine with two nic's functioning as a gateway running pf and squid as proxy. Well, is Squid running on the same host, or on a separate host? Is proxy=172.18.0.1 an IP of int_if=bge0? Daniel
Re: forwarding loop
You currently have the following rules pass out log on $ext_if inet proto tcp from $proxy to any port $proxy_services keep state # pass out pass out log What's the point of these? Whenever the first rule would match, the second one would always override it, making the first one superfluous. Further, the first rule never matches, because the proxy isn't using $proxy=172.18.0.1 as source for outgoing connections (but the IP of $ext_if). The second rule matches, but doesn't have 'keep state', so the TCP handshake fails (SYN passes out, but SYN+ACK reply is blocked). Hence, delete the first rule and add keep state to the second rule. HTH, Daniel
Re: route-to round-robin using single interface?
On Mon, Jan 14, 2013 at 03:30:21PM +0100, Johan Helsingius wrote:
I have a small network, connected by 2 ADSL connections, and
want to load-share the connections. All examples of route-to
round-robin that I have seen have used 2 separate interfaces,
but as both my ADSL modems are on the same no-mans-land
network, I have been (so far unsuccessfully) trying to do
something like this:
pass in on $int_if from $int_net \
route-to { ($ext_if $isp1_gw), ($ext_if $isp2_gw) } \
round-robin sticky-address
Is that supposed to work, or does route-to round-robin only
work with 2 separate interfaces?
AFAIK, it should work.
tcpdump with -e on $ext_if and check the destination MAC addresses.
Can you ping $isp1_gw and $isp2_gw and arp -sn is showing two
different entries for them?
What is the problem? All packets always go to $isp1_gw's MAC?
Or the sticky-address source tracking isn't working?
Are you using multiple clients on $int_net? If not, what do you expect
the sticky-address to do?
Have you tried adding keep state(soure-track global) and
set timeout source-track and checked with pfctl -sS?
Daniel
Re: forwarding loop
On Tue, Jan 15, 2013 at 09:46:37AM -0600, Karl O. Pinc wrote: Something that's not mentioned that comes to mind is ICMP redirection. (Without thinking about it a lot it seems like it should be a good candidate.) However when I tried ICMP redirection on OpenBSD years ago I couldn't get it to work. Never looked to see why, or whether it's been fixed since. Or, I might have been doing something wrong. If anyone can send a clue my way I'd appreciate knowing more. An ICMP redirect, if it is honored by a client, does not mean if you want to talk to external-server, connect to proxy-on-local-network instead, but if you want to reach external-server, route through next-hop-on-local-network. The difference is that the reaction will not change the destination IP address, but the destination MAC address. Its really the same as if you added a temporary host route on the client. The proxy will simply ignore the IP packet for a foreign IP address sent to its MAC address (like a default gateway that has IP forwarding disabled). OpenBSD ignores ICMP redirects by default, you can enable it with sysctl net.inet.icmp.rediraccept, if you want to try. The security risk is that someone on the local network could send you ICMP redirects for various destinations (like DNS server), for a man-in-the-middle attack. Daniel
Re: forwarding loop
On Sat, Jan 12, 2013 at 08:03:41AM +0100, Leslie Jensen wrote: New suggested rule that gives syntax error # rdr in on $int_if inet proto tcp from ! $proxy to any port $proxy_services - $proxy $proxyport tag rdr_proxy 1) Remove in (rdr implies incoming direction) 2) Remove tag rdr_proxy (newer OpenBSD syntax, not needed here) 3) Add port after the - part (not optional) So it reads: rdr on $int_if inet proto tcp from ! $proxy to any port $proxy_services - $proxy port $proxyport Which loads fine for me on 8.3-RELEASE-p4. HTH, Daniel
Re: Simultaneous CARP failover for multiple interfaces
On Mon, Apr 23, 2012 at 02:23:20PM -0700, Kyle Lanclos wrote: However, this does jog another potential failure mode. Some of our older OpenBSD firewalls (going back to OpenBSD) will occasionally (maybe once a year) lose a network interface. If you logged in at the console of a host while it was in this state, the interface would look perfectly normal, but it would not pass any traffic. I callously worked around this by administratively cycling each network interface on the affected machine(s) on a weekly basis. If we ran into this failure mode with our CARP firewalls, I'm assuming the master would keep right on thinking it was the master, and not attempt to demote iteslf. While it is certainly helpful for self-demotion of a master to occur, it seems reasonable for self-promotion of a slave to also occur. Without any active probing, like with ifstated, there is no way to distinguish which uplink is up but not forwarding. It could be either the master's, or the backup's, or both. Statistically, for every time you improve the situation by failing over, there is a time you shoot yourself in the foot doing the same. If you do nothing, you have the same chances, and things remain simpler. With ifstated, you only need to change one side's advskew so their order reverses, then rely on carp's election process. For instance, run ifstated only on the master, pinging next hops on all sides, and - when any ping fails (the first time), demote: increase own advskew above the backup's - when all pings succeed (again), promote: reset own advskew to the original value (below the backup's) In your example above, ifstated on the master would detect a ping failure on one next hop and demote by increasing its advskew above the backup's. With preempt enabled, the master would lose election on the other interface and therefore group failover all interfaces to backup state, while the backup would win election and group failback all interface to master state, i.e. self-promotion of the backup is done only through carp election. When you fix the interface, ifstated will see all pings succeed again, and reset advskew. Now the (preferred) master wins election and fails back. I don't think there is a case where it's helpful to run scripts on both the master and the backup. You'd have to be careful to not introduce new failure cases, for instance when a next hop is unreachable from both. Daniel
Re: Simultaneous CARP failover for multiple interfaces
On Mon, Apr 23, 2012 at 11:49:14AM -0700, Kyle Lanclos wrote: Where this presents a problem is if the current CARP master loses a single network interface (cable unplugged, isolated hardware failure, sysadmin failure, etc.), as opposed to the CARP master failing entirely. The slave will appropriately assume the master role for one CARP interface, but will *not* do so for the second. Yes, it will: net.inet.carp.preempt Allow virtual hosts to preempt each other. It is also used to failover carp interfaces as a group. When the option is enabled and one of the carp enabled physical interfaces goes down, advskew is changed to 240 on all carp interfaces. See also the first example. Disabled by default. (i.e. this single sysctl knob enables both group failover and failback) This covers link state change (unplugged cable) as well as administrative down of the physical interface. It does not cover the case where the link remains up, but the uplink switch stops forwarding, for instance, but... We would like our otherwise nicely redundant firewall configuration to be resilient against this type of failure. Short of running a cron job every sixty seconds to check the interface state, is there some way we can automatically force the promotion of a CARP slave if a second CARP interface flips from slave to master? .. see ifstated(8), which can ping uplink hops and issue ifconfig advskew changes to demote the master when appropriate. Daniel
Re: double NOT in rules is not working as expected
On Fri, Apr 08, 2011 at 03:42:41PM +0300, Bojidara Marinchovska wrote:
So the correct question is how to accomplish
pass in quick on $netif from {$test1, $test2} to x.x.x.x
block in quick on $netif from any to x.x.x.x
with only 1 rule ?
While negating a list never does what you want, negating a table
does the expected:
table test const { 1.2.3.4, 2.3.4.5 }
block from ! test to x.x.x.x
The rule matches any source except 1.2.3.4 and 2.3.4.5.
See http://www.openbsd.org/faq/pf/tables.html for more examples.
Daniel
Re: Pretty sure I don't understand my own pf.conf
On Wed, Jan 05, 2011 at 08:42:03PM -0800, Bonnie Packet wrote: So my question is, again how regular packets from the Net pass out to the wireless network over rl0. Is this somehow a function of the NAT rules that I don't understand? Or something to do with established TCP connections being already green-lit? I would think without an explicit rule they'd be blocked (default block at the very end). Those packets are replies for connections opened from a wireless client to a server on the external net? Then, yes, those are passed back in due to the 'keep state' option you use on the rule pass in quick on $wls_if inet from 192.168.1.140 to any flags S/SA keep state This allows connections to be opened from 192.168.1.140 to any host, and covers both packets flowing from 192.168.1.140 to the server AND packets (that are part of such connections) back from the server to the client, it's the whole point of the 'keep state' part. If you mean other packets, please explain what kind. Since the wireless net has a non-routable address range, the only way packets from the external net would end up there is if they were initially directed to the routable external address, then de-NAT'd, which is only possible if there's a state entry (based on an outgoing packet). Daniel
Re: [pf] Re: Weird behaviour with pass out _keep state_
On Fri, Mar 13, 2009 at 10:25:15AM +0100, Jeremie Le Hen wrote: % Mar 13 08:18:52 yoda /netbsd: pf: BAD state: TCP 82.233.239.98:39225 82.233.239.98:39225 88.187.38.85:80 [lo=3443494040 high=3443494041 win=2048 modulator=0] [lo=0 high=1 win=1 modulator=0] 2:0 S seq=3041360721 ack=0 len=0 ackskew=0 pkts=1:0 dir=out,fwd M% ar 13 08:18:52 yoda /netbsd: pf: State failure on: 2 | 6 This message means there already is a state entry for a connection from 82.233.239.98:39225 to 88.187.38.85:80. There must have been some outgoing packet that created the state entry. If it wasn't from nmap, what else was it from? If it was from nmap, obviously what has failed wasn't the FIRST sendto()... Also, in my previous email I described an unexpected behaviour in my own understanding. When I disable state tracking, legitimate outward connections still work. The SYN packet is obviously allowed to leave, but pf doesn't record the connection state so the SYN/ACK response should be dropped. this doesn't seem to be the case as the box doesn't turn deaf :). Explanation? You have to look at your entire ruleset, not just the 'pass out' rule. Either you don't have a generic block rule (default deny policy), or you have another 'pass in' rule which matches the return packets. If a packet doesn't match ANY rule in your ruleset, it will pass. Daniel
Re: Weird behaviour with pass out _keep state_
On Thu, Mar 12, 2009 at 10:13:53AM +0100, Jeremie Le Hen wrote: % yoda# nmap -sS AAA.BBB.CCC.DDD % Starting Nmap 4.65 ( http://nmap.org ) at 2009-03-12 08:00 CET % sendto in send_ip_packet: sendto(4, packet, 44, 0, AAA.BBB.CCC.DDD, 16) = No route to host % Offending packet: TCP WWW.XXX.YYY.ZZZ:57973 AAA.BBB.CCC.ZZZ:80 S ttl=39 id=39289 iplen=11264 seq=3869808471 win=4096 mss 1460 However, I can make nmap run flawlessly when I remove the keep state statement: % pass out quick all Do you have an idea about the problem? Note that pf on NetBSD 4.0.1 doesn't implicitely enable flags S/SA as in later version of OpenBSD. The following scenario would produce what you observe: 1) nmap sends a first TCP SYN to AAA.BBB.CCC.DDD with a random initial sequence number th_seq1 2) pf allows the packet out and creates a state entry (check with pfctl -vvss) 3) the external host is down or doesn't respond to the TCP SYN at all 4) nmap times out waiting for a response, retries sending the probe (see nmap --max-retries) 5) nmap crafts a second TCP SYN to AAA.BBB.CCC.DDD, but unwisely chooses another initial sequence number th_seq2 != th_seq1 6) the packet matches the state entry in pf, but the sequence number is completely off, hence pf blocks the packet 7) when a local socket's outgoing packet is blocked by pf, the syscall gets errno No route to host Try running nmap with -vv, maybe it shows what it's trying to do. Also enable verbose logging (pfctl -x misc) and check /var/log/messages for BAD state entries, which come with sequence number mismatches. If this is indeed what's happening, nmap is misbehaving, and pf is correct in blocking the invalid retransmission. With nmap -sT it usually works, because the TCP/IP stack of the OS doesn't make mistakes like this :) Daniel
Re: super simple pf.conf that doesn't work as expected.
On Wed, Nov 26, 2008 at 12:52:47PM -0600, Patric wrote:
ext_if = xl2
int_if = xl1
localnet = $int_if:network
nat on $ext_if from $localnet to any - ($ext_if)
pass from { lo0, $localnet } to any keep state
__
this is pretty much the most basic natting pf.conf described in The
Book of PF and I can't pass any traffic through it at all, pftop shows
nothing, and I am starting to doubt my sanity, any help is greatly
appreciated.
Translation occurs before filtering, so outgoing packets will have
$ext_if as source on the external interface (not $localnet), hence
your pass rule is not matching.
Daniel
Re: Need stateless NAT
On Wed, Apr 02, 2008 at 04:27:17PM -0700, Adam Richards wrote: While I'd prefer a yes pf can do this answer, I will accept a no...but here are the code sections you'll want to look at to start your patch work answer. ;) No, pf can't do it. Not because it's technically impossible or unreasonable, it's just not a typical use case. For most users, routable address space is a scarcer resource than RAM for state table entries (they have much less external IP addresses than internal ones). Take a look at pf.c, you probably have to add some short-circuit to pf_test_state_*(). Instead of looking up the state entry, simply do the address mapping. Or maybe fiddle with the state lookup, so it returns a static state entry with the fields filled out correctly for the mapping. It's probably not trivial. Good luck ;) Daniel
Re: queuing question
On Wed, Feb 27, 2008 at 11:28:28AM -0800, Daniel Duerr wrote: I understand from the pf documentation (and logic) that you cannot queue incoming packets on an interface, makes sense... In various examples around the net, however, I've seen people attaching queues to inbound rules as well. I'm confused as to whether this is just a mistake or if people are doing this on INBOUND rules which keep state in order to have all OUTBOUND packets which are state tracked to that rule get the queue assignment from the inbound rule. Can someone please clarify? The latter (queue replies of incoming connections) is true. Daniel
Re: RDR not RDRing to non-localhost IP
On Sun, Feb 24, 2008 at 01:28:43AM +1300, Michael Adams wrote: Any help most appreciated, I'm really stumped by this! Thanks, and apologies if it's a stupid question. http://www.openbsd.org/faq/pf/rdr.html#reflect Daniel
Re: block-policy return ignored with ipv6?
On Tue, Feb 12, 2008 at 07:40:14PM +0100, Helmut Schneider wrote: Is that expected? No, it's a bug introduced with pf.c 1.534 after 4.1 was released. http://www.openbsd.org/cgi-bin/cvsweb/src/sys/net/pf.c.diff?r1=1.533r2=1.534f=h For IPv6 TCP, calling pf_check_proto_cksum() with AF_INET will always fail. No RST will be generated, the 'proto-cksum' counter in pfctl -si output will increase instead. Henning? Daniel
Re: Policy based routing and RDR?
route-to != reply-to Daniel
Re: Policy based routing and RDR?
If I understand correctly, you have serveral uplinks, and you redirect requests coming from each uplink to dedicated servers, i.e. server A handles all requests coming in through uplink A. You want the return packets to go back through the appropriate uplink. This should work: Use rdr on the external interface like you are already doing. On the internal interface use pass out on fxp0 reply-to (fxp1 $ext_gw_a) from any \ to $int_net_a port 80 keep state pass out on fxp0 reply-to (fxp1 $ext_gw_b) from any \ to $int_net_b port 80 keep state Note that the rdr on the external interface has already replaced the destination address when these rules are evaluated on the internal interface. As an example, an external client C sends a SYN to $ext_gw_b. This packet will first hit the external interface, match the rdr rule, and the destination address is replaced with int_net_b. This then goes out through the internal interface, matching the second rule above. When the SYN+ACK comes back from the server, it will match the second rule's state entry, and the reply-to option gets used, bypassing the normal routing table lookup. The packet will go out through the external interface with $ext_gw_b's MAC address as destination. There it will match the rdr rule's state, and the source address is replaced. You might want to try with 'set state-policy if-bound', but it should work either way. This works as long as you have one dedicated server per external uplink. If you want to load-balance all incoming requests to a server farm, you'd have to use tags between the rdr rules and the reply-to rules. Daniel
Re: Milter-regex and authorized users...
On Wed, Jan 23, 2008 at 01:49:31AM +0300, Ogogon !!! wrote: How to me to adjust milter-regex that the authorized remote users could send mail from any address? Check what macros and values the milter gets from sendmail, either by logging daemon.debug in syslog.conf or running the milter with option -d manually. Any macro seen by the milter should then be printed. Usually looking at the values causes enlightenment ;) Daniel
Re: Strange BAD state errors ...
Most of your BAD state messages are of the form pf: BAD state: TCP X.X.129.45:80 X.X.129.45:80 X.X.246.205:1771 [lo=4006379205 high=4006444151 win=17424 modulator=0] [lo=2523483440 high=2523483440 win=65535 modulator=0] 4:4 A seq=2523483440 (2523483440) ack=4006379205 len=1452 ackskew=0 pkts=17:34 dir=out,rev pf: State failure on: 1 | The browser (client) opened a connection to the web server, and with this packet, the server is returning data. The client has a receiving window of 65535 bytes (the size of its read buffer). The server can send data only until that buffer is full. Then it must wait for the client to drain the buffer and acknowledge that it is ready to receive more data. In your case, the client window is full (lo=high=2523483440, yet the server is trying to send further data (seq=2523483440 + len=1452), which is not allowed. Either the web server is violating TCP (because of a bug in its TCP/IP stack, or due to some form of throughput optimization), or, more likely, you have built yourself a setup where this pf box is not seeing all packets sent from client to server. I see pfsync log messages, too. If there is any form of load-balancing, or if there are multiple links or paths between client and server, you have to make sure that all packets of the same TCP connection flow through the same pf box. Otherwise stateful filtering breaks, and it could look like what you're seeing. Try to capture a single TCP connection (with tcpdump on all relevant interfaces of the pf box) from handshake to the point where the BAD state message occurs (include the BAD state message, too). Daniel
Re: Strange BAD state errors ...
On Mon, Dec 17, 2007 at 03:52:00PM +0100, Henrik Johansen wrote: You can find the requested dump here: http://blog.myunix.dk/pf/tcpdump.sanitized The corresponding BAD state error is here: http://blog.myunix.dk/pf/messages_tcpdump.sanitized There is nothing obviously wrong with the dump. The peers negotiate SACK, and the bulk of the data is from server to client. Eventually, a packet got lost. The client makes use of SACK to tell the server about the gap due to the lost packet. The server takes a while to react and re-transmit just the gap. Then, the client half-closes its end of the connection (telling the server that it won't send more data, but will read more). The server tries to keep sending more data. Then the client resets the connection, setting its window to 0. The server either doesn't get the reset, or is slow to react to it, it keeps sending data, which pf blocks with the BAD state messages. I'd say it's safe to ignore those BAD state messages. After the client has sent a RST, there is no harm in not passing further packets. You might want to investigate why a) the client resets the connection (is the server sending a wrong HTTP Content-Length: header, so the client thinks it got all data, when the server thinks there should be more?) b) why the server doesn't react to the RST by shutting up immediately. c) does this pattern occur with all clients, or only with a specific client? Daniel
Re: problem with a route-to option within an anchor
On Mon, Sep 03, 2007 at 08:17:45AM +0200, Tobias Marx wrote: ### outgoing http loadbalancing ### anchor http_out out on $extif_1 from clients to any load anchor http_out from /bla/http_out And the problem is? You get a syntax error when trying to load the ruleset? If so, from which file? Try loading /bla/http_out as the main ruleset, if that already produces a syntax error, this is not related to anchors at all... daniel
Re: traffic shaping using pf
On Mon, Sep 03, 2007 at 10:22:53PM +1200, Russell Fulton wrote: I take it from the silence that the answer is that pf lacks this functionality at the moment. Bother :) Yes, that's correct. What would the overhead be of setting up a queue for every source address (1024 of them) ? Will this impact performance? I think the maximum number of classes is 256 for CBQ and 64 for HFSC, see sys/altq/altq_cbg.h CBQ_MAX_CLASSES and altq_hfsc.h HFSC_MAX_CLASSES. You can increase those, it looks like the worst effect is that the list/array is traversed linearly, so 1024 would be four times as costly as 256, i.e. I'd expect acceptable. Daniel
Re: problem with a route-to option within an anchor
On Mon, Sep 03, 2007 at 11:37:22AM +0200, Tobias Marx wrote:
can someone verify this behaviour?
I doubt this has anything to do with anchors. To verify, you can move
the contents of http_out into the main ruleset (and remove the anchor
call), and reboot, as a test. Watch out for error messages on the
console.
My first guess would be the interfaces. If either $ext_if1 or $ext_if2
do not exist at ruleset load time (because they're not physical
interfaces like xl0, but tun0 or such), you should get an error message
on the console when pfctl is invoked from rc.
In that case, you might have to add a pfctl -f call at the end of
rc.local, for instance. Not all interface references are late-binding
(can be used before the interface exists).
pass out quick route-to {(extif_2 proxy_ip1),(extif_2 proxy_ip2)} \
^^ ^^ ^^ ^^
Those are typos (missing $), right?
Daniel
Re: include macros
On Fri, Aug 17, 2007 at 11:01:20AM -0700, Dylan Martin wrote: Or, is there another way around this problem? A way to make an alias for an interface, say? Interface groups work pretty well for that, see ifconfig(8). In most cases, the default 'egress' group (containing only the interface where the default route points out through) is appropriate, and if not you can define your own. ExtIf = egress Daniel
Re: pfstad with different interfaces
On Fri, Aug 10, 2007 at 09:50:57AM +0200, Javier Solorzano wrote: I collect with /usr/local/bin/pfstat -q -r myip:port -d /var/db/adam.db and produce graphics with /usr/local/bin/pfstat -c /etc/ pfstat.adam.conf -d /var/db/adam.db -p You need to specify the alternate configuration file with both -q and -p, not just with -p. The reason is that the query will only read and store those values that you have defined in the configuration (and not everything available), to save space. Daniel
Re: more on my question : DNS answers blocked?
Looks like the blocked packets were IP fragments. For stateful filtering, IP fragments must be reassembled, try adding scrub in fragment reassemble at the top of your ruleset. Daniel
Re: Problems with PF Sync (FreeBSD 6.2)
On Fri, Feb 02, 2007 at 02:12:12PM -0800, Michael K. Smith - Adhost wrote: self tcp 10.211.100.110:110 - x.x.x.164:110 - x.x.x.98:52857 ESTABLISHED:ESTABLISHED [526026435 + 65535] wscale 1 [2600240610 + 65665] wscale 0 age 00:00:03, expires in 04:59:57, 3:2 pkts, 168:154 bytes, rule 9 id: 45c3ac450080 creatorid: 70b9fa06 What is rule 9 on the master, precisely? # pfctl -gsr | grep -A 2 '@9 ' The state entry doesn't get associated with a corresponding rule on the backup (because the rulesets are not identical), but with the default rule instead. This means that aspects of the state entry might stop working on failover (like route-/reply-to or such), effectively breaking the connection. Daniel
Re: synproxy create state while no ACK received
On Wed, Feb 07, 2007 at 04:47:00AM +0800, frank hu wrote: So my question is why PF create state while the first 3-way handshakes didn't complete? What is right usage of synproxy rule to protect port from DoS attack? That's what synproxy does, by design. It does protect the recipient from seeing packets (and allocating resources) before the handshake is complete, but at the cost of itself allocating some resources (the state entry). The state entry will not pass packets through to the real recipient before the handshake is complete, but merely holds the information needed to verify the handshake with the client is valid. Daniel
Re: Carp/pfsync kernel panic
On Mon, Jan 29, 2007 at 04:33:45PM +0100, Thomas Althoff wrote: I did the crash procedure on 3.9 and found that this is the line causing the problem if (!r-max_states || r-states r-max_states) I have upgraded my boxes to 4.0-current, no change. If you can reproduce it with a recent 4.0-current, and the location is really that line above, it looks like a different problem Can you try the patch below? Daniel Index: if_pfsync.c === RCS file: /cvs/src/sys/net/if_pfsync.c,v retrieving revision 1.73 diff -u -r1.73 if_pfsync.c --- if_pfsync.c 16 Nov 2006 13:13:38 - 1.73 +++ if_pfsync.c 29 Jan 2007 17:51:01 - @@ -243,7 +243,9 @@ * If the ruleset checksums match, it's safe to associate the state * with the rule of that number. */ - if (sp-rule != htonl(-1) sp-anchor == htonl(-1) chksum_flag) + if (sp-rule != htonl(-1) sp-anchor == htonl(-1) chksum_flag + ntohl(sp-rule) = 0 ntohl(sp-rule) pf_main_ruleset.rules[ + PF_RULESET_FILTER].active.rcount) r = pf_main_ruleset.rules[ PF_RULESET_FILTER].active.ptr_array[ntohl(sp-rule)]; else
Re: Strange disconnection problem - 2nd take
I'd first make sure it's not CARP related (i.e. all packets always pass through one box), by (temporarily) turning off the backup box. If, for some reason, packets would flow through both boxes (some through the master, some through the backup), things would break in funny ways. Now that everything must pass through the master, enable debug logging (pfctl -xm), note counters (pfctl -si), and reproduce the problem once. If you can, tcpdump one faulty connection (from the initial SYN to where the problem shows) on all relevant interfaces (two, I assume). Check /var/log/messages for lines from pf, especially BAD state. Note updated counters (pfctl -si again), and diff old vs. new. Which counters are increasing? In your previous tcpdump, the client starts to use SACK after one packet from the server is lost. Maybe that is what distinguishes the clients (some use SACK, some don't). You could confirm this theory by (temporarily) disabling SACK on the server (net.inet.tcp.sack=0 on OpenBSD). Daniel
Re: TCP timestamp clock behavior
On Thu, Jan 11, 2007 at 09:46:39PM -0600, Travis H. wrote: So, surprisingly, many OSes don't synchronize their TCP timestamp clock to their system clock, so effectively they leak the skew of that clock, even if they are synching their system clock via NTP. I am wondering what the current behavior is for OpenBSD, and if scrubbing or any other pf function (e.g. synproxy) does anything about it. The first match searching for 'timestamp' in pf.conf(5) is in TRAFFIC NORMALIZATION reassemble tcp timestamp modulation reassemble tcp will cause scrub to modulate the TCP timestamps with a random number. which sounds like it might be vaguely relevant, have you tried that? ;) Daniel
Re: Problems with PF Sync.
On Thu, Dec 14, 2006 at 02:47:16PM -0800, Michael K. Smith - Adhost wrote: Our problem is with state maintenance upon failover. It appears the state tables are properly synced between the devices but, when we fail to our secondary firewall, established connections through the firewalls fail. We have replicated this behavior with port 25 and 110. Can you further explain what fails? Do connections established prior to the failover stall after the failover? After the failover, new connections cannot be established (connection times out, or is reset)? After the failover, new connections can be established, but the sticky-address option is not honoured, so new connections go to the wrong server, breaking stuff like smtp-after-pop? For the first two cases, please enable debug logging (pfctl -xm), reproduce the problem for one connection, then check /var/log/messages for entries from pf, and post them. Also run pfctl -vvss before (on the primary) and after (on the secondary) and post the state entry that fails. The last case is currently unsolvable, as the information about which source address is assigned which redirection address is not sync'd, afaik. Daniel
Re: mismatch on route through packet/byte counts
On Mon, Dec 04, 2006 at 02:02:38PM +0100, Axel Rau wrote: If flags S/SA would just be ignored by none-tcp packets, I would be happy. Be happy, it is. ;) But the man page says: This rule only applies to TCP packets that have the flags a set out of set b. This means to me: all none-tcp packets are ignored by this rule. This probably should read instead This rule only applies to TCP packets which have the flags a set out o set b. Daniel
Re: pf on FreeBSD
On Wed, Dec 13, 2006 at 05:52:03PM +0100, Albert Shih wrote: It's a problem with FreeBSD or it's with pf ? With neither, you're assuming a state entry has the same effect in pf as in ipfw, which is not the case. For example I've put this kind of rule pass in on $first-nic proto tcp from IP-A to IP-B port 22 keep state When I try to connect from IP-A to IP-B using ssh the connection don't work. And I've got self tcp IP-B:22 - IP-A:56906 CLOSED:SYN_SENT self tcp IP-B:22 - IP-A:59496 CLOSED:SYN_SENT in my pfctl -s state and got deny for outgoing packet from IP-B to IP-A That is expected with pf. A state entry created for an incoming packet on one interface does not allow the same packet to go out through another interface, it merely allows further packets through the same interface and _replies_ back out through the same interface. If you do want to allow the packets to pass through another interface (as is usually the case with legitimate forwarded connections), you have to add pass out on $second-nic proto tcp from IP-A to IP-B port 22 keep state which will then create a _second_ state entry for the same connection. The point of this is that you can control _which_ interface(s) a connection must flow through, instead of granting a permission to pass any and all interfaces. This may seem pointless to want to control in a simple setup which only forwards between two NICs, but it isn't in a more complex case with multiple NICs and routing tables dynamically updated and/or not trusted. On my old FreeBSD I'm using something like ipfw add permit any to any established. The pf counterpart would be pass from any to any keep state i.e. leaving out the 'on $if' part makes the rule apply to all interfaces, and leaving out the 'out' or 'in' direction makes it apply to both directions. Daniel
Re: ext_if, int_if?
On Thu, Nov 30, 2006 at 04:00:37PM +, Karl O. Pinc wrote: The clean solution would be if pf had some sort of #include mechanisim. Then the macros that abstract the interfaces could be written into include-ed files and everything else would be sane. pfctl -D int_if=foo -f /etc/pf.conf or even pfctl -D int_if=`grep-o-matic` -f /etc/pf.conf comes to mind. You can also use interface groups, even as a degenerate case where each interface is in its own group, (ab)using the group name as a functional interface name, then reference only interface groups in pf.conf. Many options, all less work than adding features to pfctl ;) Daniel
Re: Bug in pf FAQ?
On Fri, Dec 01, 2006 at 02:14:14PM +1300, Russell Fulton wrote: pass in quick on fxp0 all allow-opts Am I correct in thinking that this line effectively passes *all* traffic in on fxp0 with no more checking because of the 'quick' option? Yes, it does. The rule is meant to illustrate the syntax of the allow-opts option (i.e. where in the rule to place it) with an example. Surely in the context of the FAQ this rule should not have quick so that subsequent block rules will take effect. Removing the quick option would, in this case, not have the desired effect, either. Adding the non-quick rule at the beginning would have NO effect at all, since ruleset evaluation for any packet (blocked or passed) would match subsequent rules. Only the last matching rule matters, and whether that last matching rule has allow-opts set or not. If you expect packets with IP options on any kind of connection you want to pass, you actually have to add allow-opts to ALL your existing pass rules. Most people see legitimate packets with IP options only for a few kinds of connections (say, between specific peers, or specific ports), so it's easier/safer to add allow-opts only to specific rules matching those packets. I agree the example in the FAQ is not helpful making that clear. But simply removing quick from it will not be much better. In general, all the one-line examples in the FAQ are not meant as building blocks that you can simply copy/paste into larger rulesets mechanically. I'm pretty sure this is not the only case where the one-liner only illustrates the syntax, and the effect of inserting it into complex rulesets is non-trivial. If you, or anyone else, can suggest a sentence or paragraph to add to the FAQ that makes this clear, please do. Daniel
Re: Clarification on recommended method of ACK queueing
On Fri, Nov 10, 2006 at 11:10:47AM -0800, Nathan Valentine wrote: Reference: http://www.benzedrine.cx/ackpri.html One of the rules in that document is: pass out on $ext_if proto tcp from $ext_if to any flags S/SA \ keep state queue (q_def, q_pri) The 'flags S/SA' part of this rule only affects what packets match this rule. We only create state on the initial SYN packet (ACK not set). This has nothing to do the queue assigments. Once the initial SYN packet has matched the rule and created state, all further packets related to the connection (no matter what flags are set or unset) pass based on the state entry, not the rule above. Whether one specific packet of the connection (matching the state) gets assigned to q_def or q_pri DOES depend on the flags it has, but that is hardcoded: empty ACKs go to q_pri, everything else to q_def. The 'flags S/SA' rule option has no influence on the queues. So, the purpose of the rule is not to separate empty ACKs from other packets. It merely picks the initial SYN of each connection to create the state entry on. Maybe that was the confusion. I see nothing wrong with the wording in the man page about 'flags S/SA', it's precise and correct. Daniel
Re: state-policy floating not honored ?
On Thu, Nov 09, 2006 at 03:11:55PM +0100, Pierre-Yves Ritschard wrote: It works as expected but this looks like a 'state-policy ifbound' behavior right ? I tested this with latest (11/7) current available on my mirror and the behavior is the same. You're probably misreading what 'floating' means. See http://marc.theaimsgroup.com/?l=openbsd-pfm=114372425614238w=2 i.e. a floating state does not allow packets on arbitrary interfaces in arbitrary directions, it merely allows one direction on arbitrary interfaces (when routing changes). You still need two states if you filter both directions. Daniel
Re: PF Table Size - Sanity Check
On Tue, Nov 07, 2006 at 06:08:52PM +, Paul Pruett wrote: A nominal i386 computer with only a meg of ram without limit changes would not load it. Neither would a stock GENERIC kernel on any architecture. The reason is that those 600+MB of table entries are allocated from kernel memory. And the kernel only gets a fraction of the physical RAM installed. You'll have to try, but I doubt you'll end up with more than half a gig of kernel memory even if you put in 4+GB of RAM. And, no, I don't know if it's possible to tune that or how ;) The opposite is obviously true, i.e. if you have less than 600MB of physical RAM, there is no way you can load 600+MB of table entries. Daniel
Re: PF Table Size - Sanity Check
On Mon, Nov 06, 2006 at 02:21:58PM -0800, Michael K. Smith - Adhost wrote: We are looking at pulling in a listing of about 70,000 IP entries (most of them are hosts, not subnets) into a PF Table. Is there any hard limitation to the configuration size or ability to parse through something that large? Shouldn't be a problem. # pfctl -sm [...] tableshard limit 1000 table-entries hard limit 10 I.e. the default limit on number of table entries (across all tables) is 100,000. If you need more (and have sufficient memory), you can increase it with 'set limit table-entries' in pf.conf. # wc -l file 7 file # time pfctl -t foo -Ta -f file 1 table created. 7/7 addresses added. 0m1.27s real 0m0.39s user 0m0.80s system # vmstat -m [...] Memory resource pool statistics NameSize Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg Maxpg Idle [...] pfrkentry216700 3889 0 3889 3889 0 5556 0 The pfrkentry pool holds the table entries. The size of one entry (depends on architecture, here 216 bytes) multiplied by the number of entries is 216*7 = 14.41MB. So 70,000 isn't that large. 700,000 would probably be a challenge, and 7,000,000 would be beyond reasonable ;) Daniel
Re: ACKs queueing
On Sat, Oct 07, 2006 at 06:29:46PM +0200, Federico Giannici wrote:
Having received no useful replies, let me try a simpler question: How
can I identify (i.e. filter) TCP ACKs with no data payload?
I know how to identify ACKs, but is there any way to identify packets
with no payload, something like a payload-size 0 condition?
No, that's hardcoded and you'll have to patch the source if you want to
change that. See src/sys/net/pf.c pf_test()
if ((th.th_flags TH_ACK) pd.p_len == 0)
pqid = 1;
This sets the pqid flag (use the second queue specified) for TCP packets
with TH_ACK set and no payload.
if (action == PF_PASS r-qid) {
if (pqid || (pd.tos IPTOS_LOWDELAY))
pd.pf_mtag-qid = r-pqid;
else
pd.pf_mtag-qid = r-qid;
This assigns the rule's second specified queue if the pqid flag is set
before, or if IPTOS_LOWDELAY is set.
Change as you see fit. Adding a keyword about payload size to the parser
is much more work, compared to the kernel part ;)
BTW, queue assigment happens for all packets matching a state, no matter
what direction they flow in. If you have two four queues (two for each
interface involved), you want to make sure that the right queue is
assigned last (the mbuf tag gets overwritten, and only the last write is
relevant, should it happen multiple times to the same packet).
Daniel
Re: pf mail list problem --- Daniel
On Wed, Aug 16, 2006 at 06:56:49AM +0200, Michal Soltys wrote: George Pontis wrote: Daniel - any attempt that I can muster to subscribe to the pf mailing list fails because spamassassin flags it as spam. The recommended text subscribe in the body of the message is rated as spam and the mail rejected as so: [cut] I had similar problems where anything I've tried to send was classified as spam. Keyword spamassassinexception helps in the first stage of subscribing, but doesn't during verification. At the moment, I'm just using the newsgroup. The word 'subscribe' has over time been associated with spam by the bayes database, and has a slightly spammy score. The trick is to not only send this one line, but add an 'end' line (see http://www.benzedrine.cx/majordomo-help.html) and add some lines after that to tilt the score. Quote some previous email from the list, or add a couple of pf.conf example lines, or such. Same for an empty subject. The mailing list script ignores it, but it scores spammy in SA. If it doesn't work out, drop me a mail privately, and I'll subscribe you manually. Sorry for the trouble. Daniel
Re: Dual WAN, outgoing routing problem
On Wed, Aug 16, 2006 at 02:04:32PM -0700, George Pontis wrote: pass in quick on $int_if \ route-to ($ext_if2 $ext_gw2) \ inet proto tcp from 192.168.1.120 to any flags S/SA \ keep state (floating) \ You seem to be assuming that a floating state created on $int_if will match packets on fxp0 (presumably $ext_if). That's simply not the case. More details can be found in http://marc.theaimsgroup.com/?l=openbsd-pfm=114372425614238 In short, you need a pass out on fxp0 ... keep state rule for those connections, and you'll get TWO states per connection. Daniel
Re: Dual WAN, outgoing routing problem
On Wed, Aug 16, 2006 at 08:33:10PM -0700, George Pontis wrote: # # route packets from any IPs on $ext_if1 to $ext_gw1 and the same for ext_if2 to ext_gw2 # pass out on $ext_if2 route-to ($ext_if1 $ext_gw1) from $ext_if1 to any flags S/SA keep state pass out on $ext_if1 route-to ($ext_if2 $ext_gw2) from $ext_if2 to any flags S/SA keep state There must be something about my understanding of how packets are routed that is at the root of the problem. I say that since the packets that are dropped are being routed out ext_if1 in contradiction to the route-to ext_if2 option. How can I express the logic in pf that will ensure that all the traffic for a specific LAN host will always route through a specific interface, no matter what is in the system routing tables ? Your two rules above only re-route packets with source addresses $ext_if1 and $ext_if2. A specific LAN host would only have such a source address if you were using NAT. You have given no indication of whether you are, and what your NAT rules are... How else would packets from a LAN host have one of the firewall's own IP addresses as source address? Or what did you think from $ext_if1 meant? Daniel
Re: rdr over ip alias
On Tue, Jul 25, 2006 at 01:07:06PM -0400, mnothic wrote: I need documentation it approaches incoming traffic in an IP ALIAS and port redirected to a private IP and port Start with the pf.conf(5) man page, as on http://www.openbsd.org/cgi-bin/man.cgi?query=pf.conf There is nothing special about redirecting connections that originally have an IP alias as the destination address, you just specify that in the rdr ... to $alias - $new_dest part. Daniel
Re: deleting a rule
On Wed, Jul 19, 2006 at 01:35:51PM +0530, Rajkumar S. wrote: And these rules are dynamic ie, the rule one might be for 10 minutes and after which it needs to be deleted. The current way is to flush the anchor and then load the anchor with all the rules except the one deleted. It is a pita if I want to do this with out touching the disk, that too from a snortsam pluin. Why don't you create sub-anchors, one for each single rule? Then removing one rule (and the sub-anchor that contains it) can be done by simply flushing the sub-anchor. You need one call in the main ruleset or the existing anchor, using the wildcard '*', that call evaluates all sub-anchors, and the call doesn't need to be updated when you insert/remove sub-anchors. You could even use the sub-anchor names in some clever way, like put the rule's expiration time (unix epoch) in that string, so to purge expired rules, you can traverse the list of sub-anchors alphabetically and stop when a name is larger than time(NULL). Or store some ID in the name (which your plugin associates with the entry), which helps you purge the sub-anchor without traversing them all searching for some rule. Unless you expect to have several thousand rules like this concurrently, the overhead of the sub-anchor evaluation isn't that terrible. IIRC, the ioctl API once contained a call to insert/remove one particular rule in a certain place of the ruleset, but it was cumbersome, and the entire (sub-)anchor concept makes it superfluous in most cases. Daniel
Re: getting started with pfstat
On Thu, Jul 13, 2006 at 11:07:46AM -0400, Peter wrote: I have installed the pfstat 1.7 package on my 3.8 system. The trouble is that I do not get any data being graphed. Here is my test setup: # cat /etc/pf.conf pass log all Add set loginterface fxp0, which designates one interface to collect those counters for, which pfstat collects. (newer versions of pfstat can query other interface counters, so the loginterface isn't needed anymore, and you can plot graphs for multiple interfaces) Bonus question: How does the program reconcile the data file being updated at a different interval than the one the graph is being generated with (i.e. every one minute as opposed to every five minutes)? Each line contains the timestamp (unix epoch time, seconds since 1970) of when the counters had those values of that line. The rest is simple math done by pfstat. When it needs to find out which height to draw one specific pixel at, it calculates what timestamps the left and right edge of the pixel represent. For example, if you plot a whole year on 800 pixels width, one pixel represents about 365*24*60*60/800 == 39420 seconds. If this is left-most pixel, and you plot the year ending with timestamp 1152805255 (today, 17:40:55 MEST), that pixel represents the time range 1152765835-1152805255. For that pixel, pfstat will take all log entries that fall at least partly within that range, and calculates weighted average, minimum, and maximum of all matching log entries. While it costs a little CPU time to do that, it allows to plot graphs with arbitrary widths over arbitray time ranges, no matter how often or regular the values are collected. In extreme cases (much more pixels than values), you'll see one value spread over many pixels, generating a blocky staircase or even a simple horizontal line. But the average should be mathematically correct, i.e. the area below the line (width times height) matches the amount of bytes transfered in that time period. Daniel
Re: ALTQ for a process running on PF box
On Wed, Jul 12, 2006 at 03:25:28PM +1000, Adam Clark wrote: traffic going to hosts behing the box were showing in on pflog0, but no traffic to 10.17.10.254 shows. If I put a log-all on a line that matches the traffic on the $ext_if interface it shows that in deed traffic is heading towards 10.17.10.254. Which means that even though the internal IP address is bound to the internal interface, the internal interface never sees traffic for 10.17.10.254 that can be filtered. Tcpdump does not show this either Is this true or is there a way perform what I need to do in another way? Yes, that's normal, but you're not the first one to find this surprising. :) When the host, on one interface, receives a packet with a destination IP address that matches one of the host's own IP addresses (assigned to any interface), the packet is removed from the forwarding path and dispatched to the local socket instead. The destination address of the packet can influence what local socket it is dispatched to (like when you have two different sockets bound to different addresses), or it doesn't matter at all (when you have a single socket bound to *), but the kernel doesn't simulate any passage through $int_if. If it would try to send out the packet through $int_if, that would mean writing an ethernet frame out onto $int_if's wire. The simplest solution in your case is to move the bittorrent client process onto a different host on the internal network, so inbound packets to it really pass out on $int_if. We recently had a lenghty thread about the disadvantages (requiring separate hosts) of lacking inbound queues, see http://groups.google.com/group/bit.listserv.openbsd-pf/browse_thread/thread/5de1c7731114bdae If you have to move the process to another host, maybe it's a little comforting that this is also the wise thing to do from a security perspective. In the completely hypothetical case where the process has a remotely exploitable hole, you don't risk the attacker using it to gain root on the firewall and opening up its ruleset. Daniel
Re: pfstat network client
On Tue, Jul 11, 2006 at 05:52:21PM -0500, Travis H. wrote: If pfstatd makes statistics available to network clients like pfstat, how does one tell pfstat to use the network? I see no options for such, and putting the hostname on the command line just generates an error. You need pfstat-2.2, which introduced pfstatd, the man page there is updated: -r host[:port] Query the statistics from a remote host running pfstatd(8). Default is to query the local packet filter through the de- vice special file /dev/pf, not requiring any running pfstatd(8). Daniel
Re: FW: Strange smtp problem..
On Wed, Jul 05, 2006 at 01:34:34PM +0200, Daniel Rapp wrote: pf: BAD state: TCP aaa.aaa.aaa.aaa:25 aaa.aaa.aaa.aaa:25 ccc.ccc.ccc.ccc:2554 [lo=1937461566 high=1937478751 win=65535 modulator=0] [lo=740836633 high=740902095 win=17184 modulator=0] 4:4 R seq=1937461566 ack=740836633 len=0 ackskew=0 pkts=2:4 dir=in,fwd pf: State failure on: | You're absolutely sure this is quoted/pasted correctly, the second line really has no numbers after the colon? Daniel
Re: Strange smtp problem..
On Tue, Jul 04, 2006 at 12:12:51PM +0200, Daniel Rapp wrote: pass out quick on $WAN proto tcp all flags S/SA Why no 'keep state' here? You really only pass out SYNs, don't pass SYN+ACK back in, and neither pass further (non-SYN) packets? Makes no sense. If i do a tcpdump -e -n -ttt -vv -i pflog0 i get: Jul 04 11:48:30.678318 rule 88/(match) [uid 0, pid 6857] pass in on sis3: bbb.bbb.bbb.bbb.2916 aaa.aaa.aaa.aaa.25: [|tcp] (DF) (ttl 120, id 41053, len 40, bad cksum 0! differs by d68) Bad checksum on incoming packets ?, could that be the problem? Retry with added -s 1700 to the tcpdump command. The default snaplen is too short, truncating packets ([|tcp]), producing the checksum warnings. An thoughts ? I see nothing wrong with that dump. If that winsock error is reliable and means the server got a TCP RST, it almost certainly was the external peer sending it. You'll have to get a tcpdump capture of one connection that produces the error in the server, preferably on both interfaces of the bridge. Depending on traffic volume, it might be difficult to get, but try tcpdump'ing all port 25 traffic into a file, then wait until the next server error occurs, then filter the file using the peer's random port number. To prove that it's pf at fault producing the RST, you'll have to show that the server is receiving an RST, the RST was sent out from the bridge's internal interface, and that is has not arrived in on the bridge's external interface. Those peers don't happen to be all from China [1], by any chance? :) Daniel [1] http://www.lightbluetouchpaper.org/2006/06/27/ignoring-the-great-firewall-of-china/
Re: FW: Strange smtp problem..
On Wed, Jul 05, 2006 at 02:07:42PM +0200, Daniel Rapp wrote: pf: BAD state: TCP aaa.aaa.aaa.aaa:25 aaa.aaa.aaa.aaa:25 bbb.bbb.bbb.bbb:2554 [lo=1937461566 high=1937478751 win=65535 modulator=0] [lo=740836633 high=740902095 win=17184 modulator=0] 4:4 R seq=1937461566 ack=740836633 len=0 ackskew=0 pkts=2:4 dir=in,fwd pf: State failure on: | Ah, that means the RST didn't actually have a th_seq of 1937461566, we're logging after adjusting it (Ease sequencing restrictions on no data packets), somewhat confusing seeing that the first time ;) This message means a TCP RST packet (incoming on the external interface) was blocked by pf because its sequence number th_seq did not precisely match. We check for precise match to make guessing (blind reset attacks) harder. This is a RST that was sent from the external peer, and we blocked it. So it can't explain why the server produces a connection reset by peer log message. Maybe subsequent RSTs (also from the external peer) have exactly matching th_seq and pass. But none of this shows anything except that the RSTs seem to originate from the external peer... Daniel
Re: RFC1323 Window Scaling Issues
On Fri, Jun 30, 2006 at 09:40:38PM -0700, Mark Voelker wrote: I'm frequently hearing (not only in these threads) that pf is thought to be buggy with regards to window scaling that and that it can cause problems like those described in these threads. Problems like this occur when people use pf to filter statefully, but do not create state on the initial SYN of TCP connections. Window scaling support is negotiated in the SYN and SYN+ACK packets of connection establishment. pf can only support it properly when both these packets are seen and associated with a state entry (i.e. you create state on SYN and SYN+ACK matches the state). If the ruleset passes the SYN packet statelessly (without 'keep state'), but then creates state on the returning SYN+ACK, the state entry has missed the window scaling negotiation. There were a couple of reports of people who managed to do this, all by mistake (if you intend to create state, you most certainly want to do it on the initial packet). Most people would call these misconfigurations (buggy rulesets), but some people feel that only a buggy program would allow a user to make such a mistake, hence pf must be buggy. Hence the myth of pf is buggy wrt window scaling. I suggest these people don't use tools with sharp edges, they could hurt themselves ;) If you want a simple way to ensure this is not happening with your ruleset, make sure that a) there is a default block policy b) all 'pass' rules that can match TCP have 'flags S/SA' c) all 'pass' rules have 'keep state' And, no, other packet filters have no magical way of deducing the proper scaling factors if they missed the handshake. If they don't stall connections in this scenario, it just means they are using sloppy TCP window restrictions or don't check TCP sequence numbers at all. We are proud of our strict window checks :) Daniel
Re: anchors - weirdness
On Thu, Jun 29, 2006 at 11:37:41AM +1000, David Diggles wrote: -quote section from pf.conf anchor test/* load anchor test from /etc/pf/anchors/test -quote section from pf.conf (this pf.conf has a default block set) anchor test/* will cause evaluation of all sub-anchors of test, but not evaluation of the rules in anchor test itself. If you want to evaluate both the rules in test as well as the rules in all sub-anchors of test, you'll need anchor test anchor test/* - quote /etc/pf/anchors/test - anchor test.000 load anchor test.000 from /etc/pf/anchors/test.000 - quote /etc/pf/anchors/test - Since you have calls from test's ruleset to the sub-anchors, what you probably intended, in the main ruleset, was a call to test's rules, which is anchor test instead of anchor test/* There was a bug that caused anchors defined from sub-anchors with load anchor statements to get defined directly in the root, and not relative to the position of the anchor defining them. This was fixed in OpenBSD just a couple of weeks ago with http://www.openbsd.org/cgi-bin/cvsweb/src/sys/net/pf_table.c.diff?r1=1.67r2=1.68f=h http://www.openbsd.org/cgi-bin/cvsweb/src/sbin/pfctl/parse.y.diff?r1=1.497r2=1.498f=h This isn't in FreeBSD (or OpenBSD -stable) yet, but it probably makes sense to pull it in. As a workaround, you can use absolute paths for load anchor statements in sub-anchors, like load anchor /test/test.000 from /etc/pf/anchors/test.000 Weirdness number 2... can't destroy anchors? Anchors are automatically destroyed/removed when they contain no rules (either translation or filter rules) AND contain no sub-anchors AND are not referenced by any explicit anchor ... calls # pfctl -a test.00 -F all pfctl: Anchor or Ruleset does not exist. why does this not exist??? Note the typo (test.00 vs. test.000) Daniel
Re: Open BSD 3.9 Pf issue with email with attachments.
On Tue, Jun 27, 2006 at 09:17:18AM +0530, Ajith Kumar wrote: I had modified the entry like this pass in quick log on fxp0 from any to x.x.x.x keep state flags S/SA #1 pass out quick log on fxp1 from any to x.x.x.x keep state flags S/SA #2 pass in quick log on fxp1 from x.x.x.x to any keep state flags S/SA #3 pass out quick log on fxp0 from x.x.x.x to any keep state flags S/SA #4 ( fxp0 is internal interface card. fxp1 is external interface card) where x.x.x.x is ip address of mail server.Still I am not able to send mail with big attachments. I am able to send and receive other mails. The test with disabling pf was a good one. Next, enable pf but load an empty ruleset (pfctl -Fa, pfctl -e) and retry. Still working? If so, load only the four rules you pasted above, retry. Still working? If so, take a good look at your other rules. The difference between your real ruleset and the four rules quoted above must explain the breakage. Post the real ruleset if you can't spot it. If any other rule matches and creates state (for those TCP connections), make sure all states are created on the initial SYN only. If connections break with an empty ruleset or just the four rules above, enable debug logging (pfctl -xm), reproduce the problem, then check /var/log/messages for entries from pf. Post them. Run pfctl -si before and after reproducing the problem, what counters are increasing? Post both outputs. Daniel
Re: rule conversions
On Tue, Jun 06, 2006 at 04:01:09PM +0200, Antoine Jacoutot wrote:
How would you translate the following ?
= deny ip from any to 145.238.0.0/255.255.0.255
The parser doesn't yet support such netmasks, you'll have to manually
expand to all combinations, using a table makes evaluation faster:
table foo const { 145.238.0.0, 145.238.1.0, 145.238.2.0, ...
145.238.254.0, 145.238.255.0 }
block to foo
Daniel
Re: Recursive macro expansion problems
On Fri, May 26, 2006 at 11:09:51AM +0530, Siju George wrote:
host1 = 192.168.1.1
host2 = 192.168.1.2
all_hosts = { $host1 $host2 }
is that an error ?
No, it's correct if you want to use $all_hosts in a rule like
pass from $all_hosts to any
But it won't work as right-hand-side of another macro definition, like
from_all_hosts = from $all_hosts to any
pass $from_all_hosts
Daniel
Re: Recursive macro expansion problems
On Tue, May 23, 2006 at 03:31:46PM -0700, andrew fresh wrote:
host_list = { $hosts }
port_list = { $ports }
Try adding
q_host_list = '{' $hosts '}'
q_port_list = '{' $ports '}'
then replace
end_03 = proto tcp from $host_list to any port $port_list
with
end_03 = proto tcp from $q_host_list to any port $q_port_list
The rule is that when a macro is used to define another macro, it should
contain quotes (as the right-hand-side of a macro definition is a
concatenation of strings), while a macro used in a rule definition
should not.
Like
pass from { 10.1.2.3 10.2.3.4 } to any
is not a host list, but a single string, interpreted as a host name.
And
macro = { 10.1.2.3 10.2.3.4 }
is not a valid macro definition, because the right-hand-side is not a
string (or a sequence thereof), but interpreted as tokens.
It's neither like shell variable expansion, nor like C #defines. But
something else entirely. I keep hearing it's supposed to be like that :)
Daniel
Re: Logging (lack of), driving me nuts
On Thu, May 18, 2006 at 04:10:22PM -0400, Chad M Stewart wrote: For some reason I'm not seeing every blocked packet logged. Why do you expect every blocked packet to get logged? Not all your block rules use 'log'. Packets could easily get blocked by a rule without 'log', hence get blocked but not logged. Only the last matching rule matters. If the last matching rule has 'log', the packet is logged. If the last matching rule doesn't, it isn't. Maybe the packet matched another rule, earlier, which had the 'log' option. But that is irrelevant. The flag does not 'stick' when a rule matches, or similar. If you really want every blocked packet logged, add 'log' to every single block rule. Daniel
Re: Logging (lack of), driving me nuts
On Thu, May 18, 2006 at 04:38:44PM -0400, Chad M Stewart wrote: # cat /etc/pf.conf |grep -v ^# |grep block set block-policy return block in log all block log quick inet proto tcp from ssh-denied to $ssh_servers port ssh label accessive-ssh Ok, so all your block rules do have the 'log' option. That rules out the most obvious explanation. There are other causes for a packet to get dropped by pf, besides matching a block rule. The packet could be invalid (like, be too short, or have invalid checksums), or the source-tracking limits (which I see you're using) are kicking in, or the packet might almost (but not quite) match a state entry. In these cases, a ruleset evaluation is not needed (as the packet is dropped unconditionally), hence it can't match any block rule (with or without 'log' option). But all such cases increment various counters. Run pfctl -si, note the output, reproduce the blocked packet that doesn't get logged. Then run pfctl -si again. Compare the two outputs. Are any counters increasing? At least a packet and byte counter should increase, but do other counters increase as well? Reproducably, correlating with the blocked packets that aren't logged? Also, explain what packet the test with telnet you described is expected to cause, in what direction on what interface, with what source and destination addresses and ports. Even better, verify that expectation and capture the first SYN packet of that telnet test with tcpdump and post it. Incoming packets are seen by tcpdump even when pf blocks them. If you don't see the packet with tcpdump at the pf box, it might simply never arrive there, and pf isn't blocking it at all. Daniel
Re: Logging (lack of), driving me nuts
On Thu, May 18, 2006 at 05:24:28PM -0400, Chad M Stewart wrote: Status: Enabled for 0 days 02:05:34 Debug: Urgent The differences in the pfctl -si outputs look like it MUST be a block rule without 'log' matching those packets, after all. The grep in your /etc/pf.conf might have been incomplete for two reasons: a) you're using 'antispoof', which expands to multiple block rules, depending on the networks assigned to the interfaces. run the grep again, but against pfctl -sr output instead of /etc/pf.conf. does any of those block rules not have 'log', and could it match your packet? b) you're using anchors. that means the block-without-log rule we're looking for might be hiding in an anchor. either grep pfctl -a ... -sr for all anchors, or temporarily disable the anchor hooks (comment out the 'anchor' lines in /etc/pf.conf, and reload the ruleset) and check if the problem persists. Your ruleset is non-trivial in size, and uses references not defined within itself (addresses of interfaces, tables), so it's not easy to manually evaluate it given only the ruleset. Even if it's tricky on a production firewall, the best approach in debugging is to narrow the ruleset down. I'd (briefly) disable all anchors, then (temporarily) clear all tables. That would remove a significant portion of the possible reasons. Daniel
Re: Logging (lack of), driving me nuts
On Thu, May 18, 2006 at 04:38:44PM -0400, Chad M Stewart wrote: set skip on lo0 set skip on $pfsync_if# might not want this These two lines don't add up, the second one replaces the first, so lo0 is not really skipped. Use a single set skip line, listing all interfaces to be skipped at once. block in log all antispoof for $all_if inet antispoof after a default block is superfluous. It expands to non-quick block rules. Any packet that could possibly match them has already matched your default block rule above. The expanded rules also don't have the 'log' option. Try and remove the antispoof line and reproduce. Your first post included the following rule, which I assume is from the antispoof expansion: block drop in on ! carp0 inet from 24.97.84.32/29 to any This would match your TCP SYN from 24.97.84.33, and block it without logging, assuming it comes in on any interface but carp0. Does pf see incoming packets on carp0 again incoming on the real interface associated with carp0? Try tcpdump'ing on that real interface. Daniel
Re: Logging (lack of), driving me nuts
On Thu, May 18, 2006 at 06:32:37PM -0400, Chad M Stewart wrote: Perhaps it is just my tired brain but it seems strange that in other rules carp0 is used as the incoming interface. Maybe Ryan can comment, from http://www.countersiege.com/doc/pfsync-carp/ When writing the rest of the pf ruleset, it is important to keep in mind that from pf's perspective, all traffic comes from the physical interface, even if it is routed through the carp address. However, the address is of course associated with the carp interface. Therefore, in the interface context, such as pass in on $extif ..., $extif would be the physical interface, but in the context of from $foo or to $foo, the carp interface should be used, as it's being meant in the address context. Does this mean 'antispoof for carp0' is generally (always?) a mistake? As Chad showed, packets are seen by tcpdump on carp0, are they also filtered by pf on carp0 (i.e. twice, on carp0 and the real interface)? Has this changed in the past? Daniel
Re: pfstat 2.0
And after some head scratching one realizes that endianness matters with database keys, and macppc isn't the best platform to spot that. ;) Update on http://www.benzedrine.cx/pfstat-2.2.tar.gz MD5 (pfstat-2.2.tar.gz) = 49ce4a028dfa00b65fccbabc96836c97 * fix endianness issues in database (basically things just didn't work on i386, producing weird 'blocky' graphs, changing between invokations, etc.) * add database truncate (-t) operation, explain how compression works in the man page (http://www.benzedrine.cx/pfstat.cat8) * clean up debug messages. now -q/-p don't produce any output to stdout, unless -v (or -vv) is used. * show queues and global counters in the example config. (http://www.benzedrine.cx/pfstat.html, /pfstat.conf) Daniel
Re: pfstat 2.0
On Tue, May 16, 2006 at 06:15:13PM +0200, Jonas Davidsson wrote:
Im getting some very strange numbers out of this now, number of states
for example, are shown to be around seven thousand in the graph,
while pfctl shows only 680. Most other values are just plain off.
That's via the remote TCP protocol, right? I think I botched the arg -
handling there. Can you try with the patch below?
Daniel
Index: pfstat.c
===
RCS file: /pub/cvsroot/pfstat/pfstat.c,v
retrieving revision 1.23
diff -u -r1.23 pfstat.c
--- pfstat.c15 May 2006 07:16:18 - 1.23
+++ pfstat.c16 May 2006 16:54:57 -
@@ -104,7 +104,7 @@
}
for (i = 0; i maxcol; ++i) {
if (cols[i].type != type || cols[i].idx != idx ||
- strcmp(cols[i].arg, arg))
+ (strcmp(arg, -) strcmp(cols[i].arg, arg)))
continue;
cols[i].val = val;
}
Re: pfstat 2.0
Thanks for the feedback! An update is on http://www.benzedrine.cx/pfstat-2.1.tar.gz MD5 (pfstat-2.1.tar.gz) = 82bcef47cca25b3ff28a4628ccaee26b * properly count line numbers when parsing configuration file, so syntax error messages refer to the appropriate line * support # comments in the configuration file * don't blindly multiply all values by 8, but use a configuration option 'bps' to indicate a value should be converted from bytes/s to bits/s. * support 'avg|min|max' option for graphs. when one pixel of the graph represents multiple measurements, draw the average (default, properly weighted based on time, too ;), minimum or maximum of those measurements. * support host name as -r argument, thanks to Matthias Kilian (yes, it's not finished yet ;) There's no database locking yet, so make sure you don't run multiple instances of pfstat concurrently (like, querying every minute, and also plotting every */x minutes from crontab, or such). Daniel
pfstat 2.0
Here's a major update to pfstat. The most important changes:
* Add a small daemon 'pfstatd' which listens on a TCP port and, when
connected to, sends the statistical pf data in plain text to the
peer. This program has no dependancies (gd, X11, etc.), so it can
be easily installed on a small firewall, and the main 'pfstat'
program can be installed on a different machine. Many users
complained that they had to install xbase39.tgz on a firewall,
this is now no longer required. Querying the local packet filter
through /dev/pf (without running pfstatd) is still supported and
the default.
* Switch the data storage from the simple text file to a DBT_BTREE
database (dbopen(3), no additional dependancies). Do MRTG-style
compression on insertion, so plotting graphs over large time periods
does not take longer anymore. Graph plotting is very fast now.
Expiry/purging of older data will be supported soon.
* Support for arbitrary interface and queue statistics. Label support
coming soon. This requires that values that should be stored must
be declared in advance (i.e. storing all values is possible, but
storing only user-specified values keeps the database smaller).
* The pfstat.conf syntax has therefore changed slightly, and example
is
collect 1 = interface sis0 pass bytes in ipv4 diff
collect 2 = interface sis0 pass bytes out ipv4 diff
image /var/www/htdocs/benzedrine.cx/pfstat.jpg {
from 72 hours to now
width 1000 height 400
left
graph 1 in bits/s color 0 192 0 filled
right
graph 2 out bits/s color 0 0 255
}
The 'collect' lines declare units to be collected, and the 'graph'
line reference units through the declaration numbers (1 and 2, in
the example).
* Fix several bugs. In particular one that affected scaling of values.
Before, the axis were sometimes wrongly labeled (like the maximum
value was labeled as '40 mbits/s' when it was really much higher.
The patch below is against the 3.9 ports tree. Apply it (to
ports/net/pfstat), delete the existing patches subdirectory.
NOTE: this won't work as-is on a pre-3.9 system. While it's not
complicated to get building there, I'd rather get feedback from 3.9
users at this time.
If someone with ports-foo has any suggestions about how to split pfstatd
into a separate package (so a package containing only pfstatd could be
installed on firewalls without any dependancies), please speak up ;)
Daniel
Index: Makefile
===
RCS file: /cvs/ports/net/pfstat/Makefile,v
retrieving revision 1.11
diff -u -r1.11 Makefile
--- Makefile24 Oct 2005 05:09:59 - 1.11
+++ Makefile14 May 2006 10:11:09 -
@@ -2,8 +2,8 @@
COMMENT= packet filter statistics visualization
-DISTNAME= pfstat-1.7
-PKGNAME= ${DISTNAME}p1
+DISTNAME= pfstat-2.0
+PKGNAME= ${DISTNAME}
CATEGORIES=net
MASTER_SITES= http://www.benzedrine.cx/
@@ -24,5 +24,7 @@
do-install:
${INSTALL_PROGRAM} ${WRKSRC}/pfstat ${PREFIX}/bin
${INSTALL_MAN} ${WRKSRC}/pfstat.8 ${PREFIX}/man/man8
+ ${INSTALL_PROGRAM} ${WRKSRC}/pfstatd/pfstatd ${PREFIX}/bin
+ ${INSTALL_MAN} ${WRKSRC}/pfstatd/pfstatd.8 ${PREFIX}/man/man8
.include bsd.port.mk
Index: distinfo
===
RCS file: /cvs/ports/net/pfstat/distinfo,v
retrieving revision 1.9
diff -u -r1.9 distinfo
--- distinfo5 Jan 2005 17:14:57 - 1.9
+++ distinfo14 May 2006 10:11:09 -
@@ -1,4 +1,4 @@
-MD5 (pfstat-1.7.tar.gz) = bd58864b56774aa10aa763258de8b625
-RMD160 (pfstat-1.7.tar.gz) = d15ebd14a42ddbdacf326b4ad59c2fcf4f7d8b07
-SHA1 (pfstat-1.7.tar.gz) = 7bd125039c130a56a4dda3040d309d0f445d88b5
-SIZE (pfstat-1.7.tar.gz) = 9610
+MD5 (pfstat-2.0.tar.gz) = c15416c16062da39500bb7822c71fcf1
+RMD160 (pfstat-2.0.tar.gz) = ae6d7f2db52df21282e05e45d0c8458ac0865433
+SHA1 (pfstat-2.0.tar.gz) = 798449440e1c8ac9a9eb33792a7fd49afee3def3
+SIZE (pfstat-2.0.tar.gz) = 16625
Index: pkg/PLIST
===
RCS file: /cvs/ports/net/pfstat/pkg/PLIST,v
retrieving revision 1.2
diff -u -r1.2 PLIST
--- pkg/PLIST 15 Sep 2004 18:17:44 - 1.2
+++ pkg/PLIST 14 May 2006 10:11:09 -
@@ -1,3 +1,5 @@
@comment $OpenBSD: PLIST,v 1.2 2004/09/15 18:17:44 espie Exp $
bin/pfstat
+bin/pfstatd
@man man/man8/pfstat.8
[EMAIL PROTECTED] man/man8/pfstatd.8
Re: pfsync / load balancing
On Mon, May 08, 2006 at 05:58:08PM +0300, Hisham Mardam Bey wrote: Can this be achieved using pfsync? If so, what do I need to do to get this working? If not, can pfsync be extended to allow for this or should we look into something different altogether? This currently won't work. pfsync does only synchronize state table entries, not rules. Since the firewalls can have different rules, no attempt is made to associate a state entry with a particular rule of the recipient's ruleset. Instead, such state entries are associated with the default pass rule. Because of that, many rule options (like source tracking in your case) are lost on the recipient node. Both firewalls would insert the other's state entries, but those would not be counted towards the limit. Ryan had plans to work on this, but I don't know about any progress. It's not something trivial, and may only work with completely equal ruleset (or at least well-defined mappings between rules on both nodes). Even if it did work, in your case it's not clear whether this would be a good idea. If you synchronize state entries for the sole purpose of increasing source tracking counters, you still get fully functional state entries on the recipient, with all the implications, i.e. those states would ALSO allow traffic on the recipient, which may or may not be a problem. Also, the limit would never be perfectly precise. Say, there's is one connection left to be openend, and both firewalls get one new connection at the same time. There is no (networked!) locking or such, so they would just both create a state entry, sync them, and end up exceeding the limit by one. If somewhat inprecise limits are acceptable, the most simple solution is to only guarantee X + Y = 2*N by limiting both firewalls to N connections. You guarantee the client N connections (no matter where they flow through), and count the difference between N and 2*N as imprecision. That might sound awfully imprecise at first, but in most cases, is good enough. Maybe you can explain why you need precisely N, what value N would typically have, and why 2*N would be unacceptable. Daniel
Re: Transparent Load Balancing Gateway
On Thu, May 04, 2006 at 01:46:59PM +0300, Hisham Mardam Bey wrote: I have an update on the situation. Here's what I did: [client]--[loadbal]--[my 2 backends]-[samba server] Doing this with only one interface (and bouncing incoming packets out through the same interface) sounds like asking for a headache. What happens in your case is that the pf box doesn't see the reply packets, i.e. it only sees one half of the connection (client to server). The state entries don't advance properly in this case, pf doesn't see the server advertise window sizes, etc. and starts to block packets from the client to the server. The problem is similar to the one described on http://www.openbsd.org/faq/pf/rdr.html#reflect i.e. the server sends its replies directly to the client, as the client is on the same network and the server has learned its MAC address. If you want to filter statefully, you have to make sure pf sees all packets (both directions) of connections. If and how that's possible in your case, is, well, YOUR headache. I'd not try bouncing packet out with a single interface setup, but use two interfaces, possibly bridging. ;) Daniel
Re: IP alias with OpenBSD
On Mon, May 01, 2006 at 08:26:37PM -0400, jared r r spiegel wrote:
my5addrs=1.2.0.1 1.2.0.2 1.2.0.3 1.2.0.4 1.2.0.5
nat on $ext - { $my5addrs }
i've never dealt personally with multiple egress IPs, but that
syntax passes the parser
Yes, that should work. pf will automatically cycle through those
addresses when you establish multiple non-TCP/UDP/ICMP connections to
the same external host. You don't need special syntax to enable that.
When you already have an ongoing VPN connection from, say, 10.1.2.3
to 62.65.145.30 NATed to 1.2.0.1, and then open another one (from
any other 10/8 to 62.65.145.30), it will also try to use 1.2.0.1 as
replacement address, note the conflict with the existing state entry,
then try the next one (1.2.0.2). Only when you exhaust all four
addresses (try to establish a fifth concurrent VPN connection to
62.65.145.30), there will be a state insertion failure.
Daniel
Re: PF inadequacy: queue download
On Tue, May 02, 2006 at 02:32:31AM -0700, [EMAIL PROTECTED] wrote: I'm not demanding anyone do anything, I'm not trolling, I just want to get this acknowledged as an area for potential development. Why everyone's so resistant to this is beyond me. That this is the only extra feature I'd like to have in pf I think reflects pretty well on pf. No problem. I, dhartmei@, in my role as OpenBSD and pf developer[1], hereby acknowledge inbound queueing in pf/altq as an area of potential development. Several users, on several occasions, have shown interest in this feature, and so far no strong technical reason has been named why it should not be added. I vouch that, if and when development in that area has occurred, I'll do my best to review, test, and reach a consensus to commit that work. I'm not resistant to this feature, I just don't want to implement it myself. I'd rather do laundry and watch a movie. That doesn't mean I discourage anyone else from doing it. Go ahead, knock yourself out. It's not a trivial task, you'll earn respect. Is that official enough? Can we move on now? Daniel [1] http://www.openbsd.org/cgi-bin/cvsweb/src/sys/net/pf.c?rev=1.1content-type=text/x-cvsweb-markup
Re: PF inadequacy: queue download
On Sat, Apr 29, 2006 at 04:26:19PM -0700, [EMAIL PROTECTED] wrote: I'd have a look at this problem myself, but I'm not good with C. I was hoping there was some sort of todo list I could petition this to be added too, because lots of people here seem to agree this is pf's (and ALTQ's) worst problem. I'm not sure if this thread is a statistically relevant measurement. If the main reason why the workaround with the additional machine is problematic is hardware and electricity cost, and there are 'lots of people' that care deeply about this, and each of them would save, say, $500 if the feature were implemented, it shouldn't be hard to collect $20k (two man-months, a reasonable estimate, I'd say) and send that to Theo with the request for implementation during a hackathon. Or, to turn the argument around, if the entire support for the feature is five people offering lollipops and eternal gratitute, developer time is probably better spent elsewhere. By all means, try it. There are more readers (and developers) on the misc@ and tech@ lists, so I'd start there. Daniel
Re: PF inadequacy: queue download
On Sat, Apr 29, 2006 at 06:05:47AM -0700, [EMAIL PROTECTED] wrote: I know this is possible because IPFW with dummynet doesn't have any problems. If everyone loves PF because of its elegance why can't it do something as simple as queue download traffic? http://lists.freebsd.org/mailman/htdig/freebsd-pf/2005-November/001657.html Daniel
Re: PF inadequacy: queue download
On Sat, Apr 29, 2006 at 05:10:40PM +0200, Stanislaw Halik wrote: I can speak for myself - I can't afford both the hardware and the electricity bill for a separate machine. Maybe downstream limiting isn't very robust, but IMO is the biggest thing pf/altq lacks. What I tried to express in the last paragraph of the referenced mail was that it's not pf that's lacking anything, but altq. There simply are no input queues with altq. If you added those queues to altq, you might not even have to change a single line in pf to get them used. While there are now ties between pf and altq (pf classifying packets for altq, and pfctl setting up queues), that doesn't mean the core of altq is maintained by the people who maintain pf. It's in separate source files, and if you compare the respective CVS histories, you'll see the difference. Question's of the form why hasn't this been done sound kind of weird to me, almost as if they were rhetorical. Assuming you haven't donated blood in the last month, why haven't you? Are you really interested in the petty excuses humans have for not doing things? :) Daniel
Re: PF with subanchors possible bug
On Fri, Apr 28, 2006 at 12:42:13PM +0400, Boris Polevoy wrote: Is it bug or not? Yes, it looks like a bug. Or, more than one, actually. I assume what you expect the sequence to do is the same as # echo 'anchor external' | pfctl -f - # echo 'anchor internal' | pfctl -a external -f - # echo 'pass all' | pfctl -a external/internal -f - (leaving out the table, which isn't really relevant, I think) i.e. you expect internal to be nested within external, like # pfctl -vsA external external/internal # pfctl -sr anchor external all # pfctl -a external -sr anchor internal all # pfctl -a external/internal -sr pass all Your patch fixes that. But there is another one, when doing pfctl -a external -f, it doesn't prefix the (relative!) paths within the input with the anchor specified through -a. Therefore, when I do the same (it should be the same, IMO) with files, like # cat x anchor external load anchor external from y # cat y anchor internal load anchor internal from z # cat z pass z # pfctl -f x # pfctl -vsA external external/internal internal # pfctl -a external/internal -sr [ empty ] # pfctl -a internal -sr pass all the rule loaded from z is not placed into the right anchor (external/internal), but a second anchor (internal) is created for it. I'll have to find the right place to fix THAT, too. Daniel
Re: Packet going out wrong interface in spite of route-to
On Sat, Apr 22, 2006 at 05:44:07AM +, George Pontis wrote: I am having a hard time routing a reply out the correct WAN interface. In spite of using a reply-to and creating state, the packet is routed to the other interface and then dropped. The routing to the interface with the default gateway is normal and expected. What should happen, before the packet gets passed out on that interface, is that it matches the previously created state, and the reply-to option (of the matching state entry) should re-route the packet through the desired interface. The initial routing decision is made by the TCP/IP stack outside of pf. pf's re-routing (in this scenario) applies afterwards. Is your state entry if-bound? In that case, it simply won't match the packet on the default interface, and no re-routing takes place. If so, make the state entry floating (or bind it to both interface, for instance by creating an interface group consisting of both interfaces, and bind the state to that interface group). You can bypass the TCP/IP stacks normal routing, by using a reply-to option on the rule on the _internal_ interface. Replies will match that state immediately when the come in on the internal interface, and pf then routes them out the desired external interface without the TCP/IP stack getting to (attempt to) forward them and using the default route. A problem with that scenario might be that you can't easily decide which connections going out on the internal interface should use reply-to, as when they go out there, the information about which external interface the initially came in on is already lost. You can use 'tag' on the rule passing in on the non-default external gateway and match with 'tagged' on the internal interface rules. But even in this scenario, you probably need a state entry on the external interface that's not if-bound. I also have the catch-all lines at the end of pf.conf that are supposed to switch packets to their native interfaces. pass out on $ext_if2 route-to ($ext_if1 $ext_gw1) from $ext_if1 to any flags S/SA keep state pass out on $ext_if1 route-to ($ext_if2 $ext_gw2) from $ext_if2 to any flags S/SA keep state The first packet trying to pass out is not a SYN packet, but a SYN+ACK reply, so it doesn't match the 'flags S/SA' part of those rules, and the rules don't match. Removing the 'flags S/SA' part won't solve the problem, either, as then you'd be creating a second state entry, which is not based on the initial SYN, but the first reply. Which can break connections due to pf missing the TCP window scaling negotiation in the handshake. Daniel
Re: Same rule, different results
On Sat, Apr 22, 2006 at 03:37:35PM -0700, Allie Daneman wrote: Apr 22 14:53:52.935466 rule 18/(match) pass out on xl0: 24.XX.XX.X.50599 216.XXX.XX.XX.53: [|domain] Apr 22 14:53:53.015842 rule 13/(match) block in on xl0: 216.XXX.XX.XX.61144 24.XX.XX.X.50599: udp 116 [tos 0x20] The query is to port 53, but the reply isn't coming from port 53, but from port 61144. I think that's technically legal for DNS, but has become mostly an obscurity today, because it breaks on almost any firewall (not just pf). I.e. most DNS servers don't do that anymore, and you have found one that still does. I don't know why it was made legal in the first place, maybe an existing vendor insisted that he couldn't afford to modify his unmaintainable code to do a bind(2) call ;) There's no way to match that reply to the state entry (as matching is based on port numbers), so you'd have to basically pass all such replies in statelessly (opening UDP up wide open). Or just screw that DNS server. Daniel
Re: Reloading NAT clears skip flag on interfaces
On Fri, Apr 21, 2006 at 11:38:11AM -0700, Jon Simola wrote: This is totally repeatable, and keeps biting me. Is this a bug or feature? I think it's expected that -N only reads and honours NAT rules, and ignores anything else, including any options like 'set skip'. The man page is clear on that, IMO. What isn't so clear is whether it should first clear (reset) all options before loading the new NAT rules. Basically, any invokation that changes something first resets the options. You'll have to add the -O option to the invokation to re-parse and reload the options after the (implicit) reset. -N isn't special in that regard, -R behaves the same. Whether the man page is clear about the result of using a combination of -N, -R, and -O simultanously, I'm not sure. The implict reset was added intentionally, I think Theo feels strongly about it. The idea is that you don't get different results from incremental reloads in different orders, i.e. reparsing the file produces one well-defined state no matter what the state was beforehand. The addition of -m muddied the issue further, I'm not sure what it is expected to do when combined with -N. You could argue the case that -N (and -R) shouldn't cause an implicit option reset, but only a plain -f should. But that's not up to me to decide, and it's not a simple implementation buglet, but at least somewhat intentional ;) Personally, I always reload the entire rules with -f without -N/-R, so this hasn't bothered me yet. HTH, Daniel
Re: Reloading NAT clears skip flag on interfaces
I guess what you want is simply this Index: pfctl.c === RCS file: /cvs/src/sbin/pfctl/pfctl.c,v retrieving revision 1.244 diff -u -r1.244 pfctl.c --- pfctl.c 17 Nov 2005 20:52:39 - 1.244 +++ pfctl.c 21 Apr 2006 20:00:46 - @@ -1929,7 +1929,8 @@ err(1, %s, rulesopt); } } - if ((rulesopt != NULL) (!*anchorname)) + if ((rulesopt != NULL) (loadopt PFCTL_FLAG_OPTION) + !anchorname[0]) if (pfctl_clear_interface_flags(dev, opts | PF_OPT_QUIET)) error = 1; i.e. only reset the interface flags if we're subsequently loading options. If we're not loading any options, there's no reason to reset them beforehand. Note that plain -f will set that flag, as will -O and any combination of -O with -N/-R. For me, this makes sense. This allows stuff like pfctl -sn | sed some-replacement | pfctl -Nf - without affecting options. Daniel
Re: pfctl: DIOCADDALTQ: Invalid argument
On Tue, Apr 18, 2006 at 10:27:53PM -0700, [EMAIL PROTECTED] wrote:
altq on $int_if bandwidth 100Mb cbq queue { ether, nattraffic }
queue ether bandwidth 70% cbq
queue nattraffic bandwidth 30% cbq (default) { out_, in_ }
queue out_ bandwidth 64Kb cbq { out_me, out_others }
queue out_me bandwidth 80% cbq
queue out_others bandwidth 20% cbq ( default )
queue in_ bandwidth 128Kb cbq { in_me, in_others }
queue in_me bandwidth 80% cbq
queue in_others bandwidth 20% cbq ( default )
You have multiple queues with 'default', which is invalid. pf.conf(5)
says
default Packets not matched by another queue are assigned to this
one. Exactly one default queue is required.
That literally means you must have exactly one queue as default, not one
queue per level or sub-tree or such, just one queue in the entire tree
of queues.
I don't know what your intention is with using 'default' on 'out_others'
and 'in_others', you'll have to assign to those two using filter rules
explicitely. Anything not assigned to any queue by filter rules will go
to the one default queue.
Daniel
Re: is there a way to say from or to some host?
On Wed, Apr 19, 2006 at 10:30:47AM -0400, Roy Morris wrote: huh? - I must be misreading/understanding the question pass out on $some_if from x to z proto tcp keep state That's src equals x AND dst equals y. What Travis is asking for is src equals x OR dst equals x. It can be done in two rules, like pass from x ... pass to x ... and you could make the parser expand a new keyword (like host x) to such a pair of rules. Doing it in one rule is not as simple, you'd need to add a flag to (in-kernel) rules which toggle between combining the two criteria AND or OR. I don't think that's worth it ;) Daniel
Re: pf and ie
On Mon, Apr 10, 2006 at 06:28:24PM -0400, James Nachlin wrote: I'm having a strange situation where I'm getting back errors when connecting to a web server (lighttpd) from IE, which I do not get from firefox and I don't get connecting directly, not through the pf firewall. To the client, this appears as slow connections or dropped connections. Looking at the traffic with Ethereal, the main difference seems to be the presence of tons of packets with the RST flag set. The problem will probably be obvious to someone who knows more about TCP/IP. The client is opening a lot of connections (from unique source ports) to the server, after the handshake, it sends the HTTP request, half-closes the connection (sending a FIN, telling the server it will send no more data, but only read the replies). The server sends an empty ACK back (which I'm not sure is typical, but it looks valid), then starts to send the HTTP reply. Often, the client immediately sends a RST after the first data packet. It's not obvious why the client would do that, if the RST really is generated by the client. Looks like a problem on the client. Since it affects different browsers, maybe a network setting in the OS. Try capturing traffic at the client (or close to it, at least on the firewall interface closest to the client). If it's really the client generating the RSTs, and showing the replies it gets up to that point are valid, it's the clients' fault... All your captures show this behaviour, I'm not sure why you'd experience different behaviour from the different browsers, maybe they show cached contents differently in error cases. Daniel
Re: clarification of NAT behavior
On Fri, Apr 07, 2006 at 12:04:23PM -0400, Gabriel Wachman wrote: If NAT translation happens BEFORE any filter rules are evaluated (see http://www.openbsd.org/faq/pf/nat.html), then wouldn't it be true that an outbound packet from the internal network will be seen by the filtering engine as a packet with source IP of the firewall? Yes, a packet from the internal host going out on pf's external interface will have its source address translated already when it is filtered. For incoming packets of the same connection, the point is moot. Since there always is a state entry for translated connections, the incoming replies will match that state entry and pass without any ruleset evaluation. Hence, it's irrelevant whether the ruleset would be evaluated before or after the reply's destination address is translated back, because it isn't evaluated for such packets. To me, that clearly indicates that the filtering engine sees only the post-translated packets, with no idea of the contents of the pre-translated packets. Therefore the filtering engine should only see the translated source IP and destination IP of outbound and inbound packets, respectively, from the NAT'ed internal network. On the external interface, that is true. Each packet is filtered on both interfaces, the internal one and the external one. On the external one you'll be seeing already translated packets, on the internal one not-yet (or back-)translated packets. So, if you want pf to make decisions based on the untranslated source address, you can do so on the internal interface. You can either block based on address there, or add a tag to the packet, which will be recognized by rules on the external interace. The only question from you I seem to be able to identify is is the FAQ correct, and the answer is yes. If I missed the real question, or you just didn't ask it, explain :) Daniel
Re: contributions to pf FAQ/manpage whatever
On Fri, Apr 07, 2006 at 09:09:30PM -0500, Travis H. wrote: What would be the appropriate way to submit additions to the PF FAQ and/or pf.conf manpage? Specifically, what is the source format, where can I get the source (for the FAQ, I know where to get the unformatted manpage), and to whom exactly should I send the diffs? The FAQ (its HTML source code) is maintained in the CVS repository[1], you can check out a copy using cvs(1) from one of the public mirrors[2] like $ cvs -d $CVSROOT -q checkout www/faq/pf Then edit the HTML files until they contain the changes you want to make and render properly $ cd www/faq/pf $ vi index.html $ lynx index.html and produce a unified (the -u option matters) diff $ cvs -q diff -u ~/my.diff $ more ~/my.diff Then mail it to the public www@ list or the maintainers directly, who are nick@ and [EMAIL PROTECTED] Note that the lists generally strip MIME attachments, so include the diff in the body of the mail as plain text. The man page is similar, but you'd send the diff to misc@ or tech@, there aren't dedicated maintainers, but if you want to contact someone personally, check the CVS history for someone who has recently commited to the file you have a diff for. If you plan larger changes, submit them in smaller chunks, each containing one particular aspect entirely, those are easier to digest. Daniel [1] http://www.openbsd.org/cgi-bin/cvsweb/www/faq/pf/ [2] http://www.openbsd.org/anoncvs.html
Re: PF and label expansion limitations
On Wed, Apr 05, 2006 at 11:49:12PM +0200, Per-Olov Sjöholm wrote: The PF rule... pass in quick on $EXTERNAL_INT inet from any to $COLOC_IPS_1 label TEST:$dstaddr# keep state Gives a label like TEST:65.45.128.128/25# 230 3099 1511793 1370 148914 1729 1362879 Is there an easy way to do expansion of $COLOC_IPS_1 so that the single rule above give labels like... TEST:65.45.128.128/1# 230 3099 1511793 1370 148914 1729 1362879 TEST:65.45.128.128/2# 230 3099 1511793 1370 148914 1729 1362879 TEST:65.45.128.128/3# 230 3099 1511793 1370 148914 1729 1362879 TEST:65.45.128.128/4# 230 3099 1511793 1370 148914 1729 1362879 TEST:65.45.128.128/n# 230 3099 1511793 1370 148914 1729 1362879 TEST:65.45.128.128/n+1# 230 3099 1511793 1370 148914 1729 1362879 TEST:65.45.128.128/254# 230 3099 1511793 1370 148914 1729 1362879 This so we could measure each customers dedicated server statistics. You mean counters for each individual address within one netblock, as for 65.45.128.128/25 65.45.128.128 65.45.128.129 65.45.128.130 ... 65.45.128.255 ? Note that 65.45.128.0-127 are NOT part of 65.45.128.128/25. Or did you really mean counters for different netblocks with varying width? That's something else, entirely. And /n for n 32 makes no sense for IPv4 :) Daniel
Re: PF and label expansion limitations
On Thu, Apr 06, 2006 at 09:52:34AM -0400, Peter wrote:
Do you know if there is something going on to make this possible?
And today the only way is a rule for each customer IP in pf.conf
then?
Or are there maybe other tools except labels in PF to make this
statistics to
work in an easy way?
I'm just a poor user like yourself. To me, it doesn't sound like a big
change. Maybe Daniel can let us know.
Obviously, if you want per-IP counters, the kernel needs to allocate
memory for each counter per IP. It should be clear that if you want to
have individual counters for 100,000 addresses, you need to allocate a
memory for those 100,000 counters. Since we're talking about a
non-trivial amount of memory there, there's no way pf will automatically
keep such counters by default, on the off-chance that some users will
actually query some of them.
There's two ways to get such counters already. Adding individual rules
per IP is the first one. This requires more memory (which any solution
will), but also makes ruleset evaluation slower. I agree that it's not
an elegant solution for thousands of IP addresses.
The other is address tables. If you change your ruleset to
table coloc_ips_1 const { 65.45.128.128, 65.45.128.129, \
65.45.128.130, ..., 65.45.128.254, 65.45.128.255 }
pass in quick on $EXTERNAL_INT inet from any to COLOC_IPS_1 \
keep state
you get counters allocated and updated for every address in the table,
and you can query them with
# pfctl -t COLOC_IPS_1 -vTs
65.45.128.128/32
Cleared: Tue Mar 14 14:22:32 2006
In/Block:[ Packets: 0 Bytes: 0 ]
In/Pass: [ Packets: 0 Bytes: 0 ]
Out/Block: [ Packets: 0 Bytes: 0 ]
Out/Pass:[ Packets: 0 Bytes: 0 ]
65.45.128.129/32
...
If you want multiple counters for each IP address, like
per-IP-and-protocol counters, you'll need to duplicate the table for
each protocol, i.e. multiple tables containing the same addresses, for
the sake of allocating multiple counters per address:
table coloc_ips_1_http const { 65.45.128.128, 65.45.128.129, \
65.45.128.130, ..., 65.45.128.254, 65.45.128.255 }
table coloc_ips_1_smtp const { 65.45.128.128, 65.45.128.129, \
65.45.128.130, ..., 65.45.128.254, 65.45.128.255 }
...
That's pretty optimal with regards to memory usage. The address itself
is small compared to all the counters associated with it. If you want
separate counters per protocol, you HAVE to allocate the counters per
protocol and address. Allocating the address itself twice is a
relatively small waste.
So, I consider this a sufficiently elegant (existing!) solution, the
only annoying thing is that you have to manually enumerate all IPs
within the netblock.
This could be improved by adding a little syntactic sugar to pfctl,
introducing some optional syntax for table additions, like
table ... { 10.1.1.1, 10.2.2/24*, 10.2.3/24 }
where the '*' means 'don't add the netblock itself, but instead generate
and insert all individual address within that netblock), i.e. the above
table would then contain the entries
10.1.1.1
10.2.2.0
10.2.2.1
10.2.2.2
...
10.2.2.255
10.2.3/24
This doesn't HAVE to be done by pfctl itself, you can generate the list
automatically with jot(1) or similar, but I guess it might be nice.
Anything beyond this, like 'I want to track a whole /8, but I don't have
the memory to pre-allocate 2^24 counters, I want counters allocated on
demand for those addresses actually seen', is NOT a simple change.
Daniel
Re: pf - no memory buffers
On Fri, Mar 31, 2006 at 10:25:14PM -0600, Travis H. wrote: On further experimentation, I am convinced there is a memory leak when using tagging. I would experience net death after 1-3 days of activity. Nothing I could do would free up any space, except for rebooting. If this is on OpenBSD, packet tags (see mbuf_tags(9)) are allocated with malloc(M_PACKET_TAGS) in m_tag_get() and show up in vmstat -m as 'packet tags'. A leak would show as steadily increasing 'InUse' and 'HighUse' there. Daniel
Re: TCP session desyncs
On Thu, Mar 30, 2006 at 11:51:36AM +0200, Fredrik Widlund wrote:
Is this a wscale issue desyncing the session? I'm guessing here, but
does PF set the window to 46 receiving the data push from the server,
while C still believes it's 57927 and sends out 59 bytes? What is
wrong here, PF interpretation of wscale, or the external servers wscale
implementation? Or is this something else completely, like window
calculation? What happens when PF gets a packet with a new win size from
the server but the client sends out data before this has reached it? Is
there a race condition between pkts #4 i both dumps below?
pf should correctly honour the wscale option when it creates state on
the initial TCP SYN. You can check with pfctl -vss, it should show the
right wscale factors in its output.
If you haven't seen it, see the recent thread
Subject: pf: State failure on: 1
http://marc.theaimsgroup.com/?t=11436403663
for the most common explanation of why pf would not honour wscale.
Please enable debug logging (pfctl -xm), and repeat the procedure,
capturing one failing connection from handshake to the point of failure
as you already did. Then check /var/log/messages for any lines from pf
related to this connection ('BAD state' messages, likely). Then post
both.
Daniel
Re: Proper syntax for nesting anchors
On Wed, Mar 29, 2006 at 03:07:10PM -0500, David Steinbrunner wrote: I currently have a working anchor that I would like to split into many anchors. The anchor is meant for the rules related to a table so the parent anchor defines the table and then the child anchors hold different types of blocking and passing rules in each of them. I can't seem to get the rules that are in the nested anchors to get evaluated so I have a strong suspicion that is because they are not linked up to their parent correctly. I have been through some many variations of how this could be done that I don't even know which has been my best attempt or even if I have had a best attempt ;P. Hopefully the above description along with the following diagram will be enough for someone to figure out what I am looking for and provide a small example of how it is done. pf.conf | \- main_anchor | |- first_sub_anchor | |- second_sub_anchor | \- third_sub_anchor First, anchors are not evaluated just because they exist. You always need a 'call' to get them evaluated. The main ruleset is the only thing that gets evaluated by default. Hence, you need at least one call from there. You can either call all your anchors from the main ruleset explicitely, like pf.conf anchor main_anchor anchor main_anchor/first_sub_anchor anchor main_anchor/second_sub_anchor anchor main_anchor/third_sub_anchor or use a wildcard to evaluate all, like pf.conf anchor main_anchor anchor main_anchor/* or you can use indirect calls, i.e. have pf.conf just call main_anchor, and do further calls from there, like pf.conf anchor main_anchor main_anchor anchor * More examples can be found in pf.conf(5) and authpf(8). Daniel
Re: pf: State failure on: 1
On Thu, Mar 30, 2006 at 06:34:02PM +, George Pontis wrote: For a rule that matches both UDP and TCP packets, is flags S/SA safely ignored for UDP ? Yes, the rule matches UDP packets as if the flags S/SA wasn't there. Daniel
