Re: NMAP problem with PF
Loïc Blot loic.blot at unix-experience.fr writes: Strange but with nmap -sT -p port server -PN it works. Hello, it looks like there's some kind of incompatibility between PF and libdnet, which nmap uses (and maintains) for low level network operations. The -sT -Pn switches make nmap perform non-privileged operations only, and therefore mitigates the issue. Could you please report the problem to d...@nmap.org? You might want to test the latest (nmap 6.25) version first [1]. Regards [1] http://nmap.org/download -- Henri
Re: NMAP problem with PF
On Fri, Jan 04, 2013 at 12:09:10PM +0100, Lo?c Blot wrote: Hello, since OpenBSD 5.2 i have a problem with NMAP: Starting Nmap 6.01 ( http://nmap.org ) at 2013-01-04 11:47 CET route_dst_generic: Failed to obtain system routes: getsysroutes_dnet: sysroutes_dnet_find_interfaces() failed If i disable PF the problem isn't present. Do you have an idea ? Not really, but what were the exact nmap options used? What were your PF rules? Any other relevant info? running nmap -A pointed at a host in the local net here from a somewhat-past-5.2 snapshot produces normal scan output, fwiw. -- Peter N. M. Hansteen, member of the first RFC 1149 implementation team http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/ Remember to set the evil bit on all malicious network traffic delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
NMAP problem with PF
Hello, since OpenBSD 5.2 i have a problem with NMAP: Starting Nmap 6.01 ( http://nmap.org ) at 2013-01-04 11:47 CET route_dst_generic: Failed to obtain system routes: getsysroutes_dnet: sysroutes_dnet_find_interfaces() failed If i disable PF the problem isn't present. Do you have an idea ? Thanks. -- Best regards, Loïc BLOT, Engineering UNIX Systems, Security and Networks http://www.unix-experience.fr
Re: NMAP problem with PF
Hello, It's a simple nmap : Nmap -p 1688 a.b.c.d -PN Loic Blot Le 4 janv. 2013 à 12:14, Peter N. M. Hansteen pe...@bsdly.net a écrit : On Fri, Jan 04, 2013 at 12:09:10PM +0100, Lo?c Blot wrote: Hello, since OpenBSD 5.2 i have a problem with NMAP: Starting Nmap 6.01 ( http://nmap.org ) at 2013-01-04 11:47 CET route_dst_generic: Failed to obtain system routes: getsysroutes_dnet: sysroutes_dnet_find_interfaces() failed If i disable PF the problem isn't present. Do you have an idea ? Not really, but what were the exact nmap options used? What were your PF rules? Any other relevant info? running nmap -A pointed at a host in the local net here from a somewhat-past-5.2 snapshot produces normal scan output, fwiw. -- Peter N. M. Hansteen, member of the first RFC 1149 implementation team http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/ Remember to set the evil bit on all malicious network traffic delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
Re: NMAP problem with PF
Hmmm strange but with -- Best regards, Loïc BLOT, Engineering UNIX Systems, Security and Networks http://www.unix-experience.fr Le vendredi 04 janvier 2013 à 13:04 +0100, Loïc BLOT a écrit : Hello, It's a simple nmap : Nmap -p 1688 a.b.c.d -PN Loic Blot Le 4 janv. 2013 à 12:14, Peter N. M. Hansteen pe...@bsdly.net a écrit : On Fri, Jan 04, 2013 at 12:09:10PM +0100, Lo?c Blot wrote: Hello, since OpenBSD 5.2 i have a problem with NMAP: Starting Nmap 6.01 ( http://nmap.org ) at 2013-01-04 11:47 CET route_dst_generic: Failed to obtain system routes: getsysroutes_dnet: sysroutes_dnet_find_interfaces() failed If i disable PF the problem isn't present. Do you have an idea ? Not really, but what were the exact nmap options used? What were your PF rules? Any other relevant info? running nmap -A pointed at a host in the local net here from a somewhat-past-5.2 snapshot produces normal scan output, fwiw. -- Peter N. M. Hansteen, member of the first RFC 1149 implementation team http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/ Remember to set the evil bit on all malicious network traffic delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
Re: NMAP problem with PF
Strange but with nmap -sT -p port server -PN it works. -- Best regards, Loïc BLOT, Engineering UNIX Systems, Security and Networks http://www.unix-experience.fr Le vendredi 04 janvier 2013 à 13:04 +0100, Loïc BLOT a écrit : Hello, It's a simple nmap : Nmap -p 1688 a.b.c.d -PN Loic Blot Le 4 janv. 2013 à 12:14, Peter N. M. Hansteen pe...@bsdly.net a écrit : On Fri, Jan 04, 2013 at 12:09:10PM +0100, Lo?c Blot wrote: Hello, since OpenBSD 5.2 i have a problem with NMAP: Starting Nmap 6.01 ( http://nmap.org ) at 2013-01-04 11:47 CET route_dst_generic: Failed to obtain system routes: getsysroutes_dnet: sysroutes_dnet_find_interfaces() failed If i disable PF the problem isn't present. Do you have an idea ? Not really, but what were the exact nmap options used? What were your PF rules? Any other relevant info? running nmap -A pointed at a host in the local net here from a somewhat-past-5.2 snapshot produces normal scan output, fwiw. -- Peter N. M. Hansteen, member of the first RFC 1149 implementation team http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/ Remember to set the evil bit on all malicious network traffic delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
Redundant Firewall problem with pf/carp/pfsync/ipsec
I've currently been running a redundant firewall solution in our
Production environment using OpenBSD (version 4.5-stable) with CARP (4),
PF (4), PFsync (4) and SAsyncd (8) which syncs the pf rules and IPSEC
security associations via the cross-over cable method. We're also
running an IPSEC (4) tunnel between our production and internal networks
with a single OpenBSD machine (version 4.5-stable) running PF (4) on our
internal network.
In the following year since I've implemented this solution we've
experienced a problem in which our firewalls begin to act erratically
roughly every 4 months resulting in loss of SSH connectivity, SNMP
monitoring failure and the inability to run any command from the
console. Despite these problems, both production firewalls are still
pingable and continue to filter packets as they should.
+| Production Network |+
| |
bnx2| |bnx2
+-+ +-+
| fw1 |-bnx0--bnx0-| fw2 |
+-+ +-+
bnx1| |bnx1
| |
---+--- WAN/Internet---+---
|
{IPSEC tunnel}
|
+--+
| fw |
+--+
+| Internal Network |+
* *
These problems can simply be fixed be rebooting the master and then the
slave production firewalls; however this is obviously not a long term
solution to the problem at hand.
Since I'm not able to view or salvage any of the log files or even run a
top while this problem is occurring I've had a hard time troubleshooting
this issue. However the order of events leading up to the problem seems
to be:
1.) Our monitoring reports that the process load of one or both of the
firewalls can not longer be checked via SNMP
2.) Our IPSEC tunnel goes down
3.) SSH connectivity fails and console command line usage fails (I'm
still able to type a command but then I'm not able to ctrl-c back to the
command line)
Please let me know if you have an ideas why this issue might be
occurring. Thanks in advance.
Regards,
Jeff
Problem with PF - connection is first passed, but later blocked
Hi,
I try to reach a sftp server behind my openbsd 4.6 firewall. Sometimes
it works and sometimes not.
SRC IP: 10.100.106.58
DST IP: xxx.xxx.126.244
DST Port: 7400/tcp
If I try to connect pflog shows me:
Mar 16 20:36:39.570280 rule 201/(match) pass in on em0:
10.100.106.58.35286 xxx.xxx.126.244.7400: S 3090744159:3090744159(0)
win 5840 mss 1460,sackOK,timestamp 4134057806[|tcp] (DF)
Mar 16 20:36:39.570292 rule 201/(match) pass out on em4:
10.100.106.58.35286 xxx.xxx.126.244.7400: S 3090744159:3090744159(0)
win 5840 mss 1460,sackOK,timestamp 4134057806[|tcp] (DF)
Mar 16 20:37:14.677912 rule 244/(match) block in on em0:
10.100.106.58.35286 xxx.xxx.126.244.7400: P 3090746972:3090747004(32)
ack 3225125621 win 224 nop,nop,timestamp 4134092914 1713395 (DF) [tos
0x8]
Mar 16 20:37:14.916504 rule 244/(match) block in on em0:
10.100.106.58.35286 xxx.xxx.126.244.7400: P 0:32(32) ack 1 win 224
nop,nop,timestamp 4134093152 1713395 (DF) [tos 0x8]
Mar 16 20:37:15.392461 rule 244/(match) block in on em0:
10.100.106.58.35286 xxx.xxx.126.244.7400: P 0:32(32) ack 1 win 224
nop,nop,timestamp 4134093628 1713395 (DF) [tos 0x8]
the first connection passed, but them it is blocked - why?
affected rules:
@201 pass log quick inet proto tcp from tbl.r76.s:3 to tbl.r41.d:4
port = 7400 flags S/SA keep state label RULE 76 -- ACCEPT
[ Evaluations: 13948 Packets: 390 Bytes: 300416 States:
0 ]
[ Inserted: uid 0 pid 4296 State Creations: 12]
@244 block return-icmp(port-unr) log quick inet all label deny_rest
[ Evaluations: 28108654 Packets: 28108654 Bytes: 15763202122
States: 0 ]
[ Inserted: uid 0 pid 4296 State Creations: 0 ]
table tbl.r41.d { xxx.xxx.126.246 , xxx.xxx.126.244 , xxx.xxx.105.194
, xxx.xxx.126.245 }
table tbl.r76.s { 10.100.102.30 , 10.100.106.58 , 10.100.107.58 }
thx, for any hint.
regards,
Thomas
Re: Problem with pf rules.
Well, My rules of rdr now work, but dont log on. Only the out of rdr port 8080. Any suggestion? Thanks, Bye. 2010/1/14 PsYkHe psyk...@gmail.com Damn man!!!.Holy crap.I really forgot this detail... Thanks Man. Regards. did you net.inet.ip.forwarding=1 in sysctl? regards karl-heinz On 14.01.2010, at 16:10, PsYkHe wrote: I'm in troubles to put a router/firewall Openbsd 4.6 at vmware and at Slackware 13 to can talk throught of host-only. But the main problem now is the OpenBSD make a rdr to webserver Slackware. Well, I'll try descrive the situation: The OpenBSD 4.6 has two interfaces: One bridge One host-only with ip 192.168.38.130 At Slackware 13 has a interface: host-only with ip 192.168.38.128 That are my rules of pf: if_net=vic0 if_ws=vic1 ip_ws=192.168.138.128 #black log all pass log all rdr pass log on $if_net proto tcp to port 6060 - $ip_ws port 80 rdr pass log on $if_net proto tcp to port - 127.0.0.1 port 22 nat log on $if_net from !($if_net) - ($if_net:0) PS: Which if_net is the interface of the bridge and if_wa is the host-only. The OpenBSD can ping the internal ip of host-only of Slackware 192.168.138.128 and also when I sent a telnet to him in port 80 and it answer perfectly. Therefore when it comes outside of the internet, a telnet to OpenBSD in port it come in the ssh of OpenBSD but It cant log on. To port 6060 didn't show up the log and it cant do a rdr or it didn't work. I've thought the communication Slackware, the listen port 80 that was tcp6, maybe would be ipv6 only, but I did insert tcp to ipv4 and the rdr also didn't work. I'm using the command: tcpdump -n -e -ttt -i pflog0 To verify these logs by interface pflog0 I'm needing a light, suggestion or something like that..Can you tell me something guys? Any information or anything else you can ask me that Ill send. Thanks a lot. See ya.
Problem with pf rules.
I'm in troubles to put a router/firewall Openbsd 4.6 at vmware and at Slackware 13 to can talk throught of host-only. But the main problem now is the OpenBSD make a rdr to webserver Slackware. Well, I'll try descrive the situation: The OpenBSD 4.6 has two interfaces: One bridge One host-only with ip 192.168.38.130 At Slackware 13 has a interface: host-only with ip 192.168.38.128 That are my rules of pf: if_net=vic0 if_ws=vic1 ip_ws=192.168.138.128 #black log all pass log all rdr pass log on $if_net proto tcp to port 6060 - $ip_ws port 80 rdr pass log on $if_net proto tcp to port - 127.0.0.1 port 22 nat log on $if_net from !($if_net) - ($if_net:0) PS: Which if_net is the interface of the bridge and if_wa is the host-only. The OpenBSD can ping the internal ip of host-only of Slackware 192.168.138.128 and also when I sent a telnet to him in port 80 and it answer perfectly. Therefore when it comes outside of the internet, a telnet to OpenBSD in port it come in the ssh of OpenBSD but It cant log on. To port 6060 didn't show up the log and it cant do a rdr or it didn't work. I've thought the communication Slackware, the listen port 80 that was tcp6, maybe would be ipv6 only, but I did insert tcp to ipv4 and the rdr also didn't work. I'm using the command: tcpdump -n -e -ttt -i pflog0 To verify these logs by interface pflog0 I'm needing a light, suggestion or something like that..Can you tell me something guys? Any information or anything else you can ask me that Ill send. Thanks a lot. See ya.
Re: Problem with pf rules.
did you net.inet.ip.forwarding=1 in sysctl? regards karl-heinz On 14.01.2010, at 16:10, PsYkHe wrote: I'm in troubles to put a router/firewall Openbsd 4.6 at vmware and at Slackware 13 to can talk throught of host-only. But the main problem now is the OpenBSD make a rdr to webserver Slackware. Well, I'll try descrive the situation: The OpenBSD 4.6 has two interfaces: One bridge One host-only with ip 192.168.38.130 At Slackware 13 has a interface: host-only with ip 192.168.38.128 That are my rules of pf: if_net=vic0 if_ws=vic1 ip_ws=192.168.138.128 #black log all pass log all rdr pass log on $if_net proto tcp to port 6060 - $ip_ws port 80 rdr pass log on $if_net proto tcp to port - 127.0.0.1 port 22 nat log on $if_net from !($if_net) - ($if_net:0) PS: Which if_net is the interface of the bridge and if_wa is the host-only. The OpenBSD can ping the internal ip of host-only of Slackware 192.168.138.128 and also when I sent a telnet to him in port 80 and it answer perfectly. Therefore when it comes outside of the internet, a telnet to OpenBSD in port it come in the ssh of OpenBSD but It cant log on. To port 6060 didn't show up the log and it cant do a rdr or it didn't work. I've thought the communication Slackware, the listen port 80 that was tcp6, maybe would be ipv6 only, but I did insert tcp to ipv4 and the rdr also didn't work. I'm using the command: tcpdump -n -e -ttt -i pflog0 To verify these logs by interface pflog0 I'm needing a light, suggestion or something like that..Can you tell me something guys? Any information or anything else you can ask me that Ill send. Thanks a lot. See ya.
Re: Problem with pf rules.
Damn man!!!.Holy crap.I really forgot this detail... Thanks Man. Regards. did you net.inet.ip.forwarding=1 in sysctl? regards karl-heinz On 14.01.2010, at 16:10, PsYkHe wrote: I'm in troubles to put a router/firewall Openbsd 4.6 at vmware and at Slackware 13 to can talk throught of host-only. But the main problem now is the OpenBSD make a rdr to webserver Slackware. Well, I'll try descrive the situation: The OpenBSD 4.6 has two interfaces: One bridge One host-only with ip 192.168.38.130 At Slackware 13 has a interface: host-only with ip 192.168.38.128 That are my rules of pf: if_net=vic0 if_ws=vic1 ip_ws=192.168.138.128 #black log all pass log all rdr pass log on $if_net proto tcp to port 6060 - $ip_ws port 80 rdr pass log on $if_net proto tcp to port - 127.0.0.1 port 22 nat log on $if_net from !($if_net) - ($if_net:0) PS: Which if_net is the interface of the bridge and if_wa is the host-only. The OpenBSD can ping the internal ip of host-only of Slackware 192.168.138.128 and also when I sent a telnet to him in port 80 and it answer perfectly. Therefore when it comes outside of the internet, a telnet to OpenBSD in port it come in the ssh of OpenBSD but It cant log on. To port 6060 didn't show up the log and it cant do a rdr or it didn't work. I've thought the communication Slackware, the listen port 80 that was tcp6, maybe would be ipv6 only, but I did insert tcp to ipv4 and the rdr also didn't work. I'm using the command: tcpdump -n -e -ttt -i pflog0 To verify these logs by interface pflog0 I'm needing a light, suggestion or something like that..Can you tell me something guys? Any information or anything else you can ask me that Ill send. Thanks a lot. See ya.
Re: Problem with pf/nat (bug?) and aliases in internal interface
As a test, can you try it without using the 192.168.20.1-192.168.20.10
address range format, and see if that behaves any better? You can use
this instead: {192.168.20.0/29 192.168.20.8/31 192.168.20.10}
In gmane.os.openbsd.misc, you wrote:
Scenario:
int_if with two ip addresses in two differents lans (192.168.20.254,
192.168.21.254).
more aliases in the external interfaces
nat rules: every 10 internals ip use an external address for the nat.
everything works fine, except for the second internal ip address. ip
from 192.168.21.0/24 are natted with rules of net 192.168.20.0/24
machines from internal lan use .20.254 or .21.254 as a gateway.
p.s.
both of them works, but second ones use wrong nat.
# uname -mprs
OpenBSD 4.4 amd64 Intel(R) Xeon(R) CPU 5110 @ 1.60GHz
# pfctl -vsr
pass in log quick on bnx1 inet from 192.168.20.0/24 to any flags S/SA keep
state
[ Evaluations: 61921 Packets: 370618Bytes: 216808002 States: 4230
]
[ Inserted: uid 0 pid 12418 State Creations: 23774 ]
pass in log quick on bnx1 inet from 192.168.21.0/24 to any flags S/SA keep
state
[ Evaluations: 628 Packets: 13136 Bytes: 10432453States: 117
]
[ Inserted: uid 0 pid 12418 State Creations: 202 ]
# pfctl -vvsn | grep -A2 -e '@0' -e '@24' -e '@25'
@0 nat on bnx0 inet from 192.168.20.1 - 192.168.20.10 to any - xxx.xxx.xxx.1
[ Evaluations: 34016 Packets: 57999 Bytes: 23576755States: 803
]
[ Inserted: uid 0 pid 12418 State Creations: 5402 ]
@24 nat on bnx0 inet from 192.168.20.241 - 192.168.20.254 to any -
xxx.xxx.xxx.25
[ Evaluations: 1079 Packets: 3353 Bytes: 1489982 States: 79
]
[ Inserted: uid 0 pid 12418 State Creations: 179 ]
@25 nat on bnx0 inet from 192.168.21.1 - 192.168.21.10 to any -
xxx.xxx.xxx.26
[ Evaluations: 793 Packets: 0 Bytes: 0 States: 0
]
[ Inserted: uid 0 pid 12418 State Creations: 0 ]
--
Cris, member of G.U.F.I
Italian FreeBSD User Group
http://www.gufi.org/
Re: Problem with pf/nat (bug?) and aliases in internal interface
On 5/18/09 9:46 AM, Stuart Henderson wrote:
As a test, can you try it without using the 192.168.20.1-192.168.20.10
address range format, and see if that behaves any better? You can use
this instead: {192.168.20.0/29 192.168.20.8/31 192.168.20.10}
I already tried with 192.168.21.1, 192.168.21.2 and with a table.
Nothing change in nat rules.
--
Cristiano Deana - FreeCRIS
Ho iniziato a usare FreeBSD perche' m$ usava me. ed e' spiacevole
Problem with pf/nat (bug?) and aliases in internal interface
Scenario: int_if with two ip addresses in two differents lans (192.168.20.254, 192.168.21.254). more aliases in the external interfaces nat rules: every 10 internals ip use an external address for the nat. everything works fine, except for the second internal ip address. ip from 192.168.21.0/24 are natted with rules of net 192.168.20.0/24 machines from internal lan use .20.254 or .21.254 as a gateway. p.s. both of them works, but second ones use wrong nat. # uname -mprs OpenBSD 4.4 amd64 Intel(R) Xeon(R) CPU 5110 @ 1.60GHz # pfctl -vsr pass in log quick on bnx1 inet from 192.168.20.0/24 to any flags S/SA keep state [ Evaluations: 61921 Packets: 370618Bytes: 216808002 States: 4230 ] [ Inserted: uid 0 pid 12418 State Creations: 23774 ] pass in log quick on bnx1 inet from 192.168.21.0/24 to any flags S/SA keep state [ Evaluations: 628 Packets: 13136 Bytes: 10432453States: 117 ] [ Inserted: uid 0 pid 12418 State Creations: 202 ] # pfctl -vvsn | grep -A2 -e '@0' -e '@24' -e '@25' @0 nat on bnx0 inet from 192.168.20.1 - 192.168.20.10 to any - xxx.xxx.xxx.1 [ Evaluations: 34016 Packets: 57999 Bytes: 23576755States: 803 ] [ Inserted: uid 0 pid 12418 State Creations: 5402 ] @24 nat on bnx0 inet from 192.168.20.241 - 192.168.20.254 to any - xxx.xxx.xxx.25 [ Evaluations: 1079 Packets: 3353 Bytes: 1489982 States: 79] [ Inserted: uid 0 pid 12418 State Creations: 179 ] @25 nat on bnx0 inet from 192.168.21.1 - 192.168.21.10 to any - xxx.xxx.xxx.26 [ Evaluations: 793 Packets: 0 Bytes: 0 States: 0 ] [ Inserted: uid 0 pid 12418 State Creations: 0 ] -- Cris, member of G.U.F.I Italian FreeBSD User Group http://www.gufi.org/
Problem with pf/nat (bug?) and aliases in internal interface
Scenario: int_if with two ip addresses in two differents lans (192.168.20.254, 192.168.21.254). more aliases in the external interfaces nat rules: every 10 internals ip use an external address for the nat. everything works fine, except for the second internal ip address. ip from 192.168.21.0/24 are natted with rules of net 192.168.20.0/24 machines from internal lan use .20.254 or .21.254 as a gateway. p.s. both of them works, but second ones use wrong nat. # uname -mprs OpenBSD 4.4 amd64 Intel(R) Xeon(R) CPU 5110 @ 1.60GHz # pfctl -vsr pass in log quick on bnx1 inet from 192.168.20.0/24 to any flags S/SA keep state [ Evaluations: 61921 Packets: 370618Bytes: 216808002 States: 4230 ] [ Inserted: uid 0 pid 12418 State Creations: 23774 ] pass in log quick on bnx1 inet from 192.168.21.0/24 to any flags S/SA keep state [ Evaluations: 628 Packets: 13136 Bytes: 10432453States: 117 ] [ Inserted: uid 0 pid 12418 State Creations: 202 ] # pfctl -vvsn | grep -A2 -e '@0' -e '@24' -e '@25' @0 nat on bnx0 inet from 192.168.20.1 - 192.168.20.10 to any - xxx.xxx.xxx.1 [ Evaluations: 34016 Packets: 57999 Bytes: 23576755States: 803 ] [ Inserted: uid 0 pid 12418 State Creations: 5402 ] @24 nat on bnx0 inet from 192.168.20.241 - 192.168.20.254 to any - xxx.xxx.xxx.25 [ Evaluations: 1079 Packets: 3353 Bytes: 1489982 States: 79] [ Inserted: uid 0 pid 12418 State Creations: 179 ] @25 nat on bnx0 inet from 192.168.21.1 - 192.168.21.10 to any - xxx.xxx.xxx.26 [ Evaluations: 793 Packets: 0 Bytes: 0 States: 0 ] [ Inserted: uid 0 pid 12418 State Creations: 0 ] -- Cris, member of G.U.F.I Italian FreeBSD User Group http://www.gufi.org/
Problem with Pf
Hi Guys, I hope I am posting on the right mailing list. I am sending you this email because I have been experiencing a lot of BAD State in pf recently. I don't know if this has been discussed previously. More and and more people are now using Oses that can adapt the TCP Windows Size. In pf, I could see that pf checks for the sequence number to make sure it is in the expected range. Therefore, pf will make the following check: Sequence number + tcpwindow size = Maximum expected sequence number. This check was fin when there were on on the fly tcp window change. Now, on very low latency network (few ms), we might experience a race condition where pf will not see the packet in the right order, therefore, pf will see packets coming in with a new tcp window size, but will not see the first modified packet on time. Therefore, it will produce a Bad State in the logs. To correct this, I had to remove in pf this check. From now on, I don't have any problem anymore. I think we should work to find a correct alternative solution for this. More and More oses adapt there Window size, startng with Windows Vista, Linux (from 2.6.18 I think), Mac OSX Leopard. I am also seeing a strange behavior while running backups. The backup will run for about a Gig, then I will have bad stated and the following error: Dec 5 08:34:24 pf01a-std /bsd: pf: BAD state: TCP 193.189.125.226:9103 193.189.125.226:9103 77.72.89.171:1900 [lo=1110166540 high=1110165037 win=65535 modulator=0] [lo=3660513330 high=3660578711 win=32767 modulator=0] 4:4 A seq=1110132270 (1110132270) ack=3660513330 len=1456 ackskew=0 pkts=127312:59301 dir=in,fwd Dec 5 08:34:24 pf01a-std /bsd: pf: State failure on: 2 | You could notice that the lo=1110166540 is higher than high=1110165037 and of course the Sequence Number is outbound: seq=1110132270 Any idea what could cause such a mess ? I am using OpenBSD 4.1, custom built kernel just to comment on check in pf. Lio
Problem with pf: rdr and icmp
Hello, I think there is a bug in redirecting ICMP echo requests in the current pf. I have an OpenBSD firewall currently allowing pings. In that configuration, the firewall itself responds to the pings and everything works as expected. If I decide to redirect that ping via binat or rdr, the first ping will fail and subsequent pings will succeed. Amazingly, it doesn't matter where I redirect the ping to, it could be to the same public IP address the packet would have gone to anyway, it could be the inside interface of the firewall itself, or it could be a host within my LAN (osx and windows xp hosts). The other interesting thing is that tcpdump (both on OpenBSD and the receiving host) show the first ping packet, but simply refuse to reply to it. I can only assume that when I redirect to an IP address on the OpenBSD machine, the packet reaches the machine logically after firewalling also. Yet another interesting test, I tried pinging from a well known public cisco bgp route server. Apparently cisco increments what pf considers the port number of icmp connections with each ping it sends. The result of this was that none of the pings were replied to (yet again tcpdump shows they all made it to the host). I can only assume that the first ping packet (first packet defined as the packet that generates the state) isn't quite translated right by pf. I've simplified my rules to these for testing purposes... I do have multiple public addresses and have tested binat, which fails in the exact same way as doing it with rdr. Please let me know how I can assist more on this. a.a.a.a is my public IP, b.b.b.b is the outside server I'm testing from. # pfctl -sa TRANSLATION RULES: nat on sis0 inet from 192.168.2.0/24 to any - a.a.a.a rdr on sis0 inet proto icmp from any to a.a.a.a - 192.168.2.50 # the destination IP address in the above rdr rule can be modified to *anything* (that will respond). No matter what host I pick, the first ping fails and the rest succeed, including IP addresses of the OpenBSD machine itself. FILTER RULES: scrub on sis0 all fragment reassemble block return on sis0 all pass out on sis0 inet all flags S/SA keep state queue(q_nontcp, q_nontcp_pri) pass out on sis0 inet proto tcp all flags S/SA keep state queue(q_def, q_pri) pass in on sis0 inet proto icmp all icmp-type echoreq keep state queue(q_bulk, q_bulk_pri) block drop in on ! sis1 inet from 192.168.2.0/24 to any block drop in inet from 192.168.2.254 to any ALTQ: queue q_bulk on sis0 priority 2 queue q_bulk_pri on sis0 priority 3 queue q_def on sis0 priority 4 priq( default ) queue q_pri on sis0 priority 5 queue q_nontcp on sis0 priority 6 queue q_nontcp_pri on sis0 priority 7 (mac prompt)$ sudo tcpdump -pnlvv icmp Password: tcpdump: listening on en0, link-type EN10MB (Ethernet), capture size 96 bytes 22:56:51.920768 IP (tos 0x20, ttl 49, id 37291, offset 0, flags [none], proto: ICMP (1), length: 84) b.b.b.b 192.168.2.50: ICMP echo request, id 52631, seq 0, length 64 22:56:52.927812 IP (tos 0x20, ttl 49, id 37313, offset 0, flags [none], proto: ICMP (1), length: 84) b.b.b.b 192.168.2.50: ICMP echo request, id 52631, seq 1, length 64 22:56:52.927839 IP (tos 0x20, ttl 64, id 10009, offset 0, flags [none], proto: ICMP (1), length: 84, bad cksum 0 (-c802)!) 192.168.2.50 b.b.b.b: ICMP echo reply, id 52631, seq 1, length 64 22:56:53.937344 IP (tos 0x20, ttl 49, id 37329, offset 0, flags [none], proto: ICMP (1), length: 84) b.b.b.b 192.168.2.50: ICMP echo request, id 52631, seq 2, length 64 22:56:53.937367 IP (tos 0x20, ttl 64, id 10011, offset 0, flags [none], proto: ICMP (1), length: 84, bad cksum 0 (-c800)!) 192.168.2.50 b.b.b.b: ICMP echo reply, id 52631, seq 2, length 64 OpenBSD 4.1-current (GENERIC) #315: Mon Jul 2 13:24:20 MDT 2007 [EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC cpu0: Geode(TM) Integrated Processor by National Semi (Geode by NSC 586-class) 267 MHz cpu0: FPU,TSC,MSR,CX8,CMOV,MMX cpu0: TSC disabled real mem = 133787648 (127MB) avail mem = 121815040 (116MB) mainbus0 at root bios0 at mainbus0: AT/286+ BIOS, date 20/50/29, BIOS32 rev. 0 @ 0xf7840 pcibios0 at bios0: rev 2.0 @ 0xf/0x1 pcibios0: pcibios_get_intr_routing - function not supported pcibios0: PCI IRQ Routing information unavailable. pcibios0: PCI bus #0 is the last bus bios0: ROM list: 0xc8000/0x9000 cpu0 at mainbus0 pci0 at mainbus0 bus 0: configuration mode 1 (bios) pchb0 at pci0 dev 0 function 0 Cyrix GXm PCI rev 0x00 sis0 at pci0 dev 6 function 0 NS DP83815 10/100 rev 0x00, DP83816A: irq 10, address 00:00:24:c2:c0:04 nsphyter0 at sis0 phy 0: DP83815 10/100 PHY, rev. 1 sis1 at pci0 dev 7 function 0 NS DP83815 10/100 rev 0x00, DP83816A: irq 10, address 00:00:24:c2:c0:05 nsphyter1 at sis1 phy 0: DP83815 10/100 PHY, rev. 1 sis2 at pci0 dev 8 function 0 NS DP83815 10/100 rev 0x00, DP83816A: irq 10, address 00:00:24:c2:c0:06 nsphyter2 at sis2 phy 0: DP83815 10/100 PHY, rev. 1 gscpcib0 at pci0 dev 18 function 0 NS
Re: Problem with pf: rdr and icmp
* Jason Eggleston [EMAIL PROTECTED] [2007-07-09 08:29]: I think there is a bug in redirecting ICMP echo requests in the current pf. that sounds like what mpf fixed a few days ago... pls update and retry -- Henning Brauer, [EMAIL PROTECTED], [EMAIL PROTECTED] BS Web Services, http://bsws.de Full-Service ISP - Secure Hosting, Mail and DNS Services Dedicated Servers, Rootservers, Application Hosting - Hamburg Amsterdam
Re: Problem with pf: rdr and icmp
Henning wrote: that sounds like what mpf fixed a few days ago... pls update and retry Thanks, I updated to the July 5th snapshot and it works now. Did some more research with wireshark, it was the ICMP checksum that was incorrect originally, but only on the first packet of the session. tcpdump didn't notice the incorrect checksum, at least not with the options I supplied. Thanks, -Jason On 7/8/07, Jason Eggleston [EMAIL PROTECTED] wrote: Hello, I think there is a bug in redirecting ICMP echo requests in the current pf. I have an OpenBSD firewall currently allowing pings. In that configuration, the firewall itself responds to the pings and everything works as expected. If I decide to redirect that ping via binat or rdr, the first ping will fail and subsequent pings will succeed. Amazingly, it doesn't matter where I redirect the ping to, it could be to the same public IP address the packet would have gone to anyway, it could be the inside interface of the firewall itself, or it could be a host within my LAN (osx and windows xp hosts). The other interesting thing is that tcpdump (both on OpenBSD and the receiving host) show the first ping packet, but simply refuse to reply to it. I can only assume that when I redirect to an IP address on the OpenBSD machine, the packet reaches the machine logically after firewalling also. Yet another interesting test, I tried pinging from a well known public cisco bgp route server. Apparently cisco increments what pf considers the port number of icmp connections with each ping it sends. The result of this was that none of the pings were replied to (yet again tcpdump shows they all made it to the host). I can only assume that the first ping packet (first packet defined as the packet that generates the state) isn't quite translated right by pf. I've simplified my rules to these for testing purposes... I do have multiple public addresses and have tested binat, which fails in the exact same way as doing it with rdr. Please let me know how I can assist more on this. a.a.a.a is my public IP, b.b.b.b is the outside server I'm testing from. # pfctl -sa TRANSLATION RULES: nat on sis0 inet from 192.168.2.0/24 to any - a.a.a.a rdr on sis0 inet proto icmp from any to a.a.a.a - 192.168.2.50 # the destination IP address in the above rdr rule can be modified to *anything* (that will respond). No matter what host I pick, the first ping fails and the rest succeed, including IP addresses of the OpenBSD machine itself. FILTER RULES: scrub on sis0 all fragment reassemble block return on sis0 all pass out on sis0 inet all flags S/SA keep state queue(q_nontcp, q_nontcp_pri) pass out on sis0 inet proto tcp all flags S/SA keep state queue(q_def, q_pri) pass in on sis0 inet proto icmp all icmp-type echoreq keep state queue(q_bulk, q_bulk_pri) block drop in on ! sis1 inet from 192.168.2.0/24 to any block drop in inet from 192.168.2.254 to any ALTQ: queue q_bulk on sis0 priority 2 queue q_bulk_pri on sis0 priority 3 queue q_def on sis0 priority 4 priq( default ) queue q_pri on sis0 priority 5 queue q_nontcp on sis0 priority 6 queue q_nontcp_pri on sis0 priority 7 (mac prompt)$ sudo tcpdump -pnlvv icmp Password: tcpdump: listening on en0, link-type EN10MB (Ethernet), capture size 96 bytes 22:56:51.920768 IP (tos 0x20, ttl 49, id 37291, offset 0, flags [none], proto: ICMP (1), length: 84) b.b.b.b 192.168.2.50: ICMP echo request, id 52631, seq 0, length 64 22:56:52.927812 IP (tos 0x20, ttl 49, id 37313, offset 0, flags [none], proto: ICMP (1), length: 84) b.b.b.b 192.168.2.50: ICMP echo request, id 52631, seq 1, length 64 22:56:52.927839 IP (tos 0x20, ttl 64, id 10009, offset 0, flags [none], proto: ICMP (1), length: 84, bad cksum 0 (-c802)!) 192.168.2.50 b.b.b.b: ICMP echo reply, id 52631, seq 1, length 64 22:56:53.937344 IP (tos 0x20, ttl 49, id 37329, offset 0, flags [none], proto: ICMP (1), length: 84) b.b.b.b 192.168.2.50: ICMP echo request, id 52631, seq 2, length 64 22:56:53.937367 IP (tos 0x20, ttl 64, id 10011, offset 0, flags [none], proto: ICMP (1), length: 84, bad cksum 0 (-c800)!) 192.168.2.50 b.b.b.b: ICMP echo reply, id 52631, seq 2, length 64 OpenBSD 4.1-current (GENERIC) #315: Mon Jul 2 13:24:20 MDT 2007 [EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC cpu0: Geode(TM) Integrated Processor by National Semi (Geode by NSC 586-class) 267 MHz cpu0: FPU,TSC,MSR,CX8,CMOV,MMX cpu0: TSC disabled real mem = 133787648 (127MB) avail mem = 121815040 (116MB) mainbus0 at root bios0 at mainbus0: AT/286+ BIOS, date 20/50/29, BIOS32 rev. 0 @ 0xf7840 pcibios0 at bios0: rev 2.0 @ 0xf/0x1 pcibios0: pcibios_get_intr_routing - function not supported pcibios0: PCI IRQ Routing information unavailable. pcibios0: PCI bus #0 is the last bus bios0: ROM list: 0xc8000/0x9000 cpu0 at mainbus0 pci0 at mainbus0 bus 0: configuration mode 1 (bios) pchb0 at pci0 dev 0 function 0 Cyrix GXm PCI rev
Re: Pinging redundant firewall problem (isakmpd+pf+pfsync+sasyncd+carp)
catalin visinescu [EMAIL PROTECTED] wrote: Hello, Intro: I am using isakmpd+sasyncd+carp+pf+pfsync to have a redundant firewall setup (OpenBSD 4.0). I have two firewall that carp-advertise at the same rate, and not preempt each other. This works fine. isakmpd is using x509 certificates to establish SAs. This is working fine. sasyncd is running on both and they share the SAs properly. pfsync has been configured and it is working well. I have the following setup (netmask is /24 everywhere): Redundant end FW1: Ext IP: 172.16.140.2 (static) Int IP: 172.16.36.2 (static) FW2: Ext IP: 172.16.140.3 (static) Int IP: 172.16.36.3 (static) FW1 and FW2 shared IP addresses (carp) Ext IP: 172.16.140.1 Int IP: 172.16.36.1 Non-redundant end: Ext IP: 172.16.142.1 (static) Int IP: 172.16.40.1 (static) Problem: Assume the gateway that has static IP 172.16.36.2 is the master. I ping from 172.16.40.1 to 172.16.36.1 (or 172.16.36.2) and the ping goes through. The moment I ping the backup (ping -c 1 -I 172.16.40.1 172.16.36.3) I get a reply, but I can no longer ping 172.16.36.2. Now I can only ping the second gateway (goes in through the master, goes out through the backup). Everything goes back to normal (I can ping 172.16.36.2) the moment a new quick mode is finished and new SAs are established. Question: Why is this happening? I would like to have remote access to the backup gateway, for instance for live status polling (that's why I have the static IP addresses), or sync NTP time on firewalls (time source over secure tunnel). I don't mind if when I ping 172.16.36.3 the packet goes in through the first gateway and goes out through the second (because the flows are already set). I just don't want to block the communication on messages to the backup gateway. Can anyone help with this issue? ./catalin Hello, I understand now why this happens... it is a problem with the packet filter not updating the sequence numbers correctly. When I ping the master firewall the sequence numbers used are the same for both directions (SPIs)... (100,100) let's say. When I ping the backup, the request goes through master and goes out through the backup with sequence numbers (101, and 16485). That is normal behaviour and is documented here http://members.iinet.net.au/~nathanael/OpenBSD/sasyncd.html (section 1.5) Let's say 172.16.36.2 is the master... From the non-redundant end: ping -c 1 172.16.40.1 172.16.36.2 OK seq:100 request, 100 reply (sniffing on pfsync0 of the master firewall shows an updated seq # being sent to the backup firewall for that SPI) ping -c 1 172.16.40.1 172.16.36.3 OK seq:101 request, 101+16384=16485 reply (sniffing on pfsync0 of the master firewall shows an update being sent to the backup firewall) (sniffing on pfsync0 of the backup firewall shows an update being sent to the master firewall) NOTE THAT THE MASTER USES THE UPDATE FROM BACKUP. ping -c 1 172.16.40.1 172.16.36.2 OK seq:102 request, 16485+16384= 32869 reply (sniffing on pfsync0 of the master firewall shows an update being sent to the backup firewall) ping -c 1 172.16.40.1 172.16.36.2 OK seq:103 request, 16485+16384= 32870 reply (sniffing on pfsync0 of the master firewall shows an update being sent to the backup firewall) This part is clear... whenever a firewall has something to send, it is adding 1 to the previous sequence # if it sent the last message and it adds 16384 if the sequence # it has was received using pfsync from the other firewall. That I see in if_pfsync.c However if I change the test just a little bit... ping -i .1 172.16.40.1 172.16.36.2 OK seq:100 request, 100 reply, and so on (sniffing on pfsync0 of the master firewall shows an update being sent to the backup firewall) and at some point: ping -c 1 172.16.40.1 172.16.36.3 OK seq:101 request, 101+16384=16485 reply (sniffing on pfsync0 of the backup firewall shows an update being sent to the master firewall) The communication to 172.16.36.2 stops as the master does not get the update of the seq # for that SPI. The update is sent though (sniffing pfsync). As soon as a new SA is established everything (obviously) goes back to normal. THE MASTER DOES NOT USE THE UPDATE FROM THE BACKUP. This is quite bizarre that sending this one packet stops the traffic to 172.16.36.2. I would expect some packets to be lost until the master receives the update from the backup though (up to a second). I will take a look at if_pfsync.c and check why this happens. Hope this helps. ./catalin - Be smarter than spam. See how smart SpamGuard is at giving junk email the boot with the All-new Yahoo! Mail
Pinging redundant firewall problem (isakmpd+pf+pfsync+sasyncd+carp)
Hello, Intro: I am using isakmpd+sasyncd+carp+pf+pfsync to have a redundant firewall setup (OpenBSD 4.0). I have two firewall that carp-advertise at the same rate, and not preempt each other. This works fine. isakmpd is using x509 certificates to establish SAs. This is working fine. sasyncd is running on both and they share the SAs properly. pfsync has been configured and it is working well. I have the following setup (netmask is /24 everywhere): Redundant end FW1: Ext IP: 172.16.140.2 (static) Int IP: 172.16.36.2 (static) FW2: Ext IP: 172.16.140.3 (static) Int IP: 172.16.36.3 (static) FW1 and FW2 shared IP addresses (carp) Ext IP: 172.16.140.1 Int IP: 172.16.36.1 Non-redundant end: Ext IP: 172.16.142.1 (static) Int IP: 172.16.40.1 (static) Problem: Assume the gateway that has static IP 172.16.36.2 is the master. I ping from 172.16.40.1 to 172.16.36.1 (or 172.16.36.2) and the ping goes through. The moment I ping the backup (ping -c 1 -I 172.16.40.1 172.16.36.3) I get a reply, but I can no longer ping 172.16.36.2. Now I can only ping the second gateway (goes in through the master, goes out through the backup). Everything goes back to normal (I can ping 172.16.36.2) the moment a new quick mode is finished and new SAs are established. Question: Why is this happening? I would like to have remote access to the backup gateway, for instance for live status polling (that's why I have the static IP addresses), or sync NTP time on firewalls (time source over secure tunnel). I don't mind if when I ping 172.16.36.3 the packet goes in through the first gateway and goes out through the second (because the flows are already set). I just don't want to block the communication on messages to the backup gateway. Can anyone help with this issue? ./catalin - Be smarter than spam. See how smart SpamGuard is at giving junk email the boot with the All-new Yahoo! Mail
Pinging redundant firewall problem (isakmpd+pf+pfsync+sasyncd+carp)
Hello, Intro: I am using isakmpd+sasyncd+carp+pf+pfsync to have a redundant firewall setup (OpenBSD 4.0). I have two firewall that carp-advertise at the same rate, and not preempt each other. Basically I don't care which firewall is master and which is backup. This works fine. isakmpd is using x509 certificates to establish SAs. This is working fine. sasyncd is running on both and they share the SAs properly. pfsync has been configured and it is working well. I have the following setup (netmask is /24 everywhere): Redundant end FW1: Ext IP: 172.16.140.2 (static) Int IP: 172.16.36.2 (static) FW2: Ext IP: 172.16.140.3 (static) Int IP: 172.16.36.3 (static) FW1 and FW2 shared IP addresses (carp) Ext IP: 172.16.140.1 Int IP: 172.16.36.1 Non-redundant end: Ext IP: 172.16.142.1 (static) Int IP: 172.16.40.1 (static) Problem: Assume the gateway that has static IP 172.16.36.2 is the master. I ping from 172.16.40.1 to 172.16.36.1 (or 172.16.36.2) and the ping goes through. The moment I ping the backup (ping -c 1 -I 172.16.40.1 172.16.36.3) I get a reply, but I can no longer ping 172.16.36.2. Now I can only ping the second gateway (goes in through the master, goes out through the backup). Everything goes back to normal (I can ping 172.16.36.2) the moment a new quick mode is finished and new SAs are established. Question: Why is this happening? I would like to have remote access to the backup gateway, for instance for live status polling (that's why I have the static IP addresses), or sync NTP time on firewalls (time source over secure tunnel). I don't mind if when I ping 172.16.36.3 the packet goes in through the first gateway and goes out through the second (because the flows are already set). I just don't want to block the communication on messages to the backup gateway. Additional info: 1. FYI... I wanted a faster switch over with time and I had to change carp a bit to allow polling rates of under a second. Also there was a bug where setting the advbase 0 and advskew 100 only set the proper value of advbase the second time ifconfig command is typed. The patches have been submitted to [EMAIL PROTECTED] Marco Pfatschbacher was nice and added the changes. The changes will be found in OpenBSD 4.2. With advbase 0 and advskew 25 the switchover is half a second to a second. 2. I have noted that when sasyncd is copying the SAs on the backup, it does not set the validity of the SAs to the remaining validity time of that SA (for instance when the backup is booting later). The validity time is set as if the SA has just been created. This way the backup will still have in its SADB Security Associations copied from the master that are expired and removed from the master. 3. Another problem (rebooting the master/backup in a given order) can get to pretty bizar situation where a redundant gateway has 4 unidirectional SAs, and it is using one SA from one the first main mode to send, and one SA from the latter main-mode to receive. A ping message does not go through, although both ends have the 4 SAs. This is a topic of its own, if you want to know more I can give you the detailed information how to reproduce it. Many thanks, Catalin - Be smarter than spam. See how smart SpamGuard is at giving junk email the boot with the All-new Yahoo! Mail
Weird problem with PF and Load Balancing
Hi all, I'm willing to implement altq on my firewall but, right know, there is a problem that i didn't saw a solution for. I do have 2 ADSL links, and I'm doing load balancing for outgoing connections, using the round-robin option, and I'm also using the reply-to option to route back the packets that come in my secondary link (the one that isn't the default gateway of my firewall). Know to the problem. To implement altq in places with only one link, i do the following: set the bandwidth of the interface to it's maximum, in this case, 100Mb. Then, i set up 2 queues. One for deal with the traffic to firewall itself. Things as dhcp queries and ssh. This queue is configured with the max bandwidth minus the ADSL downstream bandwidth. So, there is one queue with 99.5MB, and other with 0.5Mb, for example. All traffic from internal network to the firewall itself, is put in the larger queue, and the traffic going to the internet is divided into another sub queues, but the point is that traffic not to the firewall is directed to another queue. I already had success with this kind of setup, with one link. Know to my problem. My 2 ADSL had different downstream bandwidth. And, as i'm using round-robin, i don't know where the connection is going. I don't kndow how to implement altq in this especific situation. I was thinking in something like: one queue for normal traffic to the firewall itself, with 99.2Mb. And two other queues with 0.5Mb and 0.3Mb respectively. But i don't know if this work, because i can assign only 1 queue per rule. And, with round-robin, i don't know where the packet is going. Thanks in advance, -- Giancarlo Razzolini Linux User 172199 Moleque Sem Conteudo Numero #002 Slackware Current OpenBSD Stable Snike Tecnologia em Informatica 4386 2A6F FFD4 4D5F 5842 6EA0 7ABE BBAB 9C0E 6B85 [demime 1.01d removed an attachment of type application/pgp-signature which had a name of signature.asc]
Re: Transparent ISP proxy problem or PF problem
Hi again, Steve. With any potential MTU issue I always start with something like ping -vDs 1472 arenabg.com from various hosts and routers. As you vary the sizes you should receive either an echo-reply or a packet-too-big (confirm with a packet sniffer). If you don't receive any reply you might have found why and where PathMTU is broken. I tried the ping test. Here are some results --- pinging from the OpenBSD router --- $ ping -vDs 1472 arenabg.com PING arenabg.com (82.101.72.23): 1472 data bytes 1480 bytes from 82.101.72.23: icmp_seq=0 ttl=57 time=15.371 ms --- pinging from the OpenBSD router --- $ ping -vDs 1473 arenabg.com PING arenabg.com (82.101.72.24): 1473 data bytes ping: sendto: Message too long ping: wrote arenabg.com 1501 chars, ret=-1 --- pinging from a machine behind the router --- $ ping -vds 1472 arenabg.com PING arenabg.com (82.101.72.24) 1472(1500) bytes of data. 1480 bytes from pleasure-dome.arenabg.com (82.101.72.24): icmp_seq=1 ttl=56 time=28.1 ms --- pinging from a machine behind the router --- $ ping -vds 1473 arenabg.com PING arenabg.com (82.101.72.23) 1473(1501) bytes of data. # no reply is recieved - 100% packet loss --- pinging from a machine outside my network --- $ ping -vDs 1472 arenabg.com PING arenabg.com (82.101.72.24): 1472 data bytes 1480 bytes from 82.101.72.24: icmp_seq=0 ttl=61 time=6.288 ms --- pinging from a machine outside my network --- $ ping -vDs 1473 arenabg.com PING arenabg.com (82.101.72.23): 1473 data bytes ping: sendto: Message too long The last results are from a machine that is not in my provider's network either. I'd be happy if you could post some comment on this. Does this mean that there is a PMTU problem with my OBSD router? Thanks, Alexander
Transparent ISP proxy problem or PF problem
Hi there.
First I want to state that I don't claim the problem I'm describing
below to be OpenBSD problem. It looks to me like a problem in the
particular set of setups between me, my ISP and the problem site.
Now, to the problem.
I'm using OpenBSD 3.8-release box as a router between a private network
(192.168.1.0/24) and the internet.
internet - (83.148.x.x) [OpenBSD] (192.168.1.1) - priv. lan
So far so good - the setup works very well with just one problem. My
ISP passes the traffic for two certain sites through a transparent
proxy. I reached to this conclusion due to the following:
---
$ telnet arenabg.com 80
Trying 82.101.72.23...
Connected to arenabg.com.
Escape character is '^]'.
GET / HTTP/1.0
HTTP/1.0 503 Service Unavailable
Server: squid/2.5.STABLE5
Mime-Version: 1.0
Date: Thu, 01 Dec 2005 14:22:09 GMT
Content-Type: text/html
Content-Length: 1
Expires: Thu, 01 Dec 2005 14:22:09 GMT
X-Squid-Error: ERR_CONNECT_FAIL 111
X-Cache: MISS from url
Connection: close
---
For one of these two sites there is no problem - the traffic passes
through my ISP's transparent proxy and the site works perfectly. The
problem is with the other site. When one tries to open that other site
(arenabg.com), for example with Mozilla Firefox, the browser loads some
data (e.g. displays the title of the page) and continues loading
like forever. I don't this it's a browser problem, since the problem
exists on other browsers/versions too.
I tried to connect the cable for the internet directly to one of the
client machines behind the firewall (Debian GNU/Linux 3.1) and the site
loads perfectly, so I came to the conclusion that my PF rules are
blocking the packets. So, I left a minimal PF setup (pass all
keep state + NAT), but the problem remained.
After some research I found a common problem with very similar symptoms
called Path MTU Discovery problem - the packets sent from the server
are larger than the Path MTU of the route to the client and with DF flag
set, but the server (or some router) blocks the ICMP messages returned
to the server and does not get notified for the fact that it must
descrease the size of the packages.
Unfortunately it seems that this is not the case. Descreasing the MTU
on the router interface should have fixed the problem, but no luck.
Now, what I'm asking here is for some advice on how to proceed from
here in order to diagnose the problem.
Again, I don't claim this to be OpenBSD problem or bug, although I
tried to boot Freesco on the router machine and the problem site
(arenabg.com) did load.
Thanks in advance.
Best regards,
Alexander Iliev
configurations follow:
dmesg
OpenBSD 3.8 (GENERIC) #138: Sat Sep 10 15:41:37 MDT 2005
[EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: Intel 486DX (486-class)
real mem = 33140736 (32364K)
avail mem = 22257664 (21736K)
using 430 buffers containing 1761280 bytes (1720K) of memory
mainbus0 (root)
bios0 at mainbus0: AT/286+(c2) BIOS, date 07/20/94
pcibios at bios0 function 0x1a not configured
bios0: ROM list: 0xc/0x8000
cpu0 at mainbus0
isa0 at mainbus0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard
vga0 at isa0 port 0x3b0/48 iomem 0xa/131072
wsdisplay0 at vga0 mux 1: console (80x25, vt100 emulation), using wskbd0
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
wdc0 at isa0 port 0x1f0/8 irq 14
wd0 at wdc0 channel 0 drive 0: ST33232A
wd0: 16-sector PIO, LBA, 3077MB, 6303024 sectors
wd0(wdc0:0:0): using BIOS timings
ep0 at isa0 port 0x300/16 irq 10: address 00:20:af:07:f2:58, utp/aui\
(default utp)
pcppi0 at isa0 port 0x61
midi0 at pcppi0: PC speaker
spkr0 at pcppi0
sysbeep0 at pcppi0
npx0 at isa0 port 0xf0/16: using exception 16
pccom2 at isa0 port 0x3e8/8 irq 5: ns16450, no fifo
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
fd0 at fdc0 drive 0: 1.44MB 80 cyl, 2 head, 18 sec
isapnp0 at isa0 port 0x279: read port 0x203
ne3 at isapnp0 UMC PLUG PLAY Ethernet Chip , UMC9008, PNP80D6, \
port 0x200/32 irq 3
ne3: NE2000 Ethernet
ne3: address 40:01:00:00:30:fb
biomask fbd5 netmask ffdd ttymask ffdf
pctr: no performance counters in CPU
dkcsum: wd0 matches BIOS drive 0x80
root on wd0a
rootdev=0x0 rrootdev=0x300 rawdev=0x302
/dmesg
pf.conf
# external interface
ext_if = ne3
# internal interface
int_if = ep0
# Bianor firewall/gateway
work_fw = 212.95.x.x
webserver = 192.168.1.7
table allowed_icmp persist { $work_fw, $provider }
table allowed_ssh_to_ws persist { $work_fw }
# set logging on for ext_if
set loginterface $ext_if
scrub in
altq on $int_if cbq bandwidth 4Mb queue {std,ssh}
queue std bandwidth 70% cbq(default, borrow)
queue ssh bandwidth 30% priority 5 cbq(borrow)
# nat on local networks
nat pass on $ext_if from $int_if:network
to !$int_if:network - ($ext_if)
# redirect to internal apache and ssh
rdr pass on $ext_if
Re: Transparent ISP proxy problem or PF problem
--On 07 December 2005 12:33 +0200, Alexander Iliev wrote:
So far so good - the setup works very well with just one problem. My
ISP passes the traffic for two certain sites through a transparent
proxy. I reached to this conclusion due to the following:
It may be that the site is using Squid as an 'http-accelerator' to
relieve load on the webservers (client connects to accel., accel
connects to httpd and pulls the content, httpd connection then closed
and the content is fed to the client. resources on the real webservers
[e.g. RAM to run scripts in an interpreted language] are only used
while sending data to the accelerator, at lan-speed, rather than for
the duration of the client connection, possibly at modem speed).
I tried to connect the cable for the internet directly to one of the
client machines behind the firewall (Debian GNU/Linux 3.1) and the
site loads perfectly, so I came to the conclusion that my PF rules are
blocking the packets. So, I left a minimal PF setup (pass all keep
state + NAT), but the problem remained.
After some research I found a common problem with very similar
symptoms called Path MTU Discovery problem - the packets sent from
Your test with 'telnet' gives small enough packets that it probably
won't be affected by PMTU problems.
Again, I don't claim this to be OpenBSD problem or bug, although I
tried to boot Freesco on the router machine and the problem site
(arenabg.com) did load.
Could this have just been by chance? There are two IP addresses for
arenabg.com, perhaps one was working and one was failing..
Let's look at this error message:
$ telnet arenabg.com 80
Trying 82.101.72.23...
Connected to arenabg.com.
Escape character is '^]'.
GET / HTTP/1.0
HTTP/1.0 503 Service Unavailable
Server: squid/2.5.STABLE5
Mime-Version: 1.0
Date: Thu, 01 Dec 2005 14:22:09 GMT
Content-Type: text/html
Content-Length: 1
Expires: Thu, 01 Dec 2005 14:22:09 GMT
X-Squid-Error: ERR_CONNECT_FAIL 111
X-Cache: MISS from url
Connection: close
This says that the proxy cannot connect to the backend server. Are you
sure the website functions correctly? It doesn't work (no answer on
port 80) from the 3 ISPs I just tried it from here.
If you're sure the site is ok, try logging the blocked packets ('block
log all') and examine pflog (tcpdump -netttipflog0, as mentioned in the
man page). If that doesn't help, try setting pfctl -xmisc and look at
syslog.
Re: Transparent ISP proxy problem or PF problem
2005/12/7, Stuart Henderson [EMAIL PROTECTED]:
--On 07 December 2005 12:33 +0200, Alexander Iliev wrote:
So far so good - the setup works very well with just one problem. My
ISP passes the traffic for two certain sites through a transparent
proxy. I reached to this conclusion due to the following:
It may be that the site is using Squid as an 'http-accelerator' to
relieve load on the webservers (client connects to accel., accel
connects to httpd and pulls the content, httpd connection then closed
and the content is fed to the client. resources on the real webservers
[e.g. RAM to run scripts in an interpreted language] are only used
while sending data to the accelerator, at lan-speed, rather than for
the duration of the client connection, possibly at modem speed).
The Squid is at my ISP. I can tell that, 'cause I spoke to them
and they explained that their primary internet provider does not
have connectivity to these sites and they redirect the traffic for
just these two through another ISP. The internet bussiness in
Bulgaria is pretty messy, you know.
I tried to connect the cable for the internet directly to one of the
client machines behind the firewall (Debian GNU/Linux 3.1) and the
site loads perfectly, so I came to the conclusion that my PF rules are
blocking the packets. So, I left a minimal PF setup (pass all keep
state + NAT), but the problem remained.
After some research I found a common problem with very similar
symptoms called Path MTU Discovery problem - the packets sent from
Your test with 'telnet' gives small enough packets that it probably
won't be affected by PMTU problems.
The conclusion that my problem is not PMTU related did not come
from the telnet test. From what I've read on this, I think that
descreasing the mtu on my side enough should remove the problem
due to smaller MSS value sent to the server. Also I ran a test
with mtufinder
(here: http://users.tpg.com.au/adsln4yb/Perl/mtufinder) and it
passed without problems.
Again, I don't claim this to be OpenBSD problem or bug, although I
tried to boot Freesco on the router machine and the problem site
(arenabg.com) did load.
Could this have just been by chance? There are two IP addresses for
arenabg.com, perhaps one was working and one was failing..
Yes, it could be by chance, I'll have to test this one again to
make shure that under Freesco it really works.
As for the IP addresses for arenabg.com - yes, there are two such
addresses and they both work, but not outside BG, so any attempt
to connect to these server from outside will fail. Sorry, I should
have mentioned that before.
Let's look at this error message:
$ telnet arenabg.com 80
Trying 82.101.72.23...
Connected to arenabg.com.
Escape character is '^]'.
GET / HTTP/1.0
HTTP/1.0 503 Service Unavailable
Server: squid/2.5.STABLE5
Mime-Version: 1.0
Date: Thu, 01 Dec 2005 14:22:09 GMT
Content-Type: text/html
Content-Length: 1
Expires: Thu, 01 Dec 2005 14:22:09 GMT
X-Squid-Error: ERR_CONNECT_FAIL 111
X-Cache: MISS from url
Connection: close
This says that the proxy cannot connect to the backend server. Are you
sure the website functions correctly? It doesn't work (no answer on
port 80) from the 3 ISPs I just tried it from here.
See above. :)
If you're sure the site is ok, try logging the blocked packets ('block
log all') and examine pflog (tcpdump -netttipflog0, as mentioned in the
man page). If that doesn't help, try setting pfctl -xmisc and look at
syslog.
Yep, I tried the first one, but no blocked packets from this
site appeared. I'll try the pfctl -xmisc thing and see what will
come out.
Thanks and best regards.
Alexander
Re: Transparent ISP proxy problem or PF problem
I tried to connect the cable for the internet directly to one of the client machines behind the firewall (Debian GNU/Linux 3.1) and the site loads perfectly, so I came to the conclusion that my PF rules are blocking the packets. So, I left a minimal PF setup (pass all keep state + NAT), but the problem remained. When you tried a basic pass all + nat ruleset, did you leave in the scrub rule? It's needed to allow NAT on IP fragments. After some research I found a common problem with very similar symptoms called Path MTU Discovery problem - the packets sent from the server are larger than the Path MTU of the route to the client and with DF flag set, but the server (or some router) blocks the ICMP messages returned to the server and does not get notified for the fact that it must descrease the size of the packages. It does sound like a classic PathMTU problem... but something else might be up because although I can ping the site - with a full 1500B MTU - I can't access port 80. I googled and found something about this site being blocked outside Bulgaria. Also confusing is the fact you can access the server when using a directly connected host (or Freesco). And the proxy complicates things. Unfortunately it seems that this is not the case. Descreasing the MTU on the router interface should have fixed the problem, but no luck. Not always - because the remote server might not listen to your ICMP packet-too-big messages. Often you can get around this by rigging the MSS (MTU-40) in the TCP handshake using the max-mss option to the scrub rule. You might try this and find things start working even though you don't know where the problem is. Now, what I'm asking here is for some advice on how to proceed from here in order to diagnose the problem. With any potential MTU issue I always start with something like ping -vDs 1472 arenabg.com from various hosts and routers. As you vary the sizes you should receive either an echo-reply or a packet-too-big (confirm with a packet sniffer). If you don't receive any reply you might have found why and where PathMTU is broken. Steve
Re: Transparent ISP proxy problem or PF problem
Your test with 'telnet' gives small enough packets that it probably won't be affected by PMTU problems. The conclusion that my problem is not PMTU related did not come from the telnet test. From what I've read on this, I think that descreasing the mtu on my side enough should remove the problem due to smaller MSS value sent to the server. You'll either need to reduce MTU on the end-host (not on the box doing NAT), or (easier) use the max-mss option in pf.conf. The MTU value on the box doing NAT won't change the MSS on the NATted packets, it only affects packets coming from the box itself. See above. :) Ah, I see now (:
Re: Transparent ISP proxy problem or PF problem
2005/12/7, Stuart Henderson [EMAIL PROTECTED]: Your test with 'telnet' gives small enough packets that it probably won't be affected by PMTU problems. The conclusion that my problem is not PMTU related did not come from the telnet test. From what I've read on this, I think that descreasing the mtu on my side enough should remove the problem due to smaller MSS value sent to the server. You'll either need to reduce MTU on the end-host (not on the box doing NAT), or (easier) use the max-mss option in pf.conf. Yes, I did that too. In fact, not being sure on which interface should the MTU be altered, I tried to set the MTU on both interfaces of the NAT machine and also on the interface of the client machine to 576. Unfortunately, it did not help, thus my conclusion that the problem is not PMTU related, at least not at my side. If I make a wrong conclusion somewhere, please correct me - I am no network guru, and even if I were I could be wrong still. :) I thought the PMTU problem could be somewhere between me and arenabg.com, at my ISP for example. But if the problem is there no single client of that provider would not have access to this site and this is not true. Am I right on this? I thought some scrub configuration in pf.conf might be causing the problem and tried few scrub settings (including scrubbing disabled), but it did not help either. The MTU value on the box doing NAT won't change the MSS on the NATted packets, it only affects packets coming from the box itself. See above. :) Ah, I see now (:
Re: Transparent ISP proxy problem or PF problem
2005/12/7, Steve Welham [EMAIL PROTECTED]: I tried to connect the cable for the internet directly to one of the client machines behind the firewall (Debian GNU/Linux 3.1) and the site loads perfectly, so I came to the conclusion that my PF rules are blocking the packets. So, I left a minimal PF setup (pass all keep state + NAT), but the problem remained. When you tried a basic pass all + nat ruleset, did you leave in the scrub rule? It's needed to allow NAT on IP fragments. Not sure on this. I think the scrubbing was enabled, but I'll have to try it again to make sure. After some research I found a common problem with very similar symptoms called Path MTU Discovery problem - the packets sent from the server are larger than the Path MTU of the route to the client and with DF flag set, but the server (or some router) blocks the ICMP messages returned to the server and does not get notified for the fact that it must descrease the size of the packages. It does sound like a classic PathMTU problem... but something else might be up because although I can ping the site - with a full 1500B MTU - I can't access port 80. I googled and found something about this site being blocked outside Bulgaria. Also confusing is the fact you can access the server when using a directly connected host (or Freesco). And the proxy complicates things. Yes, sorry I didn't mention that - the site is not accesible from anywhere except Bulgaria. Unfortunately it seems that this is not the case. Descreasing the MTU on the router interface should have fixed the problem, but no luck. Not always - because the remote server might not listen to your ICMP packet-too-big messages. Often you can get around this by rigging the MSS (MTU-40) in the TCP handshake using the max-mss option to the scrub rule. You might try this and find things start working even though you don't know where the problem is. I think I tried that too, but again - I'll have to make sure. I tried so many things until now, that I can't remember anymore what I did and what I did not try. :) Now, what I'm asking here is for some advice on how to proceed from here in order to diagnose the problem. With any potential MTU issue I always start with something like ping -vDs 1472 arenabg.com from various hosts and routers. As you vary the sizes you should receive either an echo-reply or a packet-too-big (confirm with a packet sniffer). If you don't receive any reply you might have found why and where PathMTU is broken. Sorry, it seems I don't get exactly what should happen. :) I run 'ping -vDs 1472 arenabg.com' from where? From the obsd router, from the end-host, from somewhere else? And which machine should send the echo-reply/packet-too-big message? The one that I run the ping from or arenabg.com or some host inbetween (is this word spelled correctly?)? Again sorry for the dumb questions. Steve Thanks and regards, Alexander
