Re: Problem with ipfw nat and packet to local services
On Mon, 19 Jul 2010, Mamontov Roman wrote:
Hello, Ian.
UDP port 33564 on this box (xxx.xxx.xxx.xxx) is not redirected to any
other address:port, and you have specified deny_in (-deny_incoming in
natd-speak) so, well, you got what you asked for ..
See the description under -deny_incoming and the explanation of what
happens to incoming packets under -alias_address in natd(8) .. the nat
description in ipfw(8) is still a bit thin, so natd(8) is still useful.
Without deny_in, new inbound packets should be passed to the local
machine - so you will then need firewall rules to restrict which local
ports are to be accessible for connections from the outside.
cheers, Ian
I remove option deny_in from nat configuration. But inbound packets not
passed to the
local services.
#ipfw nat show config
ipfw nat 1 config ip xxx.xxx.xxx.xxx
#ipfw show
0003559 4703 nat 1 log ip from any to any via ext_if1
65000 51044734 allow ip from any to any
65535 58083 11212917 deny ip from any to any
Hi Mamontov,
What's the value of sysctl net.inet.ip.fw.one_pass ? It needs to be 0
so that packets will re-enter the firewall after NAT processing.
Otherwise, it might help to
a) run 'ipfw zero' before any tests .. I'm wondering about all those
packets hitting rule 65535; were they from before adding rule 65000?
b) add some count rules before and after nat, to show all packets
that may be eligible for NAT translation, maybe something like:
00020 count log ip from any to any in recv ${ext_if}
00022 count log ip from any to any out xmit ${ext_if}
00024 count log ip from any to any out recv ${int_if} xmit ${ext_if}
00035 nat ...
00040 count log ip from any to any in recv ${ext_if}
00042 count log ip from any to any out xmit ${ext_if}
00044 count log ip from any to any out recv ${int_if} xmit ${ext_if}
So you actually get to see the flow of packets before and after nat,
both to/from the local box and packets mapped to/rom inside addresses.
Again, an 'ipfw zero' before tests will make packet counts clearer.
Of course something like '# tcpdump -pn -i ext_if' will also show all
packets via ext_if in some detail. Be more specific if just looking for
some particular flows, like maybe appending 'udp port N' to that.
That is, try to follow packets you'd expect to be coming in for services
on the local box so if they are disappearing, you'll know where or why.
'netstat -finet -an' will show all those services that are listening.
If that doesn't help, we'll need more information.
cheers, Ian
___
freebsd-ipfw@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to freebsd-ipfw-unsubscr...@freebsd.org
Current problem reports assigned to freebsd-ipfw@FreeBSD.org
Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description o kern/148689 ipfw [ipfw] antispoof wrongly triggers on link local IPv6 a o kern/148430 ipfw [ipfw] IPFW schedule delete broken. o kern/148429 ipfw net.inet.ip.dummynet.io_fast broken or documentation i o kern/148157 ipfw [ipfw] IPFW in kernel nat BUG found in FreeBSD 8.1-PRE o conf/148144 ipfw [patch] add ipfw_nat support for rc.firewall simple ty o conf/148137 ipfw [ipfw] call order of natd and ipfw startup scripts o kern/148091 ipfw [ipfw] ipfw ipv6 handling broken. o kern/147720 ipfw [ipfw] ipfw dynamic rules and fwd o kern/145733 ipfw [ipfw] [patch] ipfw flaws with ipv6 fragments o kern/145305 ipfw [ipfw] ipfw problems, panics, data corruption, ipv6 so o kern/145167 ipfw [ipfw] ipfw nat does not follow its documentation o kern/144869 ipfw [ipfw] [panic] Instant kernel panic when adding NAT ru o kern/144269 ipfw [ipfw] problem with ipfw tables o kern/144187 ipfw [ipfw] deadlock using multiple ipfw nat and multiple l o kern/143973 ipfw [ipfw] [panic] ipfw forward option causes kernel reboo o kern/143653 ipfw [ipfw] [patch] ipfw nat redirect_port buf is too smal o kern/143621 ipfw [ipfw] [dummynet] [patch] dummynet and vnet use result o kern/143474 ipfw [ipfw] ipfw table contains the same address f kern/142951 ipfw [dummynet] using pipesqueues gives OUCH! pipe should o kern/139581 ipfw [ipfw] ipfw pipe not limiting bandwidth o kern/139226 ipfw [ipfw] install_state: entry already present, done o kern/137346 ipfw [ipfw] ipfw nat redirect_proto is broken o kern/137232 ipfw [ipfw] parser troubles o kern/136695 ipfw [ipfw] [patch] fwd reached after skipto in dynamic rul o kern/135476 ipfw [ipfw] IPFW table breaks after adding a large number o o bin/134975 ipfw [patch] ipfw(8) can't work with set in rule file. o kern/132553 ipfw [ipfw] ipfw doesn't understand ftp-data port o kern/131817 ipfw [ipfw] blocks layer2 packets that should not be blocke o kern/131601 ipfw [ipfw] [panic] 7-STABLE panic in nat_finalise (tcp=0) o kern/131558 ipfw [ipfw] Inconsistent via ipfw behavior o bin/130132 ipfw [patch] ipfw(8): no way to get mask from ipfw pipe sho o kern/129103 ipfw [ipfw] IPFW check state does not work =( o kern/129093 ipfw [ipfw] ipfw nat must not drop packets o kern/129036 ipfw [ipfw] 'ipfw fwd' does not change outgoing interface n o kern/128260 ipfw [ipfw] [patch] ipfw_divert damages IPv6 packets o kern/127230 ipfw [ipfw] [patch] Feature request to add UID and/or GID l o kern/127209 ipfw [ipfw] IPFW table become corrupted after many changes o bin/125370 ipfw [ipfw] [patch] increase a line buffer limit o conf/123119 ipfw [patch] rc script for ipfw does not handle IPv6 o kern/122963 ipfw [ipfw] tcpdump does not show packets redirected by 'ip s kern/121807 ipfw [request] TCP and UDP port_table in ipfw o kern/121382 ipfw [dummynet]: 6.3-RELEASE-p1 page fault in dummynet (cor o kern/121122 ipfw [ipfw] [patch] add support to ToS IP PRECEDENCE fields o kern/118993 ipfw [ipfw] page fault - probably it's a locking problem o bin/117214 ipfw ipfw(8) fwd with IPv6 treats input as IPv4 o kern/116009 ipfw [ipfw] [patch] Ignore errors when loading ruleset from o docs/113803 ipfw [patch] ipfw(8) - don't get bitten by the fwd rule p kern/113388 ipfw [ipfw] [patch] Addition actions with rules within spec o kern/112561 ipfw [ipfw] ipfw fwd does not work with some TCP packets o kern/105330 ipfw [ipfw] [patch] ipfw (dummynet) does not allow to set q o bin/104921 ipfw [patch] ipfw(8) sometimes treats ipv6 input as ipv4 (a o kern/104682 ipfw [ipfw] [patch] Some minor language consistency fixes a o kern/103454 ipfw [ipfw] [patch] [request] add a facility to modify DF b o kern/103328 ipfw [ipfw] [request] sugestions about ipfw table o kern/102471 ipfw [ipfw] [patch] add tos and dscp support o kern/98831 ipfw [ipfw] ipfw has UDP hickups o kern/97951 ipfw [ipfw] [patch] ipfw does not tie interface details to o kern/97504 ipfw [ipfw] IPFW Rules bug o kern/95084 ipfw [ipfw] [regression] [patch] IPFW2 ignores recv/xmit/v o kern/93300 ipfw [ipfw] ipfw pipe lost packets o kern/91847 ipfw [ipfw] ipfw with vlanX as the device o kern/88659 ipfw [modules] ipfw and ip6fw do not work properly as modul o
Re: Problem with ipfw nat and packet to local services
On Mon, 19 Jul 2010, Mamontov Roman wrote:
What's the value of sysctl net.inet.ip.fw.one_pass ? It needs to be 0
so that packets will re-enter the firewall after NAT processing.
Otherwise, it might help to
a) run 'ipfw zero' before any tests .. I'm wondering about all those
packets hitting rule 65535; were they from before adding rule 65000?
b) add some count rules before and after nat, to show all packets
that may be eligible for NAT translation, maybe something like:
00020 count log ip from any to any in recv ${ext_if}
00022 count log ip from any to any out xmit ${ext_if}
00024 count log ip from any to any out recv ${int_if} xmit ${ext_if}
00035 nat ...
00040 count log ip from any to any in recv ${ext_if}
00042 count log ip from any to any out xmit ${ext_if}
00044 count log ip from any to any out recv ${int_if} xmit ${ext_if}
So you actually get to see the flow of packets before and after nat,
both to/from the local box and packets mapped to/rom inside addresses.
Again, an 'ipfw zero' before tests will make packet counts clearer.
Of course something like '# tcpdump -pn -i ext_if' will also show all
packets via ext_if in some detail. Be more specific if just looking for
some particular flows, like maybe appending 'udp port N' to that.
That is, try to follow packets you'd expect to be coming in for services
on the local box so if they are disappearing, you'll know where or why.
'netstat -finet -an' will show all those services that are listening.
If that doesn't help, we'll need more information.
cheers, Ian
# sysctl net.inet.ip.fw.one_pass
net.inet.ip.fw.one_pass: 0
# ipfw show 20-49
00020402016 count log ip from any to me dst-port 22 in recv ext_if1
00021 0 0 count log ip from me 22 to any out xmit ext_if1
00035 13192 9028716 nat 1 ip from any to any via ext_if1
00040 0 0 count log ip from any to me dst-port 22 in recv ext_if1
00041 0 0 count log ip from me 22 to any out xmit ext_if1
Ouch. It does look like nat is just swallowing the ssh packets on the
inbound pass .. these surely should appear again after NAT (unchanged).
# ipfw nat show config
ipfw nat 1 config ip xxx.xxx.xxx.xxx
# tcpdump -pn -i ext_if1 'host yyy.yyy.yyy.yyy'
14:12:48.885011 IP yyy.yyy.yyy.yyy.2777 xxx.xxx.xxx.xxx.22: Flags [S], seq
2880611174, win 65535, options [mss 1460,nop,wscale 1,nop,nop,TS[|tcp]
14:12:51.23 IP yyy.yyy.yyy.yyy.2777 xxx.xxx.xxx.xxx.22: Flags [S], seq
2880611174, win 65535, options [mss 1460,nop,wscale 1,nop,nop,TS[|tcp]
14:12:54.884966 IP yyy.yyy.yyy.yyy.2777 xxx.xxx.xxx.xxx.22: Flags [S], seq
2880611174, win 65535, options [mss 1460,nop,wscale 1,nop,nop,TS[|tcp]
14:12:57.884090 IP yyy.yyy.yyy.yyy.2777 xxx.xxx.xxx.xxx.22: Flags [S], seq
2880611174, win 65535, options [mss 1460], length 0
14:13:00.885131 IP yyy.yyy.yyy.yyy.2777 xxx.xxx.xxx.xxx.22: Flags [S], seq
2880611174, win 65535, options [mss 1460], length 0
14:13:03.887094 IP yyy.yyy.yyy.yyy.2777 xxx.xxx.xxx.xxx.22: Flags [S], seq
2880611174, win 65535, options [mss 1460], length 0
Ah, if that 'TS' means TSO - tcpdump(1) doesn't mention either - then
from the last section of ipfw(8):
Due to the architecture of libalias(3), ipfw nat is not compatible with
the TCP segmentation offloading (TSO). Thus, to reliably nat your net-
work traffic, please disable TSO on your NICs using ifconfig(8).
ifconfig ext_if1 ?
Output
# netstat -finet -an | grep yyy.yyy.yyy.yyy
is blank.
Without rule 35 nat 1 ip from any to any via ext_if1 inbound packet to ssh
(for example)
pass correctly.
# ipfw delete 35
tcpdump -pn -i ext_if 'host yyy.yyy.yyy.yyy'
14:21:45.467233 IP yyy.yyy.yyy.yyy.2790 xxx.xxx.xxx.xxx.22: Flags [S], seq
376101413, win 65535, options [mss 1460,nop,wscale 1,nop,nop,TS[|tcp]
14:21:45.467670 IP xxx.xxx.xxx.xxx.22 xxx.xxx.xxx.xxx.2790: Flags [S.],
seq 3270699616, ack 376101414, win 65535, options [mss 1460,nop,wscale
3,nop,nop,TS[|tcp]
I guess that dest is just mis-obscured, ie yyy.yyy.yyy.yyy:2790
14:21:45.468960 IP yyy.yyy.yyy.yyy.2790 xxx.xxx.xxx.xxx.22: Flags [.], ack
1, win 33304, options [nop,nop,TS val 40088404 ecr 1166915706], length 0
14:21:45.527438 IP xxx.xxx.xxx.xxx.22 yyy.yyy.yyy.yyy.2790: Flags [P.],
ack 1, win 8326, options [nop,nop,TS val 1166915766 ecr 40088404], length 40
# netstat -finet -an | grep yyy.yyy.yyy.yyy
tcp4 0 0 xxx.xxx.xxx.xxx.22yyy.yyy.yyy.yyy.2790
FIN_WAIT_2
00020 8 1403 count log ip from any to me dst-port 22 in recv ext_if1
00021 6 2280 count log ip from me 22 to any out xmit ext_if1
00040 8 1403 count log ip from any to me dst-port 22 in recv ext_if1
00041 6 2280 count log ip from me 22 to any out xmit ext_if1
Any ideas?
[later: if you had TSO enabled on the interface,
