Hi all,
Suddenly I am facing a problem on a new PC, using a configuration that I
have been using on more than 10 servers for the last few years. The only
thing that I find that differs from my other configuratinos is the NIC
of the PC. If not, I must be missing something very trivial.
I have built a jail on this PC, following the handbook's guidelines
(section: application of jails). The PC has one NIC, msk0, where I run
pf on (built on my kernel; I have already tried using the module). My
pf.conf is as simple as possible:
# cat /etc/pf.conf
nat on msk0 from any to any - 10.0.3.6
pass quick all
when I jexec inside the jail, and pf is running, I am unable to reach
any machine except my jail (not even the host). If pf is off, the
network works just fine (of course my router knows where to find my
jail's subnet).
What is strange is that if I tcpdump on msk0, then after a few seconds
that I request something from within the jail, I see the packets going
and coming on msk0 using the correct IP (the NAT IP), but it seems that
the machine fails to route them back inside the jail.
My configuration is as follows:
#uname -a
FreeBSD filesrv.svr.noca 9.0-STABLE FreeBSD 9.0-STABLE #1: Fri Jul 27
15:40:48 EEST 2012
r...@filesrv.svr.noca:/usr/obj/usr/src/sys/MAMALOPYRINO amd64
#ifconfig -a
msk0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500
options=c011bRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,TSO4,VLAN_HWTSO,LINKSTATE
ether 80:ee:73:10:a3:58
inet 10.0.3.6 netmask 0xff00 broadcast 10.0.3.255
inet6 fe80::82ee:73ff:fe10:a358%msk0 prefixlen 64 scopeid 0x1
nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL
media: Ethernet autoselect (1000baseT
full-duplex,flowcontrol,rxpause,txpause)
status: active
pflog0: flags=0 metric 0 mtu 33152
nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL
pfsync0: flags=0 metric 0 mtu 1500
nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL
syncpeer: 0.0.0.0 maxupd: 128
lo0: flags=8049UP,LOOPBACK,RUNNING,MULTICAST metric 0 mtu 16384
options=3RXCSUM,TXCSUM
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x9
inet 127.0.0.1 netmask 0xff00
nd6 options=21PERFORMNUD,AUTO_LINKLOCAL
lo1: flags=8049UP,LOOPBACK,RUNNING,MULTICAST metric 0 mtu 16384
options=3RXCSUM,TXCSUM
inet 10.3.2.1 netmask 0xff00
nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL
tap1: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500
options=8LINKSTATE
ether 00:bd:7b:c3:0c:01
inet6 fe80::2bd:7bff:fec3:c01%tap1 prefixlen 64 scopeid 0xb
inet 10.3.2.2 netmask 0xff00 broadcast 10.3.2.255
nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL
tap2: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500
options=8LINKSTATE
ether 00:bd:7f:c3:0c:02
inet6 fe80::2bd:7fff:fec3:c02%tap2 prefixlen 64 scopeid 0xc
nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL
lo3: flags=8049UP,LOOPBACK,RUNNING,MULTICAST metric 0 mtu 16384
options=3RXCSUM,TXCSUM
inet 10.3.2.3 netmask 0xff00
nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL
lo3 is used as my jail interface, msk0 is my lan interface.
# pciconf -v
mskc0@pci0:3:0:0: class=0x02 card=0x40011297 chip=0x438011ab
rev=0x10 hdr=0x00
vendor = 'Marvell Technology Group Ltd.'
device = '88E8057 PCI-E Gigabit Ethernet Controller'
class = network
subclass = ethernet
excerpt of /etc/rc.conf:
jail_test_hostname=test.svr.noca
jail_test_rootdir=/jails/j/test
jail_test_devfs_enable=YES
jail_test_ip=10.3.2.3/24
jail_test_interface=lo3
I have even enabled forwarding and fast forwarding (just in case that
this had been the case) with non results.
# netstat -rn
Routing tables
Internet:
DestinationGatewayFlagsRefs Use Netif Expire
default10.0.3.1 UGS 0 290 msk0
10.0.3.0/24link#1 U 018825 msk0
10.0.3.6 link#1 UHS 01lo0
10.3.2.0/24link#11U 00 tap1
10.3.2.1 link#10UH 00lo1
10.3.2.2 link#11UHS 0 61lo0
10.3.2.3 link#13UH 00lo3
127.0.0.1 link#9 UH 0 64lo0
Since I don't need NAT on my configuration, I will use simple routing
instead, so there won't be a problem for me. I am just sending this info
in case this is a bug with pf-msk driver (for the specific card?) and
before I send a bug report, I'd like a second opinion in case I am
missing something fundamental.
Thanx all in advance.
--
George Mamalakis
IT and Security Officer
Electrical and Computer Engineer (Aristotle Un. of Thessaloniki),
MSc (Imperial College of London)
Department of Electrical and Computer Engineering
Faculty of Engineering
