Re: [pfSense] Connection issues after changing from 2.0.3-RELEASE to 2.1-RELEASE

2013-12-21 Thread Chris Buechler
My guess at the most likely cause is incorrect policy routing (proper
negation rules not in place), where in earlier versions the negate
rules could be excessively permissive and were tightened down in 2.1.
Check the states on the traffic, if that's the case you'll see it
getting NATed out to the Internet.

On Thu, Dec 12, 2013 at 2:46 AM, Stefan Baur
 wrote:
> Hi List,
>
> I'm kind of stumped by this.
>
> Situation:
> pfSsense with 3 Interfaces
> WAN: 192.168.0.161/24 (DHCP)
> LAN: 192.168.133.1/24
> OPT1 (renamed "DMZ"): 172.16.0.2/12
>
> What I did:
> 1) fresh install of 2.1-RELEASE as virtual machine (KVM on Linux host), with
> same virtual machine settings as for 2.0.3-RELEASE
> 2) enabled virtio network and disk devices as described on the wiki
> 3) loaded config of 2.0.3-RELEASE and changed interface/disk names as
> required
>
> What I did NOT do until the issue appeared:
> change any firewall rules
>
> Issue:
> After upgrading, all TCPv4 connections (and possibly UDPv4 too) between LAN
> and OPT1 (and vice versa) do not work
> (Haven't tried IPv6 at all, since I don't have that configured)
>
> What I'm trying is (as an example, http/proxy connections via 80/8080/3128
> don't work, either):
> ssh from 192.168.133.100 to 172.16.0.110
>
> Firewall log shows that the connection is being attempted and allowed:
> pass Dec 12 08:51:10 LAN 192.168.133.100:51876 172.16.0.110:22 TCP:S
>
> Result: Nothing happens, connection silently times out
>
> Same goes for the opposite direction, I try:
> ssh from 172.16.0.110 to 192.168.133.100
>
> Firewall log shows that the connection is being attempted and allowed:
> pass Dec 12 09:02:55 DMZ 172.16.0.110:52172 192.168.133.100:22 TCP:S
>
> Result: Nothing happens, connection silently times out
>
> On 192.168.133.100, the source machine, netstat -rn gives:
> Kernel IP routing table
> Destination Gateway Genmask Flags   MSS Window  irtt
> Iface
> 0.0.0.0 192.168.0.1 0.0.0.0 UG0 0  0 br1
> 172.16.0.0  192.168.133.1   255.240.0.0 UG0 0  0 br0
> 192.168.0.0 0.0.0.0 255.255.255.0   U 0 0  0 br1
> 192.168.133.0   0.0.0.0 255.255.255.0   U 0 0  0 br0
>
> On the pfSense box, netstat -rn gives:
> netstat -rn
> Routing tables
>
> Internet:
> DestinationGatewayFlagsRefs  Use Netif Expire
> default192.168.0.1UGS 0 7611 vtnet0
> 127.0.0.1  link#4 UH  0   28 lo0
> 172.16.0.0/12  link#3 U   017787 vtnet2
> 172.16.0.2 link#3 UHS 00 lo0
> 192.168.0.0/24 link#1 U   0 7669 vtnet0
> 192.168.0.152:54:00:60:93:00  UHS 019408 vtnet0
> 192.168.0.161  link#1 UHS 00 lo0
> 192.168.133.0/24   link#2 U   0 2193 vtnet1
> 192.168.133.1  link#2 UHS 00 lo0
>
> Internet6:
> Destination   Gateway Flags  Netif Expire
> ::1   ::1 UH  lo0
> fe80::%vtnet0/64  link#1 Uvtnet0
> fe80::5054:ff:fe60:9300%vtnet0link#1 UHS lo0
> fe80::%vtnet1/64  link#2 Uvtnet1
> fe80::5054:ff:fe60:9301%vtnet1link#2 UHS lo0
> fe80::%vtnet2/64  link#3 Uvtnet2
> fe80::5054:ff:fe60:9302%vtnet2link#3 UHS lo0
> fe80::%lo0/64 link#4 U   lo0
> fe80::1%lo0   link#4 UHS lo0
> ff01::%vtnet0/32  fe80::5054:ff:fe60:9300%vtnet0 U
> vtnet0
> ff01::%vtnet1/32  fe80::5054:ff:fe60:9301%vtnet1 U
> vtnet1
> ff01::%vtnet2/32  fe80::5054:ff:fe60:9302%vtnet2 U
> vtnet2
> ff01::%lo0/32 ::1 U   lo0
> ff02::%vtnet0/32  fe80::5054:ff:fe60:9300%vtnet0 U
> vtnet0
> ff02::%vtnet1/32  fe80::5054:ff:fe60:9301%vtnet1 U
> vtnet1
> ff02::%vtnet2/32  fe80::5054:ff:fe60:9302%vtnet2 U
> vtnet2
> ff02::%lo0/32 ::1 U   lo0
>
> Funnily, when I ssh to 192.168.133.1 (the pfSense box), I can continue to
> ssh to 172.16.0.110 just fine from there.
>
> On 172.16.0.110, the destination machine, netstat -rn gives:
> Kernel IP routing table
> Destination Gateway Genmask Flags   MSS Window  irtt
> Iface
> 0.0.0.0 172.16.0.2  0.0.0.0 UG0 0  0
> eth0
> 172.16.0.0  0.0.0.0 255.240.0.0 U 0 0  0
> eth0
>
>
> Connections between OPT1/DMZ and WAN work just fine:
> When I try:
> ssh from 172.16.0.110 to 192.168.0.10
>
> The firewall log shows:
> pass Dec 12 09:19:50 DMZ 172.16.0.110:43745 192.168.0.10:22 TCP:S
>
> And it connects just fine.
>
> Since this is a test environment, I had no pain disabling all firewall r

[pfSense] Connection issues after changing from 2.0.3-RELEASE to 2.1-RELEASE

2013-12-12 Thread Stefan Baur

Hi List,

I'm kind of stumped by this.

Situation:
pfSsense with 3 Interfaces
WAN: 192.168.0.161/24 (DHCP)
LAN: 192.168.133.1/24
OPT1 (renamed "DMZ"): 172.16.0.2/12

What I did:
1) fresh install of 2.1-RELEASE as virtual machine (KVM on Linux host), 
with same virtual machine settings as for 2.0.3-RELEASE

2) enabled virtio network and disk devices as described on the wiki
3) loaded config of 2.0.3-RELEASE and changed interface/disk names as 
required


What I did NOT do until the issue appeared:
change any firewall rules

Issue:
After upgrading, all TCPv4 connections (and possibly UDPv4 too) between 
LAN and OPT1 (and vice versa) do not work

(Haven't tried IPv6 at all, since I don't have that configured)

What I'm trying is (as an example, http/proxy connections via 
80/8080/3128 don't work, either):

ssh from 192.168.133.100 to 172.16.0.110

Firewall log shows that the connection is being attempted and allowed:
pass Dec 12 08:51:10 LAN 192.168.133.100:51876 172.16.0.110:22 TCP:S

Result: Nothing happens, connection silently times out

Same goes for the opposite direction, I try:
ssh from 172.16.0.110 to 192.168.133.100

Firewall log shows that the connection is being attempted and allowed:
pass Dec 12 09:02:55 DMZ 172.16.0.110:52172 192.168.133.100:22 TCP:S

Result: Nothing happens, connection silently times out

On 192.168.133.100, the source machine, netstat -rn gives:
Kernel IP routing table
Destination Gateway Genmask Flags   MSS Window  irtt 
Iface

0.0.0.0 192.168.0.1 0.0.0.0 UG0 0  0 br1
172.16.0.0  192.168.133.1   255.240.0.0 UG0 0  0 br0
192.168.0.0 0.0.0.0 255.255.255.0   U 0 0  0 br1
192.168.133.0   0.0.0.0 255.255.255.0   U 0 0  0 br0

On the pfSense box, netstat -rn gives:
netstat -rn
Routing tables

Internet:
DestinationGatewayFlagsRefs  Use Netif Expire
default192.168.0.1UGS 0 7611 vtnet0
127.0.0.1  link#4 UH  0   28 lo0
172.16.0.0/12  link#3 U   017787 vtnet2
172.16.0.2 link#3 UHS 00 lo0
192.168.0.0/24 link#1 U   0 7669 vtnet0
192.168.0.152:54:00:60:93:00  UHS 019408 vtnet0
192.168.0.161  link#1 UHS 00 lo0
192.168.133.0/24   link#2 U   0 2193 vtnet1
192.168.133.1  link#2 UHS 00 lo0

Internet6:
Destination   Gateway Flags  Netif Expire
::1   ::1 UH  lo0
fe80::%vtnet0/64  link#1 Uvtnet0
fe80::5054:ff:fe60:9300%vtnet0link#1 UHS lo0
fe80::%vtnet1/64  link#2 Uvtnet1
fe80::5054:ff:fe60:9301%vtnet1link#2 UHS lo0
fe80::%vtnet2/64  link#3 Uvtnet2
fe80::5054:ff:fe60:9302%vtnet2link#3 UHS lo0
fe80::%lo0/64 link#4 U   lo0
fe80::1%lo0   link#4 UHS lo0
ff01::%vtnet0/32  fe80::5054:ff:fe60:9300%vtnet0 
Uvtnet0
ff01::%vtnet1/32  fe80::5054:ff:fe60:9301%vtnet1 
Uvtnet1
ff01::%vtnet2/32  fe80::5054:ff:fe60:9302%vtnet2 
Uvtnet2

ff01::%lo0/32 ::1 U   lo0
ff02::%vtnet0/32  fe80::5054:ff:fe60:9300%vtnet0 
Uvtnet0
ff02::%vtnet1/32  fe80::5054:ff:fe60:9301%vtnet1 
Uvtnet1
ff02::%vtnet2/32  fe80::5054:ff:fe60:9302%vtnet2 
Uvtnet2

ff02::%lo0/32 ::1 U   lo0

Funnily, when I ssh to 192.168.133.1 (the pfSense box), I can continue 
to ssh to 172.16.0.110 just fine from there.


On 172.16.0.110, the destination machine, netstat -rn gives:
Kernel IP routing table
Destination Gateway Genmask Flags   MSS Window  irtt 
Iface
0.0.0.0 172.16.0.2  0.0.0.0 UG0 0  0 
eth0
172.16.0.0  0.0.0.0 255.240.0.0 U 0 0  0 
eth0



Connections between OPT1/DMZ and WAN work just fine:
When I try:
ssh from 172.16.0.110 to 192.168.0.10

The firewall log shows:
pass Dec 12 09:19:50 DMZ 172.16.0.110:43745 192.168.0.10:22 TCP:S

And it connects just fine.

Since this is a test environment, I had no pain disabling all firewall 
rules and adding a
"pass, log IPv4 * * * * * * none" at the top 
of every interface ruleset.


This doesn't change a thing with regards to the TCP packages though (and 
yes, I hit the "reload" button):


After making this change, I can Ping from anywhere to anywhere just fine.
SSH/HTTP/Proxy still doesn't work across interfaces.
So it seems only TCP (and possibly UDP) is affected by whatever is 
causing this, while ICMP goes through (as long as there's no blocking rule).