Hello all,
I have a strange problem. Over the weekend, I replaced an older M0n0wall
installation that has been faithfully serving me for several years with a
PFSense installation on a Soekris net6501. I had installed 2.1 on the
flash, and imported the config from the M0n0wall box (which worked
flawlessly) and re-assigned the interfaces. I had preconfigured everything
off-site before taking the 2 hour drive up to the customers' location.
Everything seemed to go very smoothly the VLANs and IPSec tunnels all came
up and passed traffic without a single complaint.
One of the reasons for this change was that my customer is going to be
leaning hard on a hosted solution, so a secondary WAN connection can be
justified. I had preconfigured a gateway group with both the primary
(Cable) and secondary (DSL) interfaces, and added changed the gateway to
the group for the firewall rules where failover would be needed. Of course
when I got on-site, there was an issue with the DSL so I could not test the
failover. (The ISP is still working on it).
I ran into my smaller problem as I was doing some last minute testing
before leaving. I found that If the LAN firewall rules used the gateway
group, rather than the default (cable) gateway, traffic that should have
been routed over the IPSec VPN was sent out the WAN interface instead. I
double checked my phase 2 entry, and even stopped/started IPSec but the
only way I could send traffic over the VPN was to add a rule that allowed
traffic from the LAN bound for the remote side of the VPN and send it out
on the default gateway (Cable). This is the smaller of my two problems,
since I will most likely be looking into switching over to OpenVPN once I
get the DSL up and running. I would appreciate it if someone has an idea
what I might be doing wrong here.
My bigger problem, did't come to light until this morning. My client is
unable to access the web site for their hosted application, and a few other
secure web sites. Every other site works fine. I can ping the address for
the hosted app, but I can't browse to the site. This affects every computer
in the facility. Its not like its isolated to a machine or two.
At first, I was sure that the hosting provider had us blocked somehow. I
could access the hosted app from my office, and every other place that I
tried. But users were having difficulty getting onto CMS (medicare) as
well, and a few banking sites. There are no rules preventing outbound HTTPS
and there is nothing in the firewall logs that I can find where this
traffic is getting blocked. Even so, I removed the gateway group, had them
reboot the cable modem, turned off the advanced outbound NAT rules.
There is one more strange thing:
The trace route. Take a look. (This is from a PC)
Tracing route to login.sigmacare.com [40.143.159.11]
over a maximum of 30 hops:
1 <1 ms 1 ms <1 ms ool-xxxxxxx.static.optonline.net[108.58.xxx.xxx]
2 8 ms 10 ms 7 ms 192.168.0.2
3 15 ms 7 ms 25 ms 433be0d9.cst.lightpath.net [67.59.242.217]
4 13 ms 13 ms 13 ms ool-4353f889.dyn.optonline.net[67.83.248.137]
5 12 ms 15 ms 12 ms 64.15.7.45
6 20 ms 12 ms 13 ms 451be0c6.cst.lightpath.net [65.19.120.198]
7 14 ms 14 ms 12 ms xe-10-0-2.edge3.NewYork1.Level3.net[4.31.16.157]
8 40 ms 41 ms 46 ms vlan70.csw2.NewYork1.Level3.net[4.69.155.126]
9 40 ms 40 ms 41 ms ae-72-72.ebr2.NewYork1.Level3.net[4.69.148.37]
10 47 ms 45 ms 42 ms ae-47-47.ebr2.NewYork2.Level3.net[4.69.201.34]
11 42 ms 41 ms 41 ms ae-2-2.ebr1.Chicago1.Level3.net[4.69.132.65]
12 43 ms 40 ms 43 ms ae-1-100.ebr2.Chicago1.Level3.net[4.69.151.178]
13 41 ms 65 ms 41 ms ae-5-5.ebr2.Chicago2.Level3.net[4.69.140.194]
14 42 ms 40 ms 41 ms
ae-12-12.car1.Louisville1.Level3.net[4.69.140.213]
15 73 ms 147 ms 91 ms
ae-11-11.car2.Louisville1.Level3.net[4.69.140.218]
16 40 ms 47 ms 39 ms ae-2-2.car2.Nashville1.Level3.net[4.69.140.222]
17 41 ms 50 ms 41 ms
ae-11-11.car1.Nashville1.Level3.net[4.69.140.226]
18 46 ms 45 ms 43 ms
WINDSTREAM.car1.Nashville1.Level3.net[166.90.118.222]
19 46 ms 47 ms 45 ms 209.59.31.205
20 59 ms 55 ms 47 ms h11.159.143.40.ip.windstream.net[40.143.159.11]
192.168.0.2 is the IP Address of the firewall's LAN interface.
Why is it showing up on the second hop?
Now from PFSense itself:
1 ool-xxxxxxx.static.optonline.net (108.58.xxx.xxx) 0.836 ms 0.924 ms
0.711 ms
2 0.0.0.0 (0.0.0.0) 7.697 ms 7.671 ms 10.134 ms
3 433be0d9.cst.lightpath.net (67.59.242.217) 7.911 ms 8.826 ms 8.209 ms
4 rtr3-ge1-8.mhe.prnynj.cv.net (67.83.248.129) 10.511 ms
ool-4353f889.dyn.optonline.net (67.83.248.137) 13.918 ms 14.444 ms
5 65.19.119.205 (65.19.119.205) 11.047 ms
65.19.119.221 (65.19.119.221) 11.917 ms
64.15.7.61 (64.15.7.61) 15.342 ms
6 451be0c6.cst.lightpath.net (65.19.120.198) 13.221 ms 12.337 ms
12.574 ms
7 xe-10-0-1.edge3.newyork1.level3.net (4.31.16.153) 11.429 ms
xe-5-0-0.edge3.NewYork1.Level3.net (4.31.26.73) 14.695 ms
xe-10-0-2.edge3.newyork1.level3.net (4.31.16.157) 12.677 ms
8 vlan80.csw3.NewYork1.Level3.net (4.69.155.190) 43.733 ms
vlan60.csw1.newyork1.level3.net (4.69.155.62) 46.496 ms
vlan80.csw3.NewYork1.Level3.net (4.69.155.190) 44.102 ms
9 ae-62-62.ebr2.newyork1.level3.net (4.69.148.33) 40.238 ms
ae-72-72.ebr2.newyork1.level3.net (4.69.148.37) 42.651 ms
ae-62-62.ebr2.newyork1.level3.net (4.69.148.33) 43.854 ms
10 ae-48-48.ebr2.NewYork2.Level3.net (4.69.201.38) 43.732 ms
ae-45-45.ebr2.newyork2.level3.net (4.69.141.22) 42.569 ms 39.853 ms
11 ae-2-2.ebr1.chicago1.level3.net (4.69.132.65) 40.176 ms 40.578 ms
46.843 ms
12 ae-1-100.ebr2.chicago1.level3.net (4.69.151.178) 42.693 ms
ae-6-6.ebr1.Chicago2.Level3.net (4.69.140.190) 44.580 ms 43.029 ms
13 ae-5-5.ebr2.chicago2.level3.net (4.69.140.194) 43.914 ms
ae-1-100.ebr2.chicago2.level3.net (4.69.132.114) 41.068 ms
ae-5-5.ebr2.chicago2.level3.net (4.69.140.194) 42.319 ms
14 ae-12-12.car1.louisville1.level3.net (4.69.140.213) 40.179 ms 45.389
ms 40.538 ms
15 ae-11-11.car2.louisville1.level3.net (4.69.140.218) 52.879 ms 40.046
ms 42.327 ms
16 ae-2-2.car2.nashville1.level3.net (4.69.140.222) 40.072 ms 43.921 ms
44.399 ms
17 ae-11-11.car1.nashville1.level3.net (4.69.140.226) 42.108 ms 43.246
ms 40.879 ms
18 WINDSTREAM.car1.Nashville1.Level3.net (166.90.118.222) 44.016 ms
48.296 ms 43.148 ms
19 209.59.31.205 (209.59.31.205) 49.072 ms 46.723 ms 47.830 ms
20 * * *
The second hop is 0.0.0.0?
Almost everything seems to work, so I don't even know if this is a factor.
Is it even possible that this could be a firewall issue? Only certain
secure sites have an issue, others work just fine. I am probably going to
need to make a return trip very soon, and I am hoping I have more things to
try than plugging the old m0n0wall back in.
Thanks,
Jim
_______________________________________________
List mailing list
[email protected]
https://lists.pfsense.org/mailman/listinfo/list
