Re: [pfSense-discussion] Very odd issue - Transparent Firewall - 2 Locations
On Thu, Jul 16, 2009 at 4:01 AM, Angus Jordan wrote: > > I had configured the servers behind the pfsense bridge with the > gateway pointing directly at the pfsense firewall. When I modified the > gateway on the servers to use the real upstream gateway, all is > normal. > Ah yeah, that'll do it. Logs were strange (not now that I know what you were doing), only showing 1500 byte frames getting blocked, and from your earlier description that mostly emails with attachments were having issues, seemed maybe a smaller MTU would fix things. - To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com For additional commands, e-mail: discussion-h...@pfsense.com Commercial support available - https://portal.pfsense.org
Re: [pfSense-discussion] Very odd issue - Transparent Firewall - 2 Locations
On Thu, Jul 16, 2009 at 12:37 AM, Chris Buechler wrote: > On Thu, Jul 16, 2009 at 3:22 AM, Angus Jordan wrote: >> Hi again, >> >> I've attached the logs directly from the /var/log/filter.log. These >> show up at exactly the same time the download stops... >> > > What happens if you lower the MTU on the server to 1450? Hmm..OK, I didn't try that. But I believe I found my mistake. I had configured the servers behind the pfsense bridge with the gateway pointing directly at the pfsense firewall. When I modified the gateway on the servers to use the real upstream gateway, all is normal. That'll teach me for migrating a client to a new connection at 1 in the morning. :) Ciao Angus - To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com For additional commands, e-mail: discussion-h...@pfsense.com Commercial support available - https://portal.pfsense.org
Re: [pfSense-discussion] Very odd issue - Transparent Firewall - 2 Locations
On Thu, Jul 16, 2009 at 3:22 AM, Angus Jordan wrote: > Hi again, > > I've attached the logs directly from the /var/log/filter.log. These > show up at exactly the same time the download stops... > What happens if you lower the MTU on the server to 1450? - To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com For additional commands, e-mail: discussion-h...@pfsense.com Commercial support available - https://portal.pfsense.org
Re: [pfSense-discussion] Very odd issue - Transparent Firewall - 2 Locations
Hi yet again, Here are the interface assignments: WAN: em1 LAN: em0 Regards, Angus On Thu, Jul 16, 2009 at 12:22 AM, Angus Jordan wrote: > Hi again, > > I've attached the logs directly from the /var/log/filter.log. These > show up at exactly the same time the download stops... > > Let me know what other information I can provide. > > Thanks, > Angus > > > On Thu, Jul 16, 2009 at 12:06 AM, Angus Jordan wrote: >> Hi Chris, >> >>> Make sure you're using e1000 interfaces. Also might want to try >>> "disable checksum offload" under System -> Advanced. >> >> Both of these options are selected, same symptoms..although it does >> take much longer for the problem to creep up. >> >> Unfortunately this is mainly affecting outbound email, the connections >> just seem to time out. More-so when there are attachments, but also >> sometimes even without attachments.. >> >> Regards, >> Angus >> > - To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com For additional commands, e-mail: discussion-h...@pfsense.com Commercial support available - https://portal.pfsense.org
Re: [pfSense-discussion] Very odd issue - Transparent Firewall - 2 Locations
Hi again, I've attached the logs directly from the /var/log/filter.log. These show up at exactly the same time the download stops... Let me know what other information I can provide. Thanks, Angus On Thu, Jul 16, 2009 at 12:06 AM, Angus Jordan wrote: > Hi Chris, > >> Make sure you're using e1000 interfaces. Also might want to try >> "disable checksum offload" under System -> Advanced. > > Both of these options are selected, same symptoms..although it does > take much longer for the problem to creep up. > > Unfortunately this is mainly affecting outbound email, the connections > just seem to time out. More-so when there are attachments, but also > sometimes even without attachments.. > > Regards, > Angus > Jul 16 00:10:00 bb-t-fw pf: 1. 138356 rule 54/0(match): block out on em1: (tos 0x0, ttl 47, id 56469, offset 0, flags [DF], proto TCP (6), length 1500) 80.249.99.148.80 > down.loading.ser.ver.36129: . 315937400:315938848(1448) ack 3130337126 win 12 Jul 16 00:10:00 bb-t-fw pf: 82 rule 54/0(match): block out on em1: (tos 0x0, ttl 47, id 58573, offset 0, flags [DF], proto TCP (6), length 1500) 80.249.99.148.80 > down.loading.ser.ver.36129: . 1448:2896(1448) ack 1 win 12 Jul 16 00:10:00 bb-t-fw pf: 000141 rule 54/0(match): block out on em1: (tos 0x0, ttl 47, id 11460, offset 0, flags [DF], proto TCP (6), length 1500) 80.249.99.148.80 > down.loading.ser.ver.36129: . 2896:4344(1448) ack 1 win 12 Jul 16 00:10:00 bb-t-fw pf: 000107 rule 54/0(match): block out on em1: (tos 0x0, ttl 47, id 35797, offset 0, flags [DF], proto TCP (6), length 1500) 80.249.99.148.80 > down.loading.ser.ver.36129: . 4344:5792(1448) ack 1 win 12 Jul 16 00:10:00 bb-t-fw pf: 000121 rule 54/0(match): block out on em1: (tos 0x0, ttl 47, id 42486, offset 0, flags [DF], proto TCP (6), length 1500) 80.249.99.148.80 > down.loading.ser.ver.36129: . 5792:7240(1448) ack 1 win 12 Jul 16 00:10:00 bb-t-fw pf: 000122 rule 54/0(match): block out on em1: (tos 0x0, ttl 47, id 55549, offset 0, flags [DF], proto TCP (6), length 1500) 80.249.99.148.80 > down.loading.ser.ver.36129: . 7240:8688(1448) ack 1 win 12 Jul 16 00:10:00 bb-t-fw pf: 000123 rule 54/0(match): block out on em1: (tos 0x0, ttl 47, id 41630, offset 0, flags [DF], proto TCP (6), length 1500) 80.249.99.148.80 > down.loading.ser.ver.36129: . 8688:10136(1448) ack 1 win 12 Jul 16 00:10:00 bb-t-fw pf: 000123 rule 54/0(match): block out on em1: (tos 0x0, ttl 47, id 1481, offset 0, flags [DF], proto TCP (6), length 1500) 80.249.99.148.80 > down.loading.ser.ver.36129: . 10136:11584(1448) ack 1 win 12 Jul 16 00:10:00 bb-t-fw pf: 000124 rule 54/0(match): block out on em1: (tos 0x0, ttl 47, id 749, offset 0, flags [DF], proto TCP (6), length 1500) 80.249.99.148.80 > down.loading.ser.ver.36129: . 11584:13032(1448) ack 1 win 12 Jul 16 00:10:00 bb-t-fw pf: 000123 rule 54/0(match): block out on em1: (tos 0x0, ttl 47, id 45017, offset 0, flags [DF], proto TCP (6), length 1500) 80.249.99.148.80 > down.loading.ser.ver.36129: . 13032:14480(1448) ack 1 win 12 Jul 16 00:10:00 bb-t-fw pf: 000125 rule 54/0(match): block out on em1: (tos 0x0, ttl 47, id 20362, offset 0, flags [DF], proto TCP (6), length 1500) 80.249.99.148.80 > down.loading.ser.ver.36129: . 14480:15928(1448) ack 1 win 12 Jul 16 00:10:00 bb-t-fw pf: 000124 rule 54/0(match): block out on em1: (tos 0x0, ttl 47, id 12960, offset 0, flags [DF], proto TCP (6), length 1500) 80.249.99.148.80 > down.loading.ser.ver.36129: . 15928:17376(1448) ack 1 win 12 Jul 16 00:10:00 bb-t-fw pf: 000122 rule 54/0(match): block out on em1: (tos 0x0, ttl 47, id 19675, offset 0, flags [DF], proto TCP (6), length 1500) 80.249.99.148.80 > down.loading.ser.ver.36129: . 17376:18824(1448) ack 1 win 12 Jul 16 00:10:00 bb-t-fw pf: 000122 rule 54/0(match): block out on em1: (tos 0x0, ttl 47, id 25481, offset 0, flags [DF], proto TCP (6), length 1500) 80.249.99.148.80 > down.loading.ser.ver.36129: . 18824:20272(1448) ack 1 win 12 Jul 16 00:10:00 bb-t-fw pf: 000152 rule 54/0(match): block out on em1: (tos 0x0, ttl 47, id 12747, offset 0, flags [DF], proto TCP (6), length 1500) 80.249.99.148.80 > down.loading.ser.ver.36129: . 20272:21720(1448) ack 1 win 12 Jul 16 00:10:00 bb-t-fw pf: 99 rule 54/0(match): block out on em1: (tos 0x0, ttl 47, id 51650, offset 0, flags [DF], proto TCP (6), length 1500) 80.249.99.148.80 > down.loading.ser.ver.36129: . 21720:23168(1448) ack 1 win 12 Jul 16 00:10:00 bb-t-fw pf: 000121 rule 54/0(match): block out on em1: (tos 0x0, ttl 47, id 49819, offset 0, flags [DF], proto TCP (6), length 1500) 80.249.99.148.80 > down.loading.ser.ver.36129: . 23168:24616(1448) ack 1 win 12 Jul 16 00:10:00 bb-t-fw pf: 000123 rule 54/0(match): block out on em1: (tos 0x0, ttl 47, id 62336, offset 0, flags [DF], proto TCP (6), length 1500) 80.249.99.148.80 > down.loading.ser.ver.36129: . 24616:26064(1448) ack 1 win 12 Jul 16 00:10:00 bb-t-fw pf: 000123 rule 54/0(match): block out on em1: (tos 0x
Re: [pfSense-discussion] Very odd issue - Transparent Firewall - 2 Locations
On Thu, Jul 16, 2009 at 3:06 AM, Angus Jordan wrote: > Hi Chris, > >> Make sure you're using e1000 interfaces. Also might want to try >> "disable checksum offload" under System -> Advanced. > > Both of these options are selected, same symptoms..although it does > take much longer for the problem to creep up. > > Unfortunately this is mainly affecting outbound email, the connections > just seem to time out. More-so when there are attachments, but also > sometimes even without attachments.. > Paste some of the firewall logs you're seeing, raw logs from status.php. - To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com For additional commands, e-mail: discussion-h...@pfsense.com Commercial support available - https://portal.pfsense.org
Re: [pfSense-discussion] Very odd issue - Transparent Firewall - 2 Locations
Hi Chris, > Make sure you're using e1000 interfaces. Also might want to try > "disable checksum offload" under System -> Advanced. Both of these options are selected, same symptoms..although it does take much longer for the problem to creep up. Unfortunately this is mainly affecting outbound email, the connections just seem to time out. More-so when there are attachments, but also sometimes even without attachments.. Regards, Angus - To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com For additional commands, e-mail: discussion-h...@pfsense.com Commercial support available - https://portal.pfsense.org
Re: [pfSense-discussion] Very odd issue - Transparent Firewall - 2 Locations
On Wed, Jul 15, 2009 at 6:57 PM, Angus Jordan wrote: > Hi Greg, > > Yes, the pfSense does show blocks in on the wan interface. I wish I > could send them to you, but for some reason since you sent this email > the issue seems to have stopped...but it will be back, I know that. > > One thing that I failed to mention in my earlier email is that both of > these pfSense firewalls are running inside of VMWare Server (1.0.9) on > top of Debian hosts. I know this is not the cause of the issue though, > since these problems existed before we virtualized the firewall at one > of the sites... Make sure you're using e1000 interfaces. Also might want to try "disable checksum offload" under System -> Advanced. - To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com For additional commands, e-mail: discussion-h...@pfsense.com Commercial support available - https://portal.pfsense.org
Re: [pfSense-discussion] Very odd issue - Transparent Firewall - 2 Locations
Hi Greg, Yes, the pfSense does show blocks in on the wan interface. I wish I could send them to you, but for some reason since you sent this email the issue seems to have stopped...but it will be back, I know that. One thing that I failed to mention in my earlier email is that both of these pfSense firewalls are running inside of VMWare Server (1.0.9) on top of Debian hosts. I know this is not the cause of the issue though, since these problems existed before we virtualized the firewall at one of the sites... Thanks, Angus On Wed, Jul 15, 2009 at 2:18 PM, Greg Hennessy wrote: > > > Possibly an issue with TCP window scaling or PMTU-D. > > Are the logs generating any drops for the flow ? > > -Original Message- > From: Angus Jordan [mailto:angus.jor...@gmail.com] > Sent: 15 July 2009 22:08 > To: discussion@pfsense.com > Subject: [pfSense-discussion] Very odd issue - Transparent Firewall - 2 > Locations > > Hi there, > > We have deployed 2 pfSense Transparent firewalls at 2 separate > locations. The commonality between both locations is their Internet > provider. > > 1) pfSense running directly in the providers co-location (Customer > servers -> Astaro NAT firewall -> pfSense Transparent Firewall -> > Customer Colo cabinet -> Datacenter routing -> Internet) > > 2) pfSense running on a T1 that is connected to another cabinet in the > same co-location (Customer servers -> pfSense -> T1 -> Customer Colo > cabinet -> Datacenter routing -> Internet) > > The problem: Start a download using any protocol (tcp/udp), any > application (http, https, ssh, etc) any size, from any location and > the download will stall at a random point. Sometimes the number will > be 8MB, sometimes it is 20MB. But if I restart the download > immediately, the stall will happen at EXACTLY the same point...so if > it stalled once at 8MB, it will stall immediately at 8MB again. > > This is happening at both of these locations. > > I've found that changing the "Firewall Optimization Options" to > conservative does help some, I was able to download a file up to 300MB > and it was OK...although it still does hang quite regularly, so the > problem still exists. > > Does anyone have any ideas for me? I am banging my head against the > wall at this point. Help!!! > > Thank you in advance. > > Regards, > Angus > > - > To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com > For additional commands, e-mail: discussion-h...@pfsense.com > > Commercial support available - https://portal.pfsense.org > > > - > To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com > For additional commands, e-mail: discussion-h...@pfsense.com > > Commercial support available - https://portal.pfsense.org > > - To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com For additional commands, e-mail: discussion-h...@pfsense.com Commercial support available - https://portal.pfsense.org
RE: [pfSense-discussion] Very odd issue - Transparent Firewall - 2 Locations
Possibly an issue with TCP window scaling or PMTU-D. Are the logs generating any drops for the flow ? -Original Message- From: Angus Jordan [mailto:angus.jor...@gmail.com] Sent: 15 July 2009 22:08 To: discussion@pfsense.com Subject: [pfSense-discussion] Very odd issue - Transparent Firewall - 2 Locations Hi there, We have deployed 2 pfSense Transparent firewalls at 2 separate locations. The commonality between both locations is their Internet provider. 1) pfSense running directly in the providers co-location (Customer servers -> Astaro NAT firewall -> pfSense Transparent Firewall -> Customer Colo cabinet -> Datacenter routing -> Internet) 2) pfSense running on a T1 that is connected to another cabinet in the same co-location (Customer servers -> pfSense -> T1 -> Customer Colo cabinet -> Datacenter routing -> Internet) The problem: Start a download using any protocol (tcp/udp), any application (http, https, ssh, etc) any size, from any location and the download will stall at a random point. Sometimes the number will be 8MB, sometimes it is 20MB. But if I restart the download immediately, the stall will happen at EXACTLY the same point...so if it stalled once at 8MB, it will stall immediately at 8MB again. This is happening at both of these locations. I've found that changing the "Firewall Optimization Options" to conservative does help some, I was able to download a file up to 300MB and it was OK...although it still does hang quite regularly, so the problem still exists. Does anyone have any ideas for me? I am banging my head against the wall at this point. Help!!! Thank you in advance. Regards, Angus - To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com For additional commands, e-mail: discussion-h...@pfsense.com Commercial support available - https://portal.pfsense.org - To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com For additional commands, e-mail: discussion-h...@pfsense.com Commercial support available - https://portal.pfsense.org
[pfSense-discussion] Very odd issue - Transparent Firewall - 2 Locations
Hi there, We have deployed 2 pfSense Transparent firewalls at 2 separate locations. The commonality between both locations is their Internet provider. 1) pfSense running directly in the providers co-location (Customer servers -> Astaro NAT firewall -> pfSense Transparent Firewall -> Customer Colo cabinet -> Datacenter routing -> Internet) 2) pfSense running on a T1 that is connected to another cabinet in the same co-location (Customer servers -> pfSense -> T1 -> Customer Colo cabinet -> Datacenter routing -> Internet) The problem: Start a download using any protocol (tcp/udp), any application (http, https, ssh, etc) any size, from any location and the download will stall at a random point. Sometimes the number will be 8MB, sometimes it is 20MB. But if I restart the download immediately, the stall will happen at EXACTLY the same point...so if it stalled once at 8MB, it will stall immediately at 8MB again. This is happening at both of these locations. I've found that changing the "Firewall Optimization Options" to conservative does help some, I was able to download a file up to 300MB and it was OK...although it still does hang quite regularly, so the problem still exists. Does anyone have any ideas for me? I am banging my head against the wall at this point. Help!!! Thank you in advance. Regards, Angus - To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com For additional commands, e-mail: discussion-h...@pfsense.com Commercial support available - https://portal.pfsense.org
