Re: [pfSense-discussion] transient network drops
On Wed, Aug 29, 2007 at 04:37:58PM +0200, Eugen Leitl wrote: > > I've verified it's the firewall. I could ping the host from within I've got to take that back. pfsense is innocent; there's some ARP mayhem with the machine hosting the vserver guests. I'm just the one who wound up with an unlucky vserver guest. > the firewall even while it was not accessible outside. I was able > to temporarily restore connectivity by disabling the port leading > to the WAN NIC, and reenabling it again. Connectivity was regained, > but lost shortly thereafter. After a while, the connectivity was > regained spontaneously. -- Eugen* Leitl http://leitl.org";>leitl http://leitl.org __ ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE
Re: [pfSense-discussion] transient network drops
On Wed, Aug 29, 2007 at 03:50:05PM +0200, Eugen Leitl wrote: > (it's a transparent bridge, with no rules defined but the default > pass-through-everything rule). > > The hoster advised doing an mtr, which I will do once the host > drops offline again. I've verified it's the firewall. I could ping the host from within the firewall even while it was not accessible outside. I was able to temporarily restore connectivity by disabling the port leading to the WAN NIC, and reenabling it again. Connectivity was regained, but lost shortly thereafter. After a while, the connectivity was regained spontaneously. Here's the setup from the firewall: # ifconfig -a vr0: flags=8943 mtu 1500 inet6 fe80::240:63ff:fed9:a718%vr0 prefixlen 64 scopeid 0x1 inet 10.0.0.6 netmask 0xff00 broadcast 10.0.0.255 ether 00:40:63:d9:a7:18 media: Ethernet autoselect (100baseTX ) status: active vr1: flags=8943 mtu 1500 inet6 fe80::240:63ff:fed9:a736%vr1 prefixlen 64 scopeid 0x2 inet 85.10.225.6 netmask 0xff00 broadcast 85.10.225.255 ether 00:40:63:d9:a7:36 media: Ethernet autoselect (100baseTX ) status: active pfsync0: flags=41 mtu 2020 pfsync: syncdev: lo0 syncpeer: 224.0.0.240 maxupd: 128 lo0: flags=8049 mtu 16384 inet 127.0.0.1 netmask 0xff00 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 pflog0: flags=100 mtu 33208 enc0: flags=41 mtu 1536 bridge0: flags=8843 mtu 1500 ether 06:b1:b4:39:d2:42 priority 32768 hellotime 2 fwddelay 15 maxage 20 member: vr1 flags=7 port 2 priority 128 path cost 55 forwarding member: vr0 flags=7 port 1 priority 128 path cost 55 forwarding Ah, at least something in dmesg: arp: 85.10.225.1 is on vr1 but got reply from 00:02:85:1a:e2:e0 on vr0 arp: 85.10.225.1 is on vr1 but got reply from 00:02:85:1a:e2:e0 on vr0 arp: 85.10.225.1 is on vr1 but got reply from 00:02:85:1a:e2:e0 on vr0 arp: 85.10.225.1 is on vr1 but got reply from 00:02:85:1a:e2:e0 on vr0 arp: 85.10.225.1 is on vr1 but got reply from 00:02:85:1a:e2:e0 on vr0 arp: 85.10.225.1 is on vr1 but got reply from 00:02:85:1a:e2:e0 on vr0 bridge0: Ethernet address: 06:b1:b4:39:d2:42 vr0: promiscuous mode enabled vr1: promiscuous mode enabled arp: 00:40:63:d9:a7:36 attempts to modify permanent entry for 10.0.0.6 on vr0 pflog0: promiscuous mode disabled pflog0: promiscuous mode enabled tcp_output: inc sockbuf, old 66240, new 74432, sb_cc 60444, snd_wnd 54656, sendwnd 53340 vr1: link state changed to DOWN vr1: link state changed to UP arp: 85.10.225.5 moved from 00:40:63:d9:a7:c1 to 00:40:63:d9:a7:e9 on vr1 arp: 85.10.225.5 moved from 00:40:63:d9:a7:e9 to 00:40:63:d9:a7:c1 on vr1 pflog0: promiscuous mode disabled pflog0: promiscuous mode enabled pflog0: promiscuous mode disabled pflog0: promiscuous mode enabled vr1: link state changed to DOWN vr1: link state changed to UP vr1: link state changed to DOWN vr1: link state changed to UP vr1: link state changed to DOWN vr1: link state changed to UP vr1: link state changed to DOWN vr1: link state changed to UP vr1: link state changed to DOWN vr1: link state changed to UP -- Eugen* Leitl http://leitl.org";>leitl http://leitl.org __ ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE
Re: [pfSense-discussion] transient network drops
Eugen Leitl wrote: > On Wed, Aug 29, 2007 at 02:33:45PM +0100, Paul M wrote: >> Eugen Leitl wrote: >> >>> wan-pfsense-lan >>> | >>> switch1--diverse hosts >>> >>> what's interesting is that I have transient outages to *some* IPs >>> (it could be just one IP, actually). I can still ping that IP locally >> is your switch manageable? can you turn logging on it? can you look up > > The gateway switch is a Netgear GS724T, the second switch is HP ProCurve. > Logging, as in redirecting traffic to a sniffer port, and capture > all traffic there? logging, as in getting the managed switch to send syslog messages to a server and seeing if it reports any errors. I don't know procurves, but in cisco it's fairly straight forward... add this to config for example: logging facility local5 logging 10.0.0.2 > The hoster advised doing an mtr, which I will do once the host > drops offline again. yeah, also check arp table on the disappearing host
Re: [pfSense-discussion] transient network drops
On Wed, Aug 29, 2007 at 02:33:45PM +0100, Paul M wrote: > Eugen Leitl wrote: > > > wan-pfsense-lan > > | > > switch1--diverse hosts > > > > what's interesting is that I have transient outages to *some* IPs > > (it could be just one IP, actually). I can still ping that IP locally > > is your switch manageable? can you turn logging on it? can you look up The gateway switch is a Netgear GS724T, the second switch is HP ProCurve. Logging, as in redirecting traffic to a sniffer port, and capture all traffic there? > the mac of the missing host (also check arp table on pfsense)? The missing host is a vserver guest, sharing its MAC with the host (00:E0:81:58:58:B3). The pfsense ARP table only shows Diagnostics: ARP Table IP address MAC address HostnameInterface 10.0.0.600:40:63:d9:a7:18 carbon.ativel.com LAN 85.10.225.1 00:02:85:1a:e2:e0 hydrogen.ativel.com WAN (it's a transparent bridge, with no rules defined but the default pass-through-everything rule). The hoster advised doing an mtr, which I will do once the host drops offline again. -- Eugen* Leitl http://leitl.org";>leitl http://leitl.org __ ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE
Re: [pfSense-discussion] transient network drops
Eugen Leitl wrote: > wan-pfsense-lan > | > switch1--diverse hosts > > what's interesting is that I have transient outages to *some* IPs > (it could be just one IP, actually). I can still ping that IP locally is your switch manageable? can you turn logging on it? can you look up the mac of the missing host (also check arp table on pfsense)?
[pfSense-discussion] transient network drops
I'm running a transparent firewall on 1.2-RC1 with several hosts behind them, both real and virtual. The network setup looks like this: ---gateway---switch0 | | wan-pfsense-lan | | switch1--diverse hosts what's interesting is that I have transient outages to *some* IPs (it could be just one IP, actually). I can still ping that IP locally (within above assembly), but not from the outside. I can reliably fix that by disabling the port on switch0 which connects to wan interface of pfsense, and reenabling it a few seconds later. After few 10 seconds the IP becomes again pingable. For a (long) while, until the next outage. I don't think the problem is in my setup, but there might be something in my setup which triggers the problem at my ISP. Could it be a MAC table running full? Something else? How can I debug this? (Apart from opening a trouble ticket, I mean; I'd rather narrow down the spots of trouble first). -- Eugen* Leitl http://leitl.org";>leitl http://leitl.org __ ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE
