Re: [CentOS] Server packet loss and becomes unresponsive
Does ifconfig or dmesg show any errors? If the answer to those is no, I'd start running sar to see if there is abnormal behavior which could indicate a hardware problem. Of course, another possibility is a DDOS. Do you have any kind security monitoring or protection in place? -geoff - Geoff Galitz Blankenheim, Germany http://www.galitz.org On Tue 03/11/09 04:38 , Timothy timmy...@progressivemarketingnetwork.com sent: I am at a lost with this server of mine. I have replaced everything except just replacing the entire server. The server is running at a low load but every so often it starts to have high packet loss / latency(the average ms for me is 80 but it jumps to 4000 during this period) and eventually becomes unresponsive. All traffic seems normal, no unusual activity. But once or twice a month it starts to get packet loss and after an hour or two it crashes. The network itself is fine. All other servers on the same subnet work fine. Any ideas what could be causing this kind of behavior? Thanks, Tim ___ CentOS mailing list CentOS@centos.org ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Server packet loss and becomes unresponsive
do you have remote console access i.e. DRAC or ILO ? No. did you mention if any recent OS or other software updates like 5.3 to 5.4 or otherwise? It is a newer box w/ 5.4 installed. someone could be DOS'ing the server ??? Does not appear that way and if it was I would imagine console should still work but it does not. do you run a firewall and drop bad traffic to the floor or is it wide open? I do run a firewall and drop traffic. what about thermal issues? Temp does not spike during these periods. recent BIOS update? No also, you might consider using some boot time kernel parameters to disable things hardware wise etc that are not absolutely necessary. I have disabled acpi in kernel. Anything else I should disable? does it match the switchport? i.e. hard set? auto negot? cisco should be hard set. Yes it matches but direct console crashes so I do not feel it is a network issue. If it was I would imagine you should still be able to direct console in. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Server packet loss and becomes unresponsive
Timothy wrote: snip did you mention if any recent OS or other software updates like 5.3 to 5.4 or otherwise? It is a newer box w/ 5.4 installed. someone could be DOS'ing the server ??? Does not appear that way and if it was I would imagine console should still work but it does not. do you run a firewall and drop bad traffic to the floor or is it wide open? I do run a firewall and drop traffic. what about thermal issues? Temp does not spike during these periods. recent BIOS update? No also, you might consider using some boot time kernel parameters to disable things hardware wise etc that are not absolutely necessary. I have disabled acpi in kernel. Anything else I should disable? does it match the switchport? i.e. hard set? auto negot? cisco should be hard set. Yes it matches but direct console crashes so I do not feel it is a network issue. If it was I would imagine you should still be able to direct console in. Do you have smartmontools installed and running? I just was reading about that, and it can *cause* lockups and crashes. Check the WARNING file in the distro. mark -- Right, breathing is addictive, as shown by the withdrawal symptoms. - Steve Gruenwald ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Server packet loss and becomes unresponsive
Timothy wrote: Any ideas what could be causing this kind of behavior? What kind of NIC? 80ms is normal if your going a few thousand miles, how far apart is the system your pinging from? If it's a NIC like a Realtek or something don't think twice just get a good server grade NIC. Broadcom or Intel are common ones. nate ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Server packet loss and becomes unresponsive
From: nate What kind of NIC? 80ms is normal if your going a few thousand miles, how far apart is the system your pinging from? If it's a NIC like a Realtek or something don't think twice just get a good server grade NIC. Broadcom or Intel are common ones. --- It was just an onboard nic(which is a Broadcom) which I have used this mb/onboard nic in several other servers and have never had any issues but nonetheless I changed it to an Intel and that has made no difference. I changed memory twice and the hard drives on this one too. The data center tried to console in and have not been able to when it has been non responsive. The weird latency just seems to be an early symptom. The data center is on the East Coast and I live in Arizona so 80ms is not a bad ping. I normally manage my servers but I even hired a system admin that has worked with a company I work for on occasion and he claims everything software wise is fine. The data center has tested all the hardware and everything seems fine with that as well. This whole thing has really stumped me. Only thing I have not replaced is MB/Processor. If I catch it with the high latency I have tried killing all processes which has not helped at all(load was low to begin with however). If I reboot the server it comes up fine and the latency/packet loss goes away. Thanks, Tim ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Server packet loss and becomes unresponsive
do you have remote console access i.e. DRAC or ILO ? did you mention if any recent OS or other software updates like 5.3 to 5.4 or otherwise? someone could be DOS'ing the server ??? do you run a firewall and drop bad traffic to the floor or is it wide open? you havent mentioned what the various logs on the server tell you. what about thermal issues? recent BIOS update? things in BIOS that should be turned off or changed ? also, you might consider using some boot time kernel parameters to disable things hardware wise etc that are not absolutely necessary. do you hard set NIC duplex in software? i.e. ETHTOOL_OPTS=speed 100 duplex full autoneg off does it match the switchport? i.e. hard set? auto negot? cisco should be hard set. is the switchport bad? bad cable or cable ends? bad fiber or fiber connectors? mac address or arp issues? the list goes on. - rh ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
