Re: tool to determine server stability issues
Hello, On Wed, Mar 4, 2009 at 13:41, Robert Huff wrote: > On my system: > > huff@> whereis httpd > httpd: /usr/local/sbin/httpd /usr/local/man/man8/httpd.8.gz > > Someone's looking in the wrong place. (Unless you've twiddled > /all/ the settings.) Thank you Robert and some information for the rest. It turns out these two prcoesses looking for /usr/sbin/httpd were zombies so to say (and they were the cause of my problems). Someone used a php script vulnarability and placed a script in /tmp. Apart from looking for security holes in php scripts, I am going to monitor /tmp. I am embarrased to say I haven't done that so far. I am writing it to warn people like myself. All the best, -- Zbigniew Szalbot www.slowo.pl www.fairtrade.net.pl ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: tool to determine server stability issues
Zbigniew Szalbot writes: > The process dies as soon as it ends. But this is strange: > > $ l /usr/sbin/httpd > ls: /usr/sbin/httpd: No such file or directory On my system: huff@> whereis httpd httpd: /usr/local/sbin/httpd /usr/local/man/man8/httpd.8.gz Someone's looking in the wrong place. (Unless you've twiddled /all/ the settings.) Robert Huff ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: tool to determine server stability issues
Hi there, > I am not sure if it was upgrade to perl 5.8.9 which started my > problem, but anyway I am spotting a strange server behaviour. It will I am continuing my searches for the problem and just have been able to find out this: USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND www 44888 100.0 0.2 5976 3644 ?? R11:46AM 2:35.18 /usr/sbin/httpd (perl5.8.9) root14 96.3 0.0 0 8 ?? RL Fri08AM 7161:12.89 [idle: cpu0] www 44887 96.3 0.2 5976 3644 ?? R11:46AM 2:34.58 /usr/sbin/httpd (perl5.8.9) The process dies as soon as it ends. But this is strange: $ l /usr/sbin/httpd ls: /usr/sbin/httpd: No such file or directory So it looks like these two processes made the machine unresponsive and generated a lot of outgoing traffic. However, the file does not exist. Any hint as to how to debug it further? I am determined to find the culprit. Thanks! -- Zbigniew Szalbot www.slowo.pl www.fairtrade.net.pl ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
tool to determine server stability issues
Hello, I am not sure if it was upgrade to perl 5.8.9 which started my problem, but anyway I am spotting a strange server behaviour. It will usually last about 5 minutes during which the system becomes unresponsive. Top tells me there are two perl processes run by user www both of which use 100% of a CPU%. The server has four CPUs so that's ok. What is strange, though is that during such a storm the outgoing bandwidth is all taken up and this is the reason server becomes unresponsive. Normally, it does happen that the bandwidth is taken almost completely by remote backup job but I have priority queueing with pf and it has never been a problem. A site will be served fast even though the bandwidth is taken up, because httpd traffic has higher priority. Also, in this particular case, backup job is not involved (especially that the perl processes are run by user www) so it must be something else. I have looked through apache's logs but I cannot seem to find anything strange (normal traffic without any type of DoS activity, etc.). I have turned on debugging in HotSanic which I use for traffic/system measurement but it would not generate outgoing traffic. I guess I am looking for advice how to debug this. I often spot the problem when it is about to end so I do not have enough time to start some a more detailed monitoring (also I am not sure which tool would be best to use). I'd appreciate any advice on how to troubleshoot and find out the source of the problem. Today, I have managed to run netstat during the outage (the ssh session was on so I was able to continue, otherwise I wouldn't get to the server). I can provide its output if it is of any use. I have never had anything like this before so I am in the dark here. I use FreeBSD 7.0-RELEASE-p9 #3. Many thanks in advance! -- Zbigniew Szalbot www.slowo.pl www.fairtrade.net.pl ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
