Hi, I have a reoccuring problem and maybe I've looked at it too long, but I just can't find any possible cause. Regularly (sometimes every 3 hours, sometimes once a day) the check latencies in Icinga increase significantly. My usual average is about ~1 second or less and the increased goes up to 300 seconds. Host Check latencies seem to increase faster and higher, so if host check latency is as 300, service check will be at around 100.
Setup: RHEL 6.1 Icinga 1.9.0 with idoutils compiled from source mySQL is running on the same machine pnp4nagios is also running on the same maschine nothing else running 48 GB Ram 2.8 Ghz Intel Xeon E5440 146 GB Raid 1 with 2,5" HP 10k sas drives ~1000 Hosts - checked every 5 Minutes ~10000 Checks - around 6000 checked every 5 minutes, the rest spread over the day (some only once a day) I've seen this happen 3 times today, each time it slowly increased over about 10 minutes till. Sometimes it decreases without user interaction as well. I monitored the last increase and looked at CPU Load => always around 2-4 cpu wait => around 5% Memory: 6 GB really used, 40 GB cached no swap used disks usage - all nowhere close to full, both block and inode open files: around 7k for the whole system, 475 for icinga ulimits: core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 386031 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 10240 cpu time (seconds, -t) unlimited max user processes (-u) 2048 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited everything under /usr/local/icinga/var/spool/ is on ramdisks (checkresults,temp_path,debug_file) icinga.cfg is attached. Any suggestions on what could be causing this or what else I could monitor to troubleshoot this are greatly appreciated.
icinga.cfg
Description: Binary data
------------------------------------------------------------------------------ Get your SQL database under version control now! Version control is standard for application code, but databases havent caught up. So what steps can you take to put your SQL databases under version control? Why should you start doing it? Read more to find out. http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________ icinga-users mailing list icinga-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/icinga-users
