[root@icinga ~]# file /usr/bin/icinga /usr/bin/icinga: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.18, stripped
Installed Packages Name : icinga Arch : x86_64 Version : 1.7.1 Release : 1ncs.el6 Size : 970 k Repo : installed >From repo : ncspkgs-pending Summary : Open Source host, service and network monitoring program URL : http://www.icinga.org/ License : GPLv2 Red Hat Enterprise Linux Server release 6.3 (Santiago) Linux icinga 2.6.32-220.4.2.el6.x86_64 #1 SMP Mon Feb 6 16:39:28 EST 2012 x86_64 x86_64 x86_64 GNU/Linux On Fri, Nov 2, 2012 at 9:54 AM, Michael Friedrich <michael.friedr...@gmail.com> wrote: > Zachary McGibbon wrote: >> Carl, >> >> I'm not running IDOUtils on my system and I actually HUP my Icinga >> every 4 hours to refresh the configs from our CMDB database. My >> server is quite powerful, it's an IBM x3650 M3, 16 cores, 24GB RAM. I >> run my object_cache_file, status_file, tmp_path, and check_result_path >> in a ramdisk to speed things up. >> >> Example HUP: >> >> [11-02-2012 09:27:50] Event loop started... >> [11-02-2012 09:27:40] LOG VERSION: 2.0 >> [11-02-2012 09:27:40] Local time is Fri Nov 02 09:27:40 EDT 2012 >> [11-02-2012 09:27:40] Icinga 1.7.1 starting... (PID=15502) >> [11-02-2012 09:27:39] Caught SIGHUP, restarting... >> >> ... and test scheduling: >> >> [root@etna icinga]# icinga -s /etc/icinga/icinga.cfg >> >> Icinga 1.7.1 >> Copyright (c) 2009-2012 Icinga Development Team (http://www.icinga.org) >> Copyright (c) 2009-2012 Nagios Core Development Team and Community >> Contributors >> Copyright (c) 1999-2009 Ethan Galstad >> Last Modified: 06-18-2012 >> License: GPL > > which version is that exactly? (source/package, os?) >> >> Timing information on object configuration processing is listed >> below. You can use this information to see if precaching your >> object configuration would be useful. >> >> Object Config Source: Config files (uncached) >> >> OBJECT CONFIG PROCESSING TIMES (* = Potential for precache >> savings with -u option) >> ---------------------------------- >> Read: 0.148778 sec >> Resolve: 0.005158 sec * >> Recomb Contactgroups: 0.001007 sec * >> Recomb Hostgroups: 0.183998 sec * >> Dup Services: 0.138670 sec * >> Recomb Servicegroups: 4.053202 sec * >> Duplicate: 0.042983 sec * >> Inherit: 0.013876 sec * >> Recomb Contacts: 0.000000 sec * >> Sort: 0.000000 sec * >> Register: 0.234155 sec >> Free: 0.018379 sec >> ============ >> TOTAL: 4.840206 sec * = 4.438894 sec (91.71%) estimated >> savings >> >> >> RETENTION DATA TIMES >> ---------------------------------- >> Read and Process: 0.573883 sec >> ============ >> TOTAL: 0.573883 sec >> >> >> Timing information on configuration verification is listed below. >> >> CONFIG VERIFICATION TIMES (* = Potential for speedup with -x option) >> ---------------------------------- >> Object Relationships: 0.997263 sec >> Circular Paths: 0.000942 sec * >> Misc: 0.008356 sec >> ============ >> TOTAL: 1.006561 sec * = 0.000942 sec (0.1%) estimated savings >> >> >> EVENT SCHEDULING TIMES >> ------------------------------------- >> Get service info: 0.023748 sec >> Get host info info: 0.002381 sec >> Get service params: 0.000005 sec >> Schedule service times: 0.070183 sec >> Schedule service events: 0.607110 sec >> Get host params: 0.000001 sec >> Schedule host times: 0.027887 sec >> Schedule host events: 1.531434 sec >> ============ >> TOTAL: 2.262749 sec >> >> >> Projected scheduling information for host and service checks >> is listed below. This information assumes that you are going >> to start running Icinga with your current config files. >> >> HOST SCHEDULING INFORMATION >> --------------------------- >> Total hosts: 7250 >> Total scheduled hosts: 7231 >> Host inter-check delay method: SMART >> Average host check interval: 356.67 sec >> Host inter-check delay: 0.05 sec >> Max host check spread: 30 min >> First scheduled check: Fri Nov 2 09:31:53 2012 >> Last scheduled check: Fri Nov 2 09:37:30 2012 >> >> >> SERVICE SCHEDULING INFORMATION >> ------------------------------- >> Total services: 15885 >> Total scheduled services: 15459 >> Service inter-check delay method: SMART >> Average service check interval: 3090.55 sec >> Inter-check delay: 0.12 sec >> Interleave factor method: USER-SUPPLIED VALUE >> Service interleave factor: 4 >> Max service check spread: 30 min >> First scheduled check: Fri Nov 2 09:39:23 2012 >> Last scheduled check: Fri Nov 2 10:07:59 2012 >> >> >> CHECK PROCESSING INFORMATION >> ---------------------------- >> Check result reaper interval: 2 sec >> Max concurrent service checks: Unlimited >> >> >> PERFORMANCE SUGGESTIONS >> ----------------------- >> I have no suggestions - things look okay. >> >> >> On Thu, Nov 1, 2012 at 4:38 PM, Carl R. Friend <crfri...@rcn.com> wrote: >>> On Thu, 1 Nov 2012, Zachary McGibbon wrote: >>> >>>> Has anyone seen something like this? The host went down, then came back up >>>> but the service check took 43 minutes to run. >>> Without more information, specifically other things that your >>> monitoring host was doing at the time other than restarting Icinga >>> and whether you're running idoutils, we can't really say much. >>> >>> If you're running idoutils, the main scheduling loop won't start >>> until the database entries have been recereated, and that can take >>> significant amounts of time especially if there is contention for >>> the database. This can be exacerbated if you have more than one >>> instance of Icinga pointed at the same database engine. >>> >>> As a guess, since you restart your process every day, today's >>> behaviour was anomalous, so something must have perturbed it. Do >>> you have sar configured to snapshot performance every so often? If >>> so, you can see what the machine was doing between 12:14 and 12:53 >>> in terms of CPU, memory, and I/O usage -- what that will not tell >>> you, however, is whether something was perrturbing another host -- a >>> database host, for instance. >>> >>> Here's a startup sequence on my Icinga (with idoutils) system at >>> home: >>> >>> [2012-10-28 10:19:30] Icinga 1.8.0 starting... (PID=9809) >>> [2012-10-28 10:19:30] Local time is Sun Oct 28 10:19:30 EDT 2012 >>> [2012-10-28 10:19:30] LOG VERSION: 2.0 >>> [2012-10-28 10:19:30] idomod: IDOMOD 1.8.0 (10-18-2012) Copyright(c) >>> 2005-2008 Ethan Galstad, Copyright(c) 2009-2012 Icinga Development Team >>> (https://www.icinga.org) >>> [2012-10-28 10:19:30] idomod: Successfully connected to data sink. 0 >>> queued items to flush. >>> [2012-10-28 10:19:30] Event broker module 'IDOMOD' version '1.8.0' from >>> '/usr/local/icinga/lib/idomod.so' initialized successfully. >>> [2012-10-28 10:19:30] Finished daemonizing... (New PID=9811) >>> [2012-10-28 10:19:37] Event loop started.. >>> >>> Note that my very limited system takes seven full seconds to get >>> from instantiation to the main event loop -- and that's down to the >>> idoutils repopulating the database. This delay can be extensive in >>> large installations (like yours) and, in particular, if there is >>> contention on the database in the form of table locks. NO CHECKS >>> will be run until the event loop starts, so we particularly need >>> that timestamp. >>> >>> Cheers! >>> >>> +------------------------------------------------+---------------------+ >>> | Carl Richard Friend (UNIX Sysadmin) | West Boylston | >>> | Minicomputer Collector / Enthusiast | Massachusetts, USA | >>> | mailto:crfri...@rcn.com +---------------------+ >>> | http://users.rcn.com/crfriend/museum | ICBM: 42:22N 71:47W | >>> +------------------------------------------------+---------------------+ >>> >>> ------------------------------------------------------------------------------ >>> Everyone hates slow websites. So do we. >>> Make your web apps faster with AppDynamics >>> Download AppDynamics Lite for free today: >>> http://p.sf.net/sfu/appdyn_sfd2d_oct >>> _______________________________________________ >>> icinga-users mailing list >>> icinga-users@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/icinga-users >> ------------------------------------------------------------------------------ >> LogMeIn Central: Instant, anywhere, Remote PC access and management. >> Stay in control, update software, and manage PCs from one command center >> Diagnose problems and improve visibility into emerging IT issues >> Automate, monitor and manage. Do more in less time with Central >> http://p.sf.net/sfu/logmein12331_d2d >> _______________________________________________ >> icinga-users mailing list >> icinga-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/icinga-users > > > ------------------------------------------------------------------------------ > LogMeIn Central: Instant, anywhere, Remote PC access and management. > Stay in control, update software, and manage PCs from one command center > Diagnose problems and improve visibility into emerging IT issues > Automate, monitor and manage. Do more in less time with Central > http://p.sf.net/sfu/logmein12331_d2d > _______________________________________________ > icinga-users mailing list > icinga-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/icinga-users ------------------------------------------------------------------------------ LogMeIn Central: Instant, anywhere, Remote PC access and management. Stay in control, update software, and manage PCs from one command center Diagnose problems and improve visibility into emerging IT issues Automate, monitor and manage. Do more in less time with Central http://p.sf.net/sfu/logmein12331_d2d _______________________________________________ icinga-users mailing list icinga-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/icinga-users
