Hello Jens, while our largest single installation only has 150 hosts and around 1,200 services actively monitored, I think your problem may be a result of using distributed monitoring. However I have no experience in this field, so as an example here are our performance values:
Service Check Execution Time: 0.01 / 12.01 / 0.185 sec Service Check Latency: 0.00 / 1.53 / 0.412 sec Host Check Execution Time: 0.01 / 3.02 / 0.084 sec Host Check Latency: 0.09 / 2.04 / 0.921 sec Active Host / Service Checks: 143 / 1321 As you can see, we're roughly at a fifth of the install size you mention but performance is still quite good. We're using a mysql / myisam backend for ido2db. All the best, Uwe On Mon, May 23, 2011 at 03:00:31PM +0200, Mueller, Jens wrote: > Hi, > > We are using icinga (distributed monitoring setup (central instance with two > satellites) ) to monitor around 500 server and 5900 services at the moment. > > Software version: > Icinga-version 1.3.0 > Ido2db version 1.3.0 > > Used hardware: > > Central server (HA) > 2x Quad-Core Opteron > 32GB RAM > 600GB SAS Raid 1 > > Satellite (doing the checks for the services/hardware) > 2x Quad-Core Opteron > 32GB RAM > SAS drives > > In the beginning we used the event_broker idomod to store config/status data > in a mysql db. After the number of checks were growing, we saw that the > scheduling queue is more and more behind the time, which leads to check > latencies greater then 3k sec. Also the freshness checks for the passiv > chekcs are not working anymore and so the active checks of the icinga server > itself (checks from icinga-cluster) will not be processed. > > We tried to reduce the amount of data, that will be written to the mysql db, > but this didn't change anything. > > I tired the following things to find out the root cause of this problem: > > · Decreased the amount of data that is written to the mysql db > > · Enabled debug logging of icinga/ido2db, but didn't find any > errors/warnings > > · Disabled mysql replication to the standby node > > · Tuned mysql settings > > Overall this didn't really fixed the problem, is the size of data/monitoring > checks maybe to much for the ido2db ? I doubt or hope that :) > > Has/had anybody of you, maybe a similar problem with icinga and ido2db ? > > BR, > > Jens Müller > -- NIONEX ist ein Unternehmen der DirectGroup Germany www.directgroupgermany.de ------------------------------------------------------------------------------ What Every C/C++ and Fortran developer Should Know! Read this article and learn how Intel has extended the reach of its next-generation tools to help Windows* and Linux* C/C++ and Fortran developers boost performance applications - including clusters. http://p.sf.net/sfu/intel-dev2devmay _______________________________________________ icinga-users mailing list icinga-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/icinga-users
