Hi Markus, I agree that it is the job of the plugin to determine the state of the service, but only when it can return a definitive result. The use case here is very unique to the MTA.
My plugin doesn't return 255 (its set to return -1), Icinga automatically sets it to 255 when it terminates the plugin after a timeout period (for a reason that I haven't determined yet due to lack of log data). The problem is, when the plugin can't determine the current state, it tries querying Icinga's status file (through MKLiveStatus for the Last State), and if the monitoring server is under high load, this query times out also. If this fails, Icinga kills the plugin, the state of the service is set to whatever the timeout state is in the configuration file (I set it to Unknown). The core ONLY lets you set the state to one of the 4 options (OK, Warning, Critical, or Unknown), there is no way to not change the previous state. In the MTA's use case, If a bus was previously marked as up or down, its state will become Unknown (as configured now), and send a round of notification emails due to the state change (and for 6,000 buses, this equates to 12,000 emails each for up and down since RDS tends to have problems for hours at a time). Furthermore, any previous acknowledgments are erased and we lost all of our tracking. As for your suggestion about the $LASTSERVICESTATE$ macro, I'll attempt that route, but I'm still concerned that CURL might cause this 255 error to occur because the script timed out before it could return a status to Icinga. Thanks, Phil Philip Matuskiewicz Systems Developer - MTA Bus Time (http://bustime.mta.info) BS/MS Computer Science (5/11, University @ Buffalo) Office: 2 Broadway, D27.90 Office Phone: 646-252-8509 Personal Contact info: http://www.famousphil.com/contact ________________________________________ From: Markus Frosch [mar...@lazyfrosch.de] Sent: Thursday, July 05, 2012 10:48 AM To: icinga-users@lists.sourceforge.net Subject: Re: [icinga-users] Icinga Feature Suggestion Hello Philip, > I'd like to propose that we add a new possible return code to Icinga of -1 > (or some other code), that means, leave the previous status in place and do > nothing (a no-op essentially), and simply reschedule the service as if the > check happened normally. The MTA is willing to invest in my time to develop > this functionality, but we do not want to maintain an internal version of > Icinga in parallel to the main branch. It is usually the job of the plugin to determine the state of the service. If you want to use the last/current state if it is "unknown" you should use the $LASTSERVICESTATE$ or $...ID$ macro as an attribute to your service. So the plugin can return this state instead of unknown. http://docs.icinga.org/1.7/en/macrolist.html#macrolist-lastservicestate I don't think this needs to be a feature in the core... The intelligence here has to come from the plugin. Also I recommend to use 3 as exit code when the state is "unknown" and not 255. Regards Markus -- Markus Frosch mar...@lazyfrosch.de http://www.lazyfrosch.de ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ icinga-users mailing list icinga-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/icinga-users ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ icinga-users mailing list icinga-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/icinga-users
