Hi there,
Since some days ago I am trying to configure notifications properly on our
demands. Without success. I write this first for help but as an feedback also.
Not about for bullshit storming and another kind of stupid flamewars. So I am
asking me now are these maybe bugs, missing features, an misconfiguration?
Should I give up? It’s Icinga2 stable and good enough for my company?
The first task is thats on 97% of our Infrastructure it’s necessary that people
gets notified only if an service is going in an critical state and gets
notified again even the event has been acknowledged or has gone back to the
state OK. That sounds like an simple task but it is not.
We are running FreeBSD 10 and Icinga 2.1.1 on our nodes. Our servers are almost
running with FreeBSD.
Some I hope helpful informations.
my template,
template Notification "mail-service-notification" {
command = "mail-service-notification"
states = [ OK, Warning, Critical, Unknown, ]
types = [ Problem, Acknowledgement, Recovery, Custom,
FlappingStart, FlappingEnd,
DowntimeStart, DowntimeEnd, DowntimeRemoved ]
period = "24x7"
}
my commands,
object NotificationCommand "mail-host-notification" {
import "plugin-notification-command"
command = [ SysconfDir + "/icinga2/scripts/mail-host-notification.sh" ]
env = {
NOTIFICATIONTYPE = "$notification.type$"
HOSTALIAS = "$host.display_name$"
HOSTADDRESS = "$address$"
HOSTSTATE = "$host.state$"
LONGDATETIME = "$icinga.long_date_time$"
HOSTOUTPUT = "$host.output$"
NOTIFICATIONAUTHORNAME = "$notification.author$"
NOTIFICATIONCOMMENT = "$notification.comment$"
HOSTDISPLAYNAME = "$host.display_name$"
USEREMAIL = "$user.email$"
}
}
object NotificationCommand "sms-host-notification" {
import "plugin-notification-command"
command = [ SysconfDir + "/icinga2/scripts/sms-host-notification.sh" ]
env = {
NOTIFICATIONTYPE = "$notification.type$"
HOSTALIAS = "$host.display_name$"
HOSTADDRESS = "$address$"
HOSTSTATE = "$host.state$"
LONGDATETIME = "$icinga.long_date_time$"
SHORTDATETIME = "$icinga.short_date_time$"
HOSTOUTPUT = "$host.output$"
NOTIFICATIONAUTHORNAME = "$notification.author$"
NOTIFICATIONCOMMENT = "$notification.comment$"
HOSTDISPLAYNAME = "$host.display_name$"
PAGER = "$user.pager$"
}
}
object NotificationCommand "mail-service-notification" {
import "plugin-notification-command"
command = [ SysconfDir + "/icinga2/scripts/mail-service-notification.sh" ]
env = {
NOTIFICATIONTYPE = "$notification.type$"
SERVICEDESC = "$service.name$"
HOSTALIAS = "$host.display_name$"
HOSTADDRESS = "$address$"
SERVICESTATE = "$service.state$"
LONGDATETIME = "$icinga.long_date_time$"
SERVICEOUTPUT = "$service.output$"
NOTIFICATIONAUTHORNAME = "$notification.author$"
NOTIFICATIONCOMMENT = "$notification.comment$"
HOSTDISPLAYNAME = "$host.display_name$"
SERVICEDISPLAYNAME = "$service.display_name$"
USEREMAIL = "$user.email$"
}
}
object NotificationCommand "sms-service-notification" {
import "plugin-notification-command"
command = [ SysconfDir + "/icinga2/scripts/sms-service-notification.sh" ]
env = {
NOTIFICATIONTYPE = "$notification.type$"
SERVICEDESC = "$service.name$"
HOSTALIAS = "$host.display_name$"
HOSTADDRESS = "$address$"
SERVICESTATE = "$service.state$"
LONGDATETIME = "$icinga.long_date_time$"
SHORTDATETIME = "$icinga.short_date_time$"
SERVICEOUTPUT = "$service.output$"
NOTIFICATIONAUTHORNAME = "$notification.author$"
NOTIFICATIONCOMMENT = "$notification.comment$"
HOSTDISPLAYNAME = "$host.display_name$"
SERVICEDISPLAYNAME = "$service.display_name$"
PAGER = "$user.pager$"
}
}
my inherited notifications
apply Notification "mail-service-standard-first-notification" to Service {
import "mail-service-notification"
user_groups = [ "trivago-admins-email" ]
assign where service.vars.sla == "24x7"
states = [ Critical, OK ]
types = [ Problem, Acknowledgement, Recovery, Custom,
DowntimeStart, DowntimeEnd, DowntimeRemoved ]
period = "24x7"
interval = 0
times.begin = 3m
}
apply Notification "mail-service-standard-re-notification" to Service {
import "mail-service-notification"
user_groups = [ "trivago-admins-email" ]
assign where service.vars.sla == "24x7"
states = [ Critical, OK ]
types = [ Problem, Acknowledgement, Recovery, Custom,
DowntimeStart, DowntimeEnd, DowntimeRemoved ]
period = "24x7"
interval = 120m
times.begin = 3m
}
So like described above notifications should been sent in stadiums critical and
recovery. But they has been send also notifications about recovery if the state
has been warning.
apply Notification "mail-service-standard-first-notification" to Service {
import "mail-service-notification"
user_groups = [ "trivago-admins-email" ]
assign where service.vars.sla == "24x7"
states = [ Critical ]
types = [ Problem, Acknowledgement, Recovery, Custom,
DowntimeStart, DowntimeEnd, DowntimeRemoved ]
period = "24x7"
interval = 0
times.begin = 3m
}
apply Notification "mail-service-standard-re-notification" to Service {
import "mail-service-notification"
user_groups = [ "trivago-admins-email" ]
assign where service.vars.sla == "24x7"
states = [ Critical ]
types = [ Problem, Acknowledgement, Recovery, Custom,
DowntimeStart, DowntimeEnd, DowntimeRemoved ]
period = "24x7"
interval = 120m
times.begin = 3m
}
described above then notifications was send with the state critical but no one
notification with the state recovery
in conclusion this means that people get services messages with state OK even
if they have previously seen no messages getting ahead with the status warning.
And that’s confusing people and reduces the acceptance about icinga2.
How I could reach it, that notifications for messages should be sent in
stadiums critical and gets an recovery message if the issue doesn't exist
anymore. That’s one of the things that I highly want
The second issue is that notifications doesn’t have been send in the right time.
apply Notification "mail-service-standard-re-notification" to Service {
import "mail-service-notification"
user_groups = [ "trivago-admins-email" ]
assign where service.vars.sla == "24x7"
states = [ Critical ]
types = [ Problem, Acknowledgement, Recovery, Custom,
DowntimeStart, DowntimeEnd, DowntimeRemoved ]
period = "24x7"
interval = 120m
times.begin = 3m
}
The first notification has come later then after 30 minutes and the
re-notification has been send also very late in my test-environment. And
sometimes nothing has been happen.
[2014-10-30 19:50:18 +0000] notice/Notification: Not sending notifications for
notification object 'dev3!Raid Status!mail-service-standard': before escalation
range
[2014-10-30 19:50:18 +0000] notice/Notification: Not sending notifications for
notification object 'dev3!Raid Status!sms-standard': before escalation range
[2014-10-30 19:55:15 +0000] information/NotificationComponent: Sending reminder
notification for object 'dev3!Raid Status'
[2014-10-30 19:55:15 +0000] notice/Notification: Not sending notifications for
notification object 'dev3!Raid Status!sms-standard': before escalation range
So my cruel crappy hack is
apply Notification "mail-service-standard-first-notification" to Service {
import "mail-service-notification"
user_groups = [ "trivago-admins-email" ]
assign where service.vars.sla == "24x7"
states = [ Critical ]
types = [ Problem, Acknowledgement, Recovery, Custom,
DowntimeStart, DowntimeEnd, DowntimeRemoved ]
period = "24x7"
interval = 0
times.begin = 3m
}
apply Notification "mail-service-standard-re-notification" to Service {
import "mail-service-notification"
user_groups = [ "trivago-admins-email" ]
assign where service.vars.sla == "24x7"
states = [ Critical ]
types = [ Problem, Acknowledgement, Recovery, Custom,
DowntimeStart, DowntimeEnd, DowntimeRemoved ]
period = "24x7"
interval = 10m
times.begin = 3m
}
that forces to send an notification directly and every 10 minutes after. Is
this the really way to configure notifications at the right time?
thank you all for any help, hints or suggestions.
cheers
Darko Hojnik
Datacenter Operations
[email protected]
www.trivago.com
Court of Registration: Amtsgericht Duesseldorf, Registration Number: HRB 51842
Managing directors: Rolf Schroemgens • Malte Siewert • Peter Vinnemeier
trivago GmbH • Bennigsen-Platz 1 • 40474 Duesseldorf, Germany
* This email message may contain legally privileged and/or confidential
information.
You are hereby notified that any disclosure, copying, distribution, or use of
this email message is strictly prohibited.
_______________________________________________
icinga-users mailing list
[email protected]
https://lists.icinga.org/mailman/listinfo/icinga-users
