On Thu, 29 Dec 2016, mostolog--- via rsyslog wrote:
During tests, as omelasticsearch is not v5 fully-compatible, it started to
write warnings for each indexing operation. As we were making high-traffic
tests, our /var/log/messages filled with "deprecated: blablabla" and we're
now thinking how to handle those situations.
As we "can't" change how applications log (hence, elastic will continue to
write those warnings), and considering we would like to get notified about
errors on certain procedures (eg: error when creating indexes) I was
wondering if theres something like flags/counters:
using "Artificial Ignorance" (per Marcus Ranam)
If you know that a log message is uninteresting, then you want to throw it away,
but count how many times it happened because the number of times that an
uninteresting log happens can be interesting.
So for something like these deprieciated warning, you can either throw them
away entirely, or you can use either global variables or dyn_stats to track how
many times it happens.
1.rsyslog gets message
2.if it's a notifiable error
if it's already "notified", increase notify counter
what do you do with the notify counter? do you want it spit out along with
other stats (in which case dyn_stats is the right answer) or do you want to do
something else?
drop message (don't log it)
else #not "notified"
notify error
log message
is there anything that resets "not notified"? or do you only want one
notification per syslog startup.
3.if it's a recovery
log recovery
notify recovered
4.otherwise, log normally
In the above, notification could be a snmptrap to our monitoring system, and
"if notified" could be a "global" errorCount variable or something similar...
keep in mind there are the global variables $\ that you can use for this sort of
flag, but checking them is relatively expensive, so you should think about what
you are really wanting here.
It may be good enough to not do any tracking of 'already notified' and instead
just do
if <log message type> then increase counter
and then spit the counters out to your monitoring system. If there were no
messages of that type, you have no message to your monitoring system. If there
were messages of that type, you have a notification of how many times it
happened that monitoring period. If the monitoring period is relatively short
(say 1-5 min), this may be sufficient for your system.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
