I have a service I wrote that we use, that accepts impstats output from rsyslog, calculates various metrics from them and can forward those metrics to opentsdb, graphite, etc - I could check into open sourcing it if anyone else might have a use for such a thing.
Brian On Tue, Nov 18, 2014 at 1:17 PM, Nathan Brown <[email protected]> wrote: > Same here, collectd tail and a specific file for the stats output > > rsyslog configuration: > > module(load="impstats" interval="10" severity="7" format="json" > log.file="/var/log/rsyslog_stats.log") > > collectd configuration: > > https://gist.github.com/nbrownus/7a8fa65e644d4c371b3b > > We use a specific file to avoid collectd having to tail everything going > through rsyslog > > On Tue, Nov 18, 2014 at 8:17 AM, singh.janmejay <[email protected]> > wrote: > > > Collectd tail plugin has been working well for me with impstats output. > > > > -- > > Regards, > > Janmejay > > > > PS: Please blame the typos in this mail on my phone's uncivilized soft > > keyboard sporting it's not-so-smart-assist technology. > > > > On Nov 18, 2014 9:36 PM, "Boylan, James" <[email protected]> > wrote: > > > > > I also have a python application I made for parsing the impstats file > > > output and submitting them to graphite. > > > > > > I'm going to be working on daemonizing the utility and documenting how > > > best to configure to use it. Please feel free to look at it and open > > issues > > > if you have suggestions of features you'd like to see. > > > > > > The benefit of this setup is that it allows the parsing of the impstats > > > based entirely on the names you have assigned to the various actions, > > > queues and rulesets. > > > > > > https://github.com/Ralnoc/rsyslog-statcollector > > > > > > -- James > > > ________________________________________ > > > From: [email protected] < > > [email protected]> > > > on behalf of Michael Hart <[email protected]> > > > Sent: Tuesday, November 18, 2014 9:28 AM > > > To: rsyslog-users; Damian > > > Subject: Re: [rsyslog] Monitor rsyslog performance > > > > > > I have rsyslog configured with impstats, forwarding to statsd/graphite, > > so > > > I can graph the metrics and monitor them (I have Nagios pulling metrics > > > from graphite). > > > > > > I keep meaning to put a blog post together to document this properly, > but > > > here’s a config snippet that gets you the basics. The hard part is > > > figuring out which queues you want monitored. I’ve only got “main Q” > > > showing here for brevity but I have a lot more defined. > > > > > > <snip> > > > module(load="impstats" interval="10" severity="7" format="cee”) > > > module(load="mmjsonparse”) > > > > > > #json format: {"name":"main > > > Q","size":25,"enqueued":32,"full":0,"discarded.full":0,"discarded.nf > > > ":0,"ma > > > xqsize":25} > > > template(name="mainQTemplate" type="list") { > > > constant(value="rsyslog.myhost_example_com.main_q.size:") > > > property(name="$!size") > > > constant(value="|g\n") > > > constant(value="rsyslog.myhost_example_com.main_q.enqueued:") > > > property(name="$!enqueued") > > > constant(value="|c|@10\n") > > > constant(value="rsyslog.myhost_example_com.main_q.discarded.full:") > > > property(name="$!discarded.full") > > > constant(value="|c|@10\n") > > > constant(value="rsyslog.myhost_example_com.main_q.discarded.nf:") > > > property(name="$!discarded.nf") > > > constant(value="|c|@10\n") > > > constant(value=“rsyslog.myhost_example_com.main_q.maxqsize:") > > > property(name="$!maxqsize") > > > constant(value="|g\n") > > > } > > > > > > > > > if $syslogtag contains "rsyslogd-pstats" then { > > > action(type="mmjsonparse”) > > > #write to file here for debugging. > > > action(type=“omfile” file=“/var/log/stats.log”) > > > if $!name == "main Q" then { > > > action(type="omfwd" Target="127.0.0.1" Protocol="udp" > Port="8125" > > > template="mainQTemplate”) > > > } > > > stop > > > } > > > > > > </snip> > > > > > > There is still some wonkiness in the enqueued stat as occasionally it > has > > > an absolutely massive unrealistic spike, I have never tracked down why > it > > > does that, but this should give you a start. > > > > > > Cheers > > > mike > > > > > > -- > > > Michael Hart > > > Arctic Wolf Networks > > > M: 226-388-4773 > > > > > > > > > > > > > > > > > > > > > > > > > > > On 2014-11-18, 15:14, "Dave Caplinger" <[email protected]> > > > wrote: > > > > > > >Absolutely. Rsyslog has statistics counters via the impstats module; > > you > > > >can process the log lines it generates to determine the health of the > > > >rsyslog instance, including individual queues, drop rates, forwarding > > > >rates, etc. > > > > > > > >See: > > > > > > > >http://www.rsyslog.com/rsyslog-statistic-counter/ > > > >http://www.rsyslog.com/how-to-use-impstats/ > > > > > > > >-- > > > >Dave Caplinger, Director of Architecture | Ph: (402) 361-3063 | > > > >Solutionary — An NTT Group Security Company > > > > > > > >> On Nov 18, 2014, at 6:46 AM, Damian <[email protected]> > wrote: > > > >> > > > >> Hi, > > > >> I'm trying to determine whether it's possible to monitor the health > of > > > >>an rsyslog daemon running as a forwarder. > > > >> ie. If I'm running it as a component in a logging service, how do I > > > >>check the event rates, or know it's not losing events or queuing > > > >>incoming data. Are there any 'self-monitoring' events that I can > > > >>generate and forward from it, in order to keep an eye on its health? > > > >> Thanks! > > > >> > > > >> Damo > > > >> _______________________________________________ > > > >> rsyslog mailing list > > > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > > >> http://www.rsyslog.com/professional-services/ > > > >> What's up with rsyslog? Follow https://twitter.com/rgerhards > > > >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > > > >>myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT > POST > > > >>if you DON'T LIKE THAT. > > > > > > > >_______________________________________________ > > > >rsyslog mailing list > > > >http://lists.adiscon.net/mailman/listinfo/rsyslog > > > >http://www.rsyslog.com/professional-services/ > > > >What's up with rsyslog? Follow https://twitter.com/rgerhards > > > >NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > myriad > > > >of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > > > >DON'T LIKE THAT. > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com/professional-services/ > > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > myriad > > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > > > DON'T LIKE THAT. > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com/professional-services/ > > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > myriad > > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > > > DON'T LIKE THAT. > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > > DON'T LIKE THAT. > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
