Rainer - I would double check. It looks like even when successful it increments both failed and submit. But it does an extra failed increment when there is an error. (This should be confirmed since I'm commenting based on observed behavior.
-- James -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Boylan, James Sent: Friday, August 16, 2013 6:06 AM To: rsyslog-users Subject: Re: [rsyslog] Elasticsearch and impstats reading question That's exactly what I needed. Thanks Rainer. -- James T. Boylan -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Rainer Gerhards Sent: Thursday, August 15, 2013 9:16 PM To: rsyslog-users Subject: Re: [rsyslog] Elasticsearch and impstats reading question On Thu, Aug 15, 2013 at 5:57 PM, Radu Gheorghe <[email protected]>wrote: > Hello, > > Disclaimer: I've never worked with impstats, only with Elaseticsearch. > But it sounds like I should, because it provides nice info. > > The fact that you have success=0 and failed=$A_LOT sounds like > something isn't working at all. So I'd start rsyslog in debug mode and > see what the reply from Elasticsearch looks like. Feel free to post it > here if you need additional help. For me, 9 times out of 10 it's an > invalid JSON :) > > connfail says 0, so I guess it connects OK. But I can't find any > documentation on impstats for elasticsearch here: > http://www.rsyslog.com/rsyslog-statistic-counter/ > > looks like this was a recent addition where I either forgot or did not manage to update the counters. > I'd gladly do some testing and contribute documentation to this (I'm > interested in these counters). Can anyone give some hints? I've looked > in omelasticsearch.c and I see the counters there (submit, connfail, > success, failed), but I don't understand how they're gathered. > > had a quick look as well, it's essentially straightforward: connfail - connection to ES failed submit - successful submit of record failed - some error occured (Debug log will show more detail) success -- seems currently unused (need to check when I am back). Rainer > Best regards, > Radu > > > 2013/8/15 Boylan, James <[email protected]> > > > Hey Everyone! > > > > I'm trying to better understand what I'm seeing in the output of the > > impstats module. Could someone help confirm that I'm seeing that > > Elasticsearch is failing to receive a large portion of the messages > > being sent at it. > > > > 2013-07-02T14:27:26.298686-05:00 > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: main Q: size=44400 > enqueued=1010337 full=0 > > discarded.full=0 discarded.nf=0 maxqsize=45886 > > 2013-07-02T14:27:26.298642-05:00 > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: elasticsearch: connfail=0 > submits=4763 failed=10538 > > success=0 > > 2013-07-02T14:27:26.298647-05:00 > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 1: processed=11913 > failed=0 > > 2013-07-02T14:27:27.299756-05:00 > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 2: processed=11913 > failed=0 > > 2013-07-02T14:27:27.299758-05:00 > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 3: > processed=955943 > failed=0 > > 2013-07-02T14:27:27.299761-05:00 > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 4: > processed=955874 > failed=0 > > 2013-07-02T14:27:27.299764-05:00 > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 5: > processed=967728 > failed=0 > > 2013-07-02T14:27:27.299772-05:00 > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 7: processed=0 > failed=0 > > 2013-07-02T14:27:27.299775-05:00 > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 8: processed=0 > failed=0 > > 2013-07-02T14:27:27.299777-05:00 > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 9: processed=0 > failed=0 > > 2013-07-02T14:27:27.299779-05:00 > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 10: processed=0 > failed=0 > > 2013-07-02T14:27:27.299781-05:00 > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 11: processed=0 > failed=0 > > 2013-07-02T14:27:27.299783-05:00 > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 12: processed=0 > failed=0 > > 2013-07-02T14:27:27.299785-05:00 > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 13: processed=0 > failed=0 > > 2013-07-02T14:27:27.299789-05:00 > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: imptcp(*/22516/IPv4): > submitted=0 > > 2013-07-02T14:27:27.299791-05:00 > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: imptcp(*/22516/IPv6): > submitted=0 > > 2013-07-02T14:27:27.299794-05:00 > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: imptcp(*/21516/IPv4): > submitted=1000681 > > 2013-07-02T14:27:27.299795-05:00 > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: imptcp(*/21516/IPv6): > submitted=0 > > 2013-07-02T14:27:27.299799-05:00 > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 4 queue: > size=65536 > enqueued=955874 full=72 > > discarded.full=0 discarded.nf=0 maxqsize=65536 > > > > Thanks! > > > > --James > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE > > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > > you DON'T LIKE THAT. > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
