Would someone be willing to test counters (including fail cases) when I improve the instrumentation? (I want to save the time to setup a test lab just for this case...).
Rainer On Thu, Aug 22, 2013 at 1:49 PM, Boylan, James <[email protected]>wrote: > That is interesting! We'll definitely need to narrow it down a bit. > > I'm currently running off 7.2.5. I haven't sat down and pulled in the > 7.4.x branch yet as I've been focusing on implementing the Rsyslog to > Elasticsearch infrastructure. I'm agree you were probably in the process of > correcting how the stats were being generated and weren't able to finish > it. The fact that it isn't incrementing failed at all tends to lend to that > fact since it appears it is the failed counter that is incrementing in an > unexpected way. > > Thankfully the way it is implemented in 7.2.5 does at least offer the > ability to recognize when it is failing. You just have to account for the > difference between the submits and fails. Whatever you have left when you > subtract submit from fail is the number of failures. > > --James > > > -----Original Message----- > From: [email protected] [mailto: > [email protected]] On Behalf Of Rainer Gerhards > Sent: Thursday, August 22, 2013 6:12 AM > To: rsyslog-users > Subject: Re: [rsyslog] Elasticsearch and impstats reading question > > On Tue, Aug 20, 2013 at 6:34 PM, Boylan, James <[email protected] > >wrote: > > > Rainer - > > > > I would double check. It looks like even when successful it increments > > both failed and submit. But it does an extra failed increment when > > there is an error. (This should be confirmed since I'm commenting > > based on observed behavior. > > > > > First of all, the current state really looks like I either did > experimental work or was dragged away right in the middle of some > implementation. I reviewed current git master branch and have to admit that > what I see makes limited sense (if it were complete). Anyhow, I have > documented the current state here: > > http://www.rsyslog.com/rsyslog-statistic-counter/ > > The interesting thing is that the current code does not even increment > "failed" at all... So which version are you working with? > > Let's see if we can clean up the mess ;) > > Rainer > > > > -- James > > > > -----Original Message----- > > From: [email protected] [mailto: > > [email protected]] On Behalf Of Boylan, James > > Sent: Friday, August 16, 2013 6:06 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] Elasticsearch and impstats reading question > > > > That's exactly what I needed. Thanks Rainer. > > > > -- James T. Boylan > > > > -----Original Message----- > > From: [email protected] [mailto: > > [email protected]] On Behalf Of Rainer Gerhards > > Sent: Thursday, August 15, 2013 9:16 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] Elasticsearch and impstats reading question > > > > On Thu, Aug 15, 2013 at 5:57 PM, Radu Gheorghe > > <[email protected] > > >wrote: > > > > > Hello, > > > > > > Disclaimer: I've never worked with impstats, only with Elaseticsearch. > > > But it sounds like I should, because it provides nice info. > > > > > > The fact that you have success=0 and failed=$A_LOT sounds like > > > something isn't working at all. So I'd start rsyslog in debug mode > > > and see what the reply from Elasticsearch looks like. Feel free to > > > post it here if you need additional help. For me, 9 times out of 10 > > > it's an invalid JSON :) > > > > > > connfail says 0, so I guess it connects OK. But I can't find any > > > documentation on impstats for elasticsearch here: > > > http://www.rsyslog.com/rsyslog-statistic-counter/ > > > > > > > > looks like this was a recent addition where I either forgot or did not > > manage to update the counters. > > > > > > > I'd gladly do some testing and contribute documentation to this (I'm > > > interested in these counters). Can anyone give some hints? I've > > > looked in omelasticsearch.c and I see the counters there (submit, > > > connfail, success, failed), but I don't understand how they're > gathered. > > > > > > > > had a quick look as well, it's essentially straightforward: > > > > connfail - connection to ES failed > > submit - successful submit of record > > failed - some error occured (Debug log will show more detail) success > > -- seems currently unused (need to check when I am back). > > > > Rainer > > > > > Best regards, > > > Radu > > > > > > > > > 2013/8/15 Boylan, James <[email protected]> > > > > > > > Hey Everyone! > > > > > > > > I'm trying to better understand what I'm seeing in the output of > > > > the impstats module. Could someone help confirm that I'm seeing > > > > that Elasticsearch is failing to receive a large portion of the > > > > messages being sent at it. > > > > > > > > 2013-07-02T14:27:26.298686-05:00 > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: main Q: size=44400 > > > enqueued=1010337 full=0 > > > > discarded.full=0 discarded.nf=0 maxqsize=45886 > > > > 2013-07-02T14:27:26.298642-05:00 > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: elasticsearch: > > > connfail=0 > > > submits=4763 failed=10538 > > > > success=0 > > > > 2013-07-02T14:27:26.298647-05:00 > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 1: > > > processed=11913 > > > failed=0 > > > > 2013-07-02T14:27:27.299756-05:00 > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 2: > > > processed=11913 > > > failed=0 > > > > 2013-07-02T14:27:27.299758-05:00 > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 3: > > > processed=955943 > > > failed=0 > > > > 2013-07-02T14:27:27.299761-05:00 > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 4: > > > processed=955874 > > > failed=0 > > > > 2013-07-02T14:27:27.299764-05:00 > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 5: > > > processed=967728 > > > failed=0 > > > > 2013-07-02T14:27:27.299772-05:00 > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 7: processed=0 > > > failed=0 > > > > 2013-07-02T14:27:27.299775-05:00 > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 8: processed=0 > > > failed=0 > > > > 2013-07-02T14:27:27.299777-05:00 > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 9: processed=0 > > > failed=0 > > > > 2013-07-02T14:27:27.299779-05:00 > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 10: processed=0 > > > failed=0 > > > > 2013-07-02T14:27:27.299781-05:00 > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 11: processed=0 > > > failed=0 > > > > 2013-07-02T14:27:27.299783-05:00 > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 12: processed=0 > > > failed=0 > > > > 2013-07-02T14:27:27.299785-05:00 > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 13: processed=0 > > > failed=0 > > > > 2013-07-02T14:27:27.299789-05:00 > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: imptcp(*/22516/IPv4): > > > submitted=0 > > > > 2013-07-02T14:27:27.299791-05:00 > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: imptcp(*/22516/IPv6): > > > submitted=0 > > > > 2013-07-02T14:27:27.299794-05:00 > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: imptcp(*/21516/IPv4): > > > submitted=1000681 > > > > 2013-07-02T14:27:27.299795-05:00 > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: imptcp(*/21516/IPv6): > > > submitted=0 > > > > 2013-07-02T14:27:27.299799-05:00 > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 4 queue: > > > size=65536 > > > enqueued=955874 full=72 > > > > discarded.full=0 discarded.nf=0 maxqsize=65536 > > > > > > > > Thanks! > > > > > > > > --James > > > > > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com/professional-services/ > > > > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE > > > > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > > > > myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT > > > > POST if you DON'T LIKE THAT. > > > > > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com/professional-services/ > > > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE > > > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > > > you DON'T LIKE THAT. > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: > > This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites > > beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > > LIKE THAT. > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: > > This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites > > beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > > LIKE THAT. > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE > > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > > DON'T LIKE THAT. > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: > This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites > beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE > THAT. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
