On Mon, Aug 26, 2013 at 4:58 PM, Radu Gheorghe <[email protected]>wrote:
> OK, Rainer! Sounds like a plan :) > > yupp, like one that didn't work out ;) When 7.5.3 is out (hopefully today), I'll try to do this as next thing... No need to hurry for you, but I want to get this done from my PoV before s/t else comes in between again ;) Rainer > > 2013/8/26 Rainer Gerhards <[email protected]> > > > On Mon, Aug 26, 2013 at 2:44 PM, Radu Gheorghe <[email protected] > > >wrote: > > > > > Hi Rainer, > > > > > > Unless someone else jumps in, I'm willing to do it. I'm hesitant > because > > > I'm squeezed with time in this period. But it shouldn't take long, so > > I'll > > > find some time this week. Just let me know when the patch is available > > and > > > where should I apply it (dev/stable). > > > > > > > > great - my plan is to do a new v7-devel release and then look at the > code. > > If all goes well, that would probably mean end of week. The plus is that > I > > don't need immediate feedback, as the next devel release is probably not > > done within a day or two ;) > > > > and, yes, devel would be it. > > > > Rainer > > > > > Best regards, > > > Radu > > > > > > > > > 2013/8/26 Rainer Gerhards <[email protected]> > > > > > > > Would someone be willing to test counters (including fail cases) > when I > > > > improve the instrumentation? (I want to save the time to setup a test > > lab > > > > just for this case...). > > > > > > > > Rainer > > > > > > > > On Thu, Aug 22, 2013 at 1:49 PM, Boylan, James < > > [email protected] > > > > >wrote: > > > > > > > > > That is interesting! We'll definitely need to narrow it down a bit. > > > > > > > > > > I'm currently running off 7.2.5. I haven't sat down and pulled in > the > > > > > 7.4.x branch yet as I've been focusing on implementing the Rsyslog > to > > > > > Elasticsearch infrastructure. I'm agree you were probably in the > > > process > > > > of > > > > > correcting how the stats were being generated and weren't able to > > > finish > > > > > it. The fact that it isn't incrementing failed at all tends to lend > > to > > > > that > > > > > fact since it appears it is the failed counter that is incrementing > > in > > > an > > > > > unexpected way. > > > > > > > > > > Thankfully the way it is implemented in 7.2.5 does at least offer > the > > > > > ability to recognize when it is failing. You just have to account > for > > > the > > > > > difference between the submits and fails. Whatever you have left > when > > > you > > > > > subtract submit from fail is the number of failures. > > > > > > > > > > --James > > > > > > > > > > > > > > > -----Original Message----- > > > > > From: [email protected] [mailto: > > > > > [email protected]] On Behalf Of Rainer Gerhards > > > > > Sent: Thursday, August 22, 2013 6:12 AM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] Elasticsearch and impstats reading question > > > > > > > > > > On Tue, Aug 20, 2013 at 6:34 PM, Boylan, James < > > > [email protected] > > > > > >wrote: > > > > > > > > > > > Rainer - > > > > > > > > > > > > I would double check. It looks like even when successful it > > > increments > > > > > > both failed and submit. But it does an extra failed increment > when > > > > > > there is an error. (This should be confirmed since I'm commenting > > > > > > based on observed behavior. > > > > > > > > > > > > > > > > > First of all, the current state really looks like I either did > > > > > experimental work or was dragged away right in the middle of some > > > > > implementation. I reviewed current git master branch and have to > > admit > > > > that > > > > > what I see makes limited sense (if it were complete). Anyhow, I > have > > > > > documented the current state here: > > > > > > > > > > http://www.rsyslog.com/rsyslog-statistic-counter/ > > > > > > > > > > The interesting thing is that the current code does not even > > increment > > > > > "failed" at all... So which version are you working with? > > > > > > > > > > Let's see if we can clean up the mess ;) > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > -- James > > > > > > > > > > > > -----Original Message----- > > > > > > From: [email protected] [mailto: > > > > > > [email protected]] On Behalf Of Boylan, James > > > > > > Sent: Friday, August 16, 2013 6:06 AM > > > > > > To: rsyslog-users > > > > > > Subject: Re: [rsyslog] Elasticsearch and impstats reading > question > > > > > > > > > > > > That's exactly what I needed. Thanks Rainer. > > > > > > > > > > > > -- James T. Boylan > > > > > > > > > > > > -----Original Message----- > > > > > > From: [email protected] [mailto: > > > > > > [email protected]] On Behalf Of Rainer Gerhards > > > > > > Sent: Thursday, August 15, 2013 9:16 PM > > > > > > To: rsyslog-users > > > > > > Subject: Re: [rsyslog] Elasticsearch and impstats reading > question > > > > > > > > > > > > On Thu, Aug 15, 2013 at 5:57 PM, Radu Gheorghe > > > > > > <[email protected] > > > > > > >wrote: > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > Disclaimer: I've never worked with impstats, only with > > > > Elaseticsearch. > > > > > > > But it sounds like I should, because it provides nice info. > > > > > > > > > > > > > > The fact that you have success=0 and failed=$A_LOT sounds like > > > > > > > something isn't working at all. So I'd start rsyslog in debug > > mode > > > > > > > and see what the reply from Elasticsearch looks like. Feel free > > to > > > > > > > post it here if you need additional help. For me, 9 times out > of > > 10 > > > > > > > it's an invalid JSON :) > > > > > > > > > > > > > > connfail says 0, so I guess it connects OK. But I can't find > any > > > > > > > documentation on impstats for elasticsearch here: > > > > > > > http://www.rsyslog.com/rsyslog-statistic-counter/ > > > > > > > > > > > > > > > > > > > > looks like this was a recent addition where I either forgot or > did > > > not > > > > > > manage to update the counters. > > > > > > > > > > > > > > > > > > > I'd gladly do some testing and contribute documentation to this > > > (I'm > > > > > > > interested in these counters). Can anyone give some hints? I've > > > > > > > looked in omelasticsearch.c and I see the counters there > (submit, > > > > > > > connfail, success, failed), but I don't understand how they're > > > > > gathered. > > > > > > > > > > > > > > > > > > > > had a quick look as well, it's essentially straightforward: > > > > > > > > > > > > connfail - connection to ES failed > > > > > > submit - successful submit of record > > > > > > failed - some error occured (Debug log will show more detail) > > success > > > > > > -- seems currently unused (need to check when I am back). > > > > > > > > > > > > Rainer > > > > > > > > > > > > > Best regards, > > > > > > > Radu > > > > > > > > > > > > > > > > > > > > > 2013/8/15 Boylan, James <[email protected]> > > > > > > > > > > > > > > > Hey Everyone! > > > > > > > > > > > > > > > > I'm trying to better understand what I'm seeing in the output > > of > > > > > > > > the impstats module. Could someone help confirm that I'm > seeing > > > > > > > > that Elasticsearch is failing to receive a large portion of > the > > > > > > > > messages being sent at it. > > > > > > > > > > > > > > > > 2013-07-02T14:27:26.298686-05:00 > > > > > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: main Q: size=44400 > > > > > > > enqueued=1010337 full=0 > > > > > > > > discarded.full=0 discarded.nf=0 maxqsize=45886 > > > > > > > > 2013-07-02T14:27:26.298642-05:00 > > > > > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: elasticsearch: > > > > > > > connfail=0 > > > > > > > submits=4763 failed=10538 > > > > > > > > success=0 > > > > > > > > 2013-07-02T14:27:26.298647-05:00 > > > > > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 1: > > > > > > > processed=11913 > > > > > > > failed=0 > > > > > > > > 2013-07-02T14:27:27.299756-05:00 > > > > > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 2: > > > > > > > processed=11913 > > > > > > > failed=0 > > > > > > > > 2013-07-02T14:27:27.299758-05:00 > > > > > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 3: > > > > > > > processed=955943 > > > > > > > failed=0 > > > > > > > > 2013-07-02T14:27:27.299761-05:00 > > > > > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 4: > > > > > > > processed=955874 > > > > > > > failed=0 > > > > > > > > 2013-07-02T14:27:27.299764-05:00 > > > > > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 5: > > > > > > > processed=967728 > > > > > > > failed=0 > > > > > > > > 2013-07-02T14:27:27.299772-05:00 > > > > > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 7: > > processed=0 > > > > > > > failed=0 > > > > > > > > 2013-07-02T14:27:27.299775-05:00 > > > > > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 8: > > processed=0 > > > > > > > failed=0 > > > > > > > > 2013-07-02T14:27:27.299777-05:00 > > > > > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 9: > > processed=0 > > > > > > > failed=0 > > > > > > > > 2013-07-02T14:27:27.299779-05:00 > > > > > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 10: > > processed=0 > > > > > > > failed=0 > > > > > > > > 2013-07-02T14:27:27.299781-05:00 > > > > > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 11: > > processed=0 > > > > > > > failed=0 > > > > > > > > 2013-07-02T14:27:27.299783-05:00 > > > > > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 12: > > processed=0 > > > > > > > failed=0 > > > > > > > > 2013-07-02T14:27:27.299785-05:00 > > > > > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 13: > > processed=0 > > > > > > > failed=0 > > > > > > > > 2013-07-02T14:27:27.299789-05:00 > > > > > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: > > imptcp(*/22516/IPv4): > > > > > > > submitted=0 > > > > > > > > 2013-07-02T14:27:27.299791-05:00 > > > > > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: > > imptcp(*/22516/IPv6): > > > > > > > submitted=0 > > > > > > > > 2013-07-02T14:27:27.299794-05:00 > > > > > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: > > imptcp(*/21516/IPv4): > > > > > > > submitted=1000681 > > > > > > > > 2013-07-02T14:27:27.299795-05:00 > > > > > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: > > imptcp(*/21516/IPv6): > > > > > > > submitted=0 > > > > > > > > 2013-07-02T14:27:27.299799-05:00 > > > > > > > wmlogserv01s.stag.orbitz.netrsyslogd-pstats: action 4 queue: > > > > > > > size=65536 > > > > > > > enqueued=955874 full=72 > > > > > > > > discarded.full=0 discarded.nf=0 maxqsize=65536 > > > > > > > > > > > > > > > > Thanks! > > > > > > > > > > > > > > > > --James > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > > rsyslog mailing list > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > http://www.rsyslog.com/professional-services/ > > > > > > > > What's up with rsyslog? Follow > > https://twitter.com/rgerhardsNOTE > > > > > > > > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > > > > > > > > myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO > > NOT > > > > > > > > POST if you DON'T LIKE THAT. > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > rsyslog mailing list > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > http://www.rsyslog.com/professional-services/ > > > > > > > What's up with rsyslog? Follow > https://twitter.com/rgerhardsNOTE > > > > > > > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > > myriad > > > > > > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST > > if > > > > > > > you DON'T LIKE THAT. > > > > > > > > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com/professional-services/ > > > > > > What's up with rsyslog? Follow https://twitter.com/rgerhardsNOTE > > > > WELL: > > > > > > This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > > > sites > > > > > > beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T > > > > > > LIKE THAT. > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com/professional-services/ > > > > > > What's up with rsyslog? Follow https://twitter.com/rgerhardsNOTE > > > > WELL: > > > > > > This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > > > sites > > > > > > beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T > > > > > > LIKE THAT. > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com/professional-services/ > > > > > > What's up with rsyslog? Follow https://twitter.com/rgerhardsNOTE > > > > > > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > myriad > > > of > > > > > > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > you > > > > > > DON'T LIKE THAT. > > > > > > > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com/professional-services/ > > > > > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE > > > WELL: > > > > > This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > > sites > > > > > beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > > > LIKE > > > > > THAT. > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com/professional-services/ > > > > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > > > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > > > myriad > > > > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > > you > > > > > DON'T LIKE THAT. > > > > > > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com/professional-services/ > > > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > > myriad > > > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > you > > > > DON'T LIKE THAT. > > > > > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com/professional-services/ > > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > myriad > > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > > > DON'T LIKE THAT. > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > > DON'T LIKE THAT. > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
