2015-10-01 15:14 GMT+02:00 singh.janmejay <[email protected]>: > If you can share output of all thread backtrace we can confirm if this > is the cause.
let's first double-check: Christopher, did you use yesterday evening's master branch? Because that contains a patch from Janmejay that I think causes the problem for you. Or am I wrong, Janmejay? Rainer > > On Thu, Oct 1, 2015 at 2:30 PM, <[email protected]> wrote: >> Hi, >> Ups I was not detailed enough. >> The problem with rsyslog-die does not always occur. But sometimes >> unexpectedly. >> In my environments the files grow or reduce sometimes, so maybe this has >> something do with it (or the processing delay). >> >> regards >> Chris >> >> >> -----Ursprüngliche Nachricht----- >> Gesendet: Donnerstag, 01 Oktober 2015 um 10:57:37 Uhr >> Von: [email protected] >> An: "singh.janmejay" <[email protected]>,rsyslog-users >> <[email protected]> >> Betreff: Re: [rsyslog] Separation of actions based on log source - with good >> performance >> Hi, >> For my opinion it is really good to support looku-tables official. >> Thanks for the work on the implementation David & Rainer. >> >> I have some experiences using lookup-Tables with > 2500 Entries. >> >> There are 2 open issues: >> >> 1. There is a bug when sending SIGHUP and reprocessing big lists, which >> leads to die of rsyslogd. >> I spend some time to identify this bug, unfortunately I'm still not able to >> find the exact reason. >> The problem seems to occur not directly after sending SIGHUP, but later. >> Maybe this has something to do with Queues. >> >> 2. The "default" Value is not implemented. This should be mentioned in the >> documentation or implemented. >> I guess its quite less work, but I'm not sure how soon I find the time to do >> all the things arround the pure developement... ;) >> >> >> >> regards >> Chris >> >> -----Ursprüngliche Nachricht----- >> Gesendet: Donnerstag, 01 Oktober 2015 um 09:41:26 Uhr >> Von: "singh.janmejay" <[email protected]> >> An: rsyslog-users <[email protected]> >> Betreff: Re: [rsyslog] Separation of actions based on log source - with good >> performance >> OK, allow me a few days, I'll add one more test for multiple tables. Will >> make the doc change after that. >> >> -- >> Regards, >> Janmejay >> >> PS: Please blame the typos in this mail on my phone's uncivilized soft >> keyboard sporting it's not-so-smart-assist technology. >> >> On Oct 1, 2015 12:29 PM, "Rainer Gerhards" <[email protected]> wrote: >> >>> 2015-09-29 20:58 GMT+02:00 singh.janmejay <[email protected]>: >>> > Sweet, plan on playing with it tomorrow. >>> >>> If you have verified that the current functionality works fine after >>> your patch, I wouldn't object if you modify the doc to tell the world >>> that this part of lookup tables is now officially supported. we could >>> release with 8.14. I think what currently exists is already pretty >>> useful and if we feel confident enough it works, we should release it. >>> >>> Rainer >>> > >>> > -- >>> > Regards, >>> > Janmejay >>> > >>> > PS: Please blame the typos in this mail on my phone's uncivilized soft >>> > keyboard sporting it's not-so-smart-assist technology. >>> > >>> > On Sep 30, 2015 12:16 AM, "Rainer Gerhards" <[email protected]> >>> > wrote: >>> > >>> >> It's a long time since I implemented what currently is there. It should >>> be >>> >> relatively solid with probably some minor glitches. It provides the code >>> >> functionality as far as I remember. >>> >> >>> >> Rainer >>> >> >>> >> Sent from phone, thus brief. >>> >> Am 29.09.2015 20:07 schrieb "singh.janmejay" <[email protected] >>> >: >>> >> >>> >> > Rainer/David, >>> >> > >>> >> > Exactly how much of lookup_table functionality is implemented? >>> >> > >>> >> > What can I not do with it? (you mentioned something about single table >>> >> > in this thread, can you please elaborate?). >>> >> > >>> >> > On Tue, Mar 31, 2015 at 7:23 PM, Rainer Gerhards >>> >> > <[email protected]> wrote: >>> >> > > 2015-03-31 15:46 GMT+02:00 <[email protected]>: >>> >> > >> Hi, >>> >> > >> Do you have some experience how large Lookup-tables can be until >>> there >>> >> > are "negative" effects? >>> >> > >> 2400 entries seems to work fine :) >>> >> > > >>> >> > > IIRC the current partial implementation is O(log n), so no problem. >>> >> > > >>> >> > >> >>> >> > >> And another question, do I loose events, when doing a kill -HUP >>> (for >>> >> > update of lookup-table)? >>> >> > >> (e.g. client threads are hard "terminated"...) >>> >> > > >>> >> > > *should* not cause any issues. >>> >> > > >>> >> > > Rainer >>> >> > >> >>> >> > >> best regards >>> >> > >> Chris >>> >> > >> >>> >> > >> >>> >> > >> >>> >> > >> Gesendet: Mittwoch, 25. März 2015 um 19:28 Uhr >>> >> > >> Von: "David Lang" <[email protected]> >>> >> > >> An: rsyslog-users <[email protected]> >>> >> > >> Betreff: Re: [rsyslog] Separation of actions based on log source - >>> >> with >>> >> > good performance >>> >> > >> On Wed, 25 Mar 2015, [email protected] wrote: > Hi, > I was >>> >> > doing some experiments with the lookup-table. > Looks really nice and >>> the >>> >> > performance is promising. > (Unfortunately the evaluation of "nomatch" >>> >> > attribute is currently not implemented...) > > Never the less: > My >>> plan >>> >> > is, to do diffent actions based on the type of host, mapped in the >>> >> > lookup-list. > For testing purposes, I use alway omfile. > > >>> >> Unfortunately >>> >> > it does not work, to change the ruleset based on the variable. > Is >>> there >>> >> > any other option or is there any mistake? for omfile you can use the >>> >> > dynafile approach to use the return variable, for remote things you >>> would >>> >> > need to do an if then else approach for performance reasons many of >>> the >>> >> > fields in rsyslog do not accept variables. This allows them to be >>> >> > computed/parsed once at startup rather than having to be evaluated for >>> >> each >>> >> > log message. It's a bit of a hassle when you do want to do something >>> >> > dynamic, but even in cases where you have some dynamic things, you >>> tend >>> >> to >>> >> > have other static things that benefit from the speedup. David Lang > >>> *** >>> >> > syslog.conf *** > lookup_table(name="lookuptable" >>> >> > file="/etc/rsyslog.lookup") > set $!dst = lookup("lookuptable", >>> >> > $fromhost-ip); > ruleset(name="typea"){ > action(type="omfile" >>> >> > file="/var/log/file_typea.log") > } > ruleset(name="typea"){ > >>> >> > action(type="omfile" file="/var/log/file_typeb.log") > } > > # Change >>> set >>> >> > default ruleset, based on sourceip > $DefaultRuleset $!dst > > >>> >> > module(load="imtcp" KeepAlive="on" KeepAlive.Probes="1" >>> >> > KeepAlive.Interval="2" KeepAlive.Time="20") > input(type="imtcp" >>> >> > port="7714") > > *** lookup-table *** > { "version":1, >>> "nomatch":"unk", >>> >> > "type":"string", > "table":[ {"index":"10.3.5.4", "value":"typea" }, > >>> >> > {"index":"10.2.2.1", "value":"typea" }, > {"index":"10.0.2.2", >>> >> > "value":"typeb" }, > {"index":"10.2.2.3", "value":"typeb" } > ] > } > >>> > > >>> >> > best regards > Chris > > > > Gesendet: Dienstag, 24. März 2015 um >>> >> 17:14 >>> >> > Uhr > Von: [email protected] > An: [email protected] > >>> >> > Betreff: Re: [rsyslog] Separation of actions based on log source - >>> with >>> >> > good performance > Hi David, > > Thanks sounds great, I will try this >>> in >>> >> > the next days :) > > Chris > > > > Gesendet: Montag, 23. März >>> 2015 um >>> >> > 17:44 Uhr > Von: "David Lang" > An: rsyslog-users > Betreff: Re: >>> >> [rsyslog] >>> >> > Separation of actions based on log source - with good performance > >>> This >>> >> is >>> >> > the sort of thing that the table lookup functionality was designed >>> for. > >>> >> > It wasn't fully implemented to the design (funding fell through), but >>> I >>> >> > think it works for a single table. > you could use it to do the >>> mapping >>> >> > from your many hosts to a couple of values and then have your test be >>> >> based >>> >> > on the resulting value. > > David Lang On Mon, 23 Mar 2015 > [...] > >>> >> > >> >>> >> > >> _______________________________________________ >>> >> > >> rsyslog mailing list >>> >> > >> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >> > >> http://www.rsyslog.com/professional-services/ >>> >> > >> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> >> > >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>> >> > myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >>> POST if >>> >> > you DON'T LIKE THAT. >>> >> > > _______________________________________________ >>> >> > > rsyslog mailing list >>> >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >> > > http://www.rsyslog.com/professional-services/ >>> >> > > What's up with rsyslog? Follow https://twitter.com/rgerhards >>> >> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>> >> myriad >>> >> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>> >> > DON'T LIKE THAT. >>> >> > >>> >> > >>> >> > >>> >> > -- >>> >> > Regards, >>> >> > Janmejay >>> >> > http://codehunk.wordpress.com >>> >> > _______________________________________________ >>> >> > rsyslog mailing list >>> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >> > http://www.rsyslog.com/professional-services/ >>> >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >>> >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>> myriad >>> >> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>> >> > DON'T LIKE THAT. >>> >> _______________________________________________ >>> >> rsyslog mailing list >>> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >> http://www.rsyslog.com/professional-services/ >>> >> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>> >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>> >> DON'T LIKE THAT. >>> > _______________________________________________ >>> > rsyslog mailing list >>> > http://lists.adiscon.net/mailman/listinfo/rsyslog >>> > http://www.rsyslog.com/professional-services/ >>> > What's up with rsyslog? Follow https://twitter.com/rgerhards >>> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>> DON'T LIKE THAT. >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>> DON'T LIKE THAT. >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of >> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T >> LIKE THAT. >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of >> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T >> LIKE THAT. > > > > -- > Regards, > Janmejay > http://codehunk.wordpress.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
