Hi, No, I didn't. I even didn't realize the patch. It seems to be exactly the related issue. So I don't expect any further issues. I will use the new version on 2 systems. If there is any other issue, I will let you know. Release data for next rsyslog version is quite far so enough time to test... ;)
The missing implementation of "nomatch" (default) entry as described at http://www.rsyslog.com/doc/lookup_tables.html would from my opinion require changes: Arround line 132 of lookup.c file (save of value) Arround line 243 of lookup.c file (search in lookuptable fails, so return nomatch value. regards Chris > Gesendet: Donnerstag, 01. Oktober 2015 um 16:57 Uhr > Von: "singh.janmejay" <[email protected]> > An: rsyslog-users <[email protected]> > Betreff: Re: [rsyslog] Separation of actions based on log source - with good > performance > > Yes, if you build off master, that problem should go away (if it was due to > lookup-table). > > On Thu, Oct 1, 2015, 7:00 PM Rainer Gerhards <[email protected]> > wrote: > > > 2015-10-01 15:14 GMT+02:00 singh.janmejay <[email protected]>: > > > If you can share output of all thread backtrace we can confirm if this > > > is the cause. > > > > let's first double-check: Christopher, did you use yesterday evening's > > master branch? Because that contains a patch from Janmejay that I > > think causes the problem for you. Or am I wrong, Janmejay? > > > > Rainer > > > > > > On Thu, Oct 1, 2015 at 2:30 PM, <[email protected]> wrote: > > >> Hi, > > >> Ups I was not detailed enough. > > >> The problem with rsyslog-die does not always occur. But sometimes > > unexpectedly. > > >> In my environments the files grow or reduce sometimes, so maybe this > > has something do with it (or the processing delay). > > >> > > >> regards > > >> Chris > > >> > > >> > > >> -----Ursprüngliche Nachricht----- > > >> Gesendet: Donnerstag, 01 Oktober 2015 um 10:57:37 Uhr > > >> Von: [email protected] > > >> An: "singh.janmejay" <[email protected]>,rsyslog-users < > > [email protected]> > > >> Betreff: Re: [rsyslog] Separation of actions based on log source - with > > good performance > > >> Hi, > > >> For my opinion it is really good to support looku-tables official. > > >> Thanks for the work on the implementation David & Rainer. > > >> > > >> I have some experiences using lookup-Tables with > 2500 Entries. > > >> > > >> There are 2 open issues: > > >> > > >> 1. There is a bug when sending SIGHUP and reprocessing big lists, which > > leads to die of rsyslogd. > > >> I spend some time to identify this bug, unfortunately I'm still not > > able to find the exact reason. > > >> The problem seems to occur not directly after sending SIGHUP, but > > later. Maybe this has something to do with Queues. > > >> > > >> 2. The "default" Value is not implemented. This should be mentioned in > > the documentation or implemented. > > >> I guess its quite less work, but I'm not sure how soon I find the time > > to do all the things arround the pure developement... ;) > > >> > > >> > > >> > > >> regards > > >> Chris > > >> > > >> -----Ursprüngliche Nachricht----- > > >> Gesendet: Donnerstag, 01 Oktober 2015 um 09:41:26 Uhr > > >> Von: "singh.janmejay" <[email protected]> > > >> An: rsyslog-users <[email protected]> > > >> Betreff: Re: [rsyslog] Separation of actions based on log source - with > > good performance > > >> OK, allow me a few days, I'll add one more test for multiple tables. > > Will > > >> make the doc change after that. > > >> > > >> -- > > >> Regards, > > >> Janmejay > > >> > > >> PS: Please blame the typos in this mail on my phone's uncivilized soft > > >> keyboard sporting it's not-so-smart-assist technology. > > >> > > >> On Oct 1, 2015 12:29 PM, "Rainer Gerhards" <[email protected]> > > wrote: > > >> > > >>> 2015-09-29 20:58 GMT+02:00 singh.janmejay <[email protected]>: > > >>> > Sweet, plan on playing with it tomorrow. > > >>> > > >>> If you have verified that the current functionality works fine after > > >>> your patch, I wouldn't object if you modify the doc to tell the world > > >>> that this part of lookup tables is now officially supported. we could > > >>> release with 8.14. I think what currently exists is already pretty > > >>> useful and if we feel confident enough it works, we should release it. > > >>> > > >>> Rainer > > >>> > > > >>> > -- > > >>> > Regards, > > >>> > Janmejay > > >>> > > > >>> > PS: Please blame the typos in this mail on my phone's uncivilized > > soft > > >>> > keyboard sporting it's not-so-smart-assist technology. > > >>> > > > >>> > On Sep 30, 2015 12:16 AM, "Rainer Gerhards" < > > [email protected]> > > >>> > wrote: > > >>> > > > >>> >> It's a long time since I implemented what currently is there. It > > should > > >>> be > > >>> >> relatively solid with probably some minor glitches. It provides the > > code > > >>> >> functionality as far as I remember. > > >>> >> > > >>> >> Rainer > > >>> >> > > >>> >> Sent from phone, thus brief. > > >>> >> Am 29.09.2015 20:07 schrieb "singh.janmejay" < > > [email protected] > > >>> >: > > >>> >> > > >>> >> > Rainer/David, > > >>> >> > > > >>> >> > Exactly how much of lookup_table functionality is implemented? > > >>> >> > > > >>> >> > What can I not do with it? (you mentioned something about single > > table > > >>> >> > in this thread, can you please elaborate?). > > >>> >> > > > >>> >> > On Tue, Mar 31, 2015 at 7:23 PM, Rainer Gerhards > > >>> >> > <[email protected]> wrote: > > >>> >> > > 2015-03-31 15:46 GMT+02:00 <[email protected]>: > > >>> >> > >> Hi, > > >>> >> > >> Do you have some experience how large Lookup-tables can be > > until > > >>> there > > >>> >> > are "negative" effects? > > >>> >> > >> 2400 entries seems to work fine :) > > >>> >> > > > > >>> >> > > IIRC the current partial implementation is O(log n), so no > > problem. > > >>> >> > > > > >>> >> > >> > > >>> >> > >> And another question, do I loose events, when doing a kill -HUP > > >>> (for > > >>> >> > update of lookup-table)? > > >>> >> > >> (e.g. client threads are hard "terminated"...) > > >>> >> > > > > >>> >> > > *should* not cause any issues. > > >>> >> > > > > >>> >> > > Rainer > > >>> >> > >> > > >>> >> > >> best regards > > >>> >> > >> Chris > > >>> >> > >> > > >>> >> > >> > > >>> >> > >> > > >>> >> > >> Gesendet: Mittwoch, 25. März 2015 um 19:28 Uhr > > >>> >> > >> Von: "David Lang" <[email protected]> > > >>> >> > >> An: rsyslog-users <[email protected]> > > >>> >> > >> Betreff: Re: [rsyslog] Separation of actions based on log > > source - > > >>> >> with > > >>> >> > good performance > > >>> >> > >> On Wed, 25 Mar 2015, [email protected] wrote: > Hi, > > > I was > > >>> >> > doing some experiments with the lookup-table. > Looks really nice > > and > > >>> the > > >>> >> > performance is promising. > (Unfortunately the evaluation of > > "nomatch" > > >>> >> > attribute is currently not implemented...) > > Never the less: > > > My > > >>> plan > > >>> >> > is, to do diffent actions based on the type of host, mapped in the > > >>> >> > lookup-list. > For testing purposes, I use alway omfile. > > > > >>> >> Unfortunately > > >>> >> > it does not work, to change the ruleset based on the variable. > > > Is > > >>> there > > >>> >> > any other option or is there any mistake? for omfile you can use > > the > > >>> >> > dynafile approach to use the return variable, for remote things > > you > > >>> would > > >>> >> > need to do an if then else approach for performance reasons many > > of > > >>> the > > >>> >> > fields in rsyslog do not accept variables. This allows them to be > > >>> >> > computed/parsed once at startup rather than having to be > > evaluated for > > >>> >> each > > >>> >> > log message. It's a bit of a hassle when you do want to do > > something > > >>> >> > dynamic, but even in cases where you have some dynamic things, you > > >>> tend > > >>> >> to > > >>> >> > have other static things that benefit from the speedup. David > > Lang > > > >>> *** > > >>> >> > syslog.conf *** > lookup_table(name="lookuptable" > > >>> >> > file="/etc/rsyslog.lookup") > set $!dst = lookup("lookuptable", > > >>> >> > $fromhost-ip); > ruleset(name="typea"){ > action(type="omfile" > > >>> >> > file="/var/log/file_typea.log") > } > ruleset(name="typea"){ > > > >>> >> > action(type="omfile" file="/var/log/file_typeb.log") > } > > # > > Change > > >>> set > > >>> >> > default ruleset, based on sourceip > $DefaultRuleset $!dst > > > > >>> >> > module(load="imtcp" KeepAlive="on" KeepAlive.Probes="1" > > >>> >> > KeepAlive.Interval="2" KeepAlive.Time="20") > input(type="imtcp" > > >>> >> > port="7714") > > *** lookup-table *** > { "version":1, > > >>> "nomatch":"unk", > > >>> >> > "type":"string", > "table":[ {"index":"10.3.5.4", "value":"typea" > > }, > > > >>> >> > {"index":"10.2.2.1", "value":"typea" }, > {"index":"10.0.2.2", > > >>> >> > "value":"typeb" }, > {"index":"10.2.2.3", "value":"typeb" } > ] > > > } > > > >>> > > > > >>> >> > best regards > Chris > > > > Gesendet: Dienstag, 24. März > > 2015 um > > >>> >> 17:14 > > >>> >> > Uhr > Von: [email protected] > An: > > [email protected] > > > >>> >> > Betreff: Re: [rsyslog] Separation of actions based on log source - > > >>> with > > >>> >> > good performance > Hi David, > > Thanks sounds great, I will try > > this > > >>> in > > >>> >> > the next days :) > > Chris > > > > Gesendet: Montag, 23. März > > >>> 2015 um > > >>> >> > 17:44 Uhr > Von: "David Lang" > An: rsyslog-users > Betreff: Re: > > >>> >> [rsyslog] > > >>> >> > Separation of actions based on log source - with good performance > > > > > >>> This > > >>> >> is > > >>> >> > the sort of thing that the table lookup functionality was designed > > >>> for. > > > >>> >> > It wasn't fully implemented to the design (funding fell through), > > but > > >>> I > > >>> >> > think it works for a single table. > you could use it to do the > > >>> mapping > > >>> >> > from your many hosts to a couple of values and then have your > > test be > > >>> >> based > > >>> >> > on the resulting value. > > David Lang On Mon, 23 Mar 2015 > > > [...] > > > >>> >> > >> > > >>> >> > >> _______________________________________________ > > >>> >> > >> rsyslog mailing list > > >>> >> > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > >>> >> > >> http://www.rsyslog.com/professional-services/ > > >>> >> > >> What's up with rsyslog? Follow https://twitter.com/rgerhards > > >>> >> > >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED > > by a > > >>> >> > myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT > > >>> POST if > > >>> >> > you DON'T LIKE THAT. > > >>> >> > > _______________________________________________ > > >>> >> > > rsyslog mailing list > > >>> >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > >>> >> > > http://www.rsyslog.com/professional-services/ > > >>> >> > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > >>> >> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by > > a > > >>> >> myriad > > >>> >> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST > > if you > > >>> >> > DON'T LIKE THAT. > > >>> >> > > > >>> >> > > > >>> >> > > > >>> >> > -- > > >>> >> > Regards, > > >>> >> > Janmejay > > >>> >> > http://codehunk.wordpress.com > > >>> >> > _______________________________________________ > > >>> >> > rsyslog mailing list > > >>> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog > > >>> >> > http://www.rsyslog.com/professional-services/ > > >>> >> > What's up with rsyslog? Follow https://twitter.com/rgerhards > > >>> >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > > >>> myriad > > >>> >> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST > > if you > > >>> >> > DON'T LIKE THAT. > > >>> >> _______________________________________________ > > >>> >> rsyslog mailing list > > >>> >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > >>> >> http://www.rsyslog.com/professional-services/ > > >>> >> What's up with rsyslog? Follow https://twitter.com/rgerhards > > >>> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > > myriad > > >>> >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > > you > > >>> >> DON'T LIKE THAT. > > >>> > _______________________________________________ > > >>> > rsyslog mailing list > > >>> > http://lists.adiscon.net/mailman/listinfo/rsyslog > > >>> > http://www.rsyslog.com/professional-services/ > > >>> > What's up with rsyslog? Follow https://twitter.com/rgerhards > > >>> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > > myriad > > >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > > >>> DON'T LIKE THAT. > > >>> _______________________________________________ > > >>> rsyslog mailing list > > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > > >>> http://www.rsyslog.com/professional-services/ > > >>> What's up with rsyslog? Follow https://twitter.com/rgerhards > > >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > > myriad > > >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > > >>> DON'T LIKE THAT. > > >> _______________________________________________ > > >> rsyslog mailing list > > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > >> http://www.rsyslog.com/professional-services/ > > >> What's up with rsyslog? Follow https://twitter.com/rgerhards > > >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > > myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > > you DON'T LIKE THAT. > > >> _______________________________________________ > > >> rsyslog mailing list > > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > >> http://www.rsyslog.com/professional-services/ > > >> What's up with rsyslog? Follow https://twitter.com/rgerhards > > >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > > myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > > you DON'T LIKE THAT. > > > > > > > > > > > > -- > > > Regards, > > > Janmejay > > > http://codehunk.wordpress.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com/professional-services/ > > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > > DON'T LIKE THAT. > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > > DON'T LIKE THAT. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
