If you can share output of all thread backtrace we can confirm if this is the cause.
On Thu, Oct 1, 2015 at 2:30 PM, <[email protected]> wrote: > Hi, > Ups I was not detailed enough. > The problem with rsyslog-die does not always occur. But sometimes > unexpectedly. > In my environments the files grow or reduce sometimes, so maybe this has > something do with it (or the processing delay). > > regards > Chris > > > -----Ursprüngliche Nachricht----- > Gesendet: Donnerstag, 01 Oktober 2015 um 10:57:37 Uhr > Von: [email protected] > An: "singh.janmejay" <[email protected]>,rsyslog-users > <[email protected]> > Betreff: Re: [rsyslog] Separation of actions based on log source - with good > performance > Hi, > For my opinion it is really good to support looku-tables official. > Thanks for the work on the implementation David & Rainer. > > I have some experiences using lookup-Tables with > 2500 Entries. > > There are 2 open issues: > > 1. There is a bug when sending SIGHUP and reprocessing big lists, which leads > to die of rsyslogd. > I spend some time to identify this bug, unfortunately I'm still not able to > find the exact reason. > The problem seems to occur not directly after sending SIGHUP, but later. > Maybe this has something to do with Queues. > > 2. The "default" Value is not implemented. This should be mentioned in the > documentation or implemented. > I guess its quite less work, but I'm not sure how soon I find the time to do > all the things arround the pure developement... ;) > > > > regards > Chris > > -----Ursprüngliche Nachricht----- > Gesendet: Donnerstag, 01 Oktober 2015 um 09:41:26 Uhr > Von: "singh.janmejay" <[email protected]> > An: rsyslog-users <[email protected]> > Betreff: Re: [rsyslog] Separation of actions based on log source - with good > performance > OK, allow me a few days, I'll add one more test for multiple tables. Will > make the doc change after that. > > -- > Regards, > Janmejay > > PS: Please blame the typos in this mail on my phone's uncivilized soft > keyboard sporting it's not-so-smart-assist technology. > > On Oct 1, 2015 12:29 PM, "Rainer Gerhards" <[email protected]> wrote: > >> 2015-09-29 20:58 GMT+02:00 singh.janmejay <[email protected]>: >> > Sweet, plan on playing with it tomorrow. >> >> If you have verified that the current functionality works fine after >> your patch, I wouldn't object if you modify the doc to tell the world >> that this part of lookup tables is now officially supported. we could >> release with 8.14. I think what currently exists is already pretty >> useful and if we feel confident enough it works, we should release it. >> >> Rainer >> > >> > -- >> > Regards, >> > Janmejay >> > >> > PS: Please blame the typos in this mail on my phone's uncivilized soft >> > keyboard sporting it's not-so-smart-assist technology. >> > >> > On Sep 30, 2015 12:16 AM, "Rainer Gerhards" <[email protected]> >> > wrote: >> > >> >> It's a long time since I implemented what currently is there. It should >> be >> >> relatively solid with probably some minor glitches. It provides the code >> >> functionality as far as I remember. >> >> >> >> Rainer >> >> >> >> Sent from phone, thus brief. >> >> Am 29.09.2015 20:07 schrieb "singh.janmejay" <[email protected] >> >: >> >> >> >> > Rainer/David, >> >> > >> >> > Exactly how much of lookup_table functionality is implemented? >> >> > >> >> > What can I not do with it? (you mentioned something about single table >> >> > in this thread, can you please elaborate?). >> >> > >> >> > On Tue, Mar 31, 2015 at 7:23 PM, Rainer Gerhards >> >> > <[email protected]> wrote: >> >> > > 2015-03-31 15:46 GMT+02:00 <[email protected]>: >> >> > >> Hi, >> >> > >> Do you have some experience how large Lookup-tables can be until >> there >> >> > are "negative" effects? >> >> > >> 2400 entries seems to work fine :) >> >> > > >> >> > > IIRC the current partial implementation is O(log n), so no problem. >> >> > > >> >> > >> >> >> > >> And another question, do I loose events, when doing a kill -HUP >> (for >> >> > update of lookup-table)? >> >> > >> (e.g. client threads are hard "terminated"...) >> >> > > >> >> > > *should* not cause any issues. >> >> > > >> >> > > Rainer >> >> > >> >> >> > >> best regards >> >> > >> Chris >> >> > >> >> >> > >> >> >> > >> >> >> > >> Gesendet: Mittwoch, 25. März 2015 um 19:28 Uhr >> >> > >> Von: "David Lang" <[email protected]> >> >> > >> An: rsyslog-users <[email protected]> >> >> > >> Betreff: Re: [rsyslog] Separation of actions based on log source - >> >> with >> >> > good performance >> >> > >> On Wed, 25 Mar 2015, [email protected] wrote: > Hi, > I was >> >> > doing some experiments with the lookup-table. > Looks really nice and >> the >> >> > performance is promising. > (Unfortunately the evaluation of "nomatch" >> >> > attribute is currently not implemented...) > > Never the less: > My >> plan >> >> > is, to do diffent actions based on the type of host, mapped in the >> >> > lookup-list. > For testing purposes, I use alway omfile. > > >> >> Unfortunately >> >> > it does not work, to change the ruleset based on the variable. > Is >> there >> >> > any other option or is there any mistake? for omfile you can use the >> >> > dynafile approach to use the return variable, for remote things you >> would >> >> > need to do an if then else approach for performance reasons many of >> the >> >> > fields in rsyslog do not accept variables. This allows them to be >> >> > computed/parsed once at startup rather than having to be evaluated for >> >> each >> >> > log message. It's a bit of a hassle when you do want to do something >> >> > dynamic, but even in cases where you have some dynamic things, you >> tend >> >> to >> >> > have other static things that benefit from the speedup. David Lang > >> *** >> >> > syslog.conf *** > lookup_table(name="lookuptable" >> >> > file="/etc/rsyslog.lookup") > set $!dst = lookup("lookuptable", >> >> > $fromhost-ip); > ruleset(name="typea"){ > action(type="omfile" >> >> > file="/var/log/file_typea.log") > } > ruleset(name="typea"){ > >> >> > action(type="omfile" file="/var/log/file_typeb.log") > } > > # Change >> set >> >> > default ruleset, based on sourceip > $DefaultRuleset $!dst > > >> >> > module(load="imtcp" KeepAlive="on" KeepAlive.Probes="1" >> >> > KeepAlive.Interval="2" KeepAlive.Time="20") > input(type="imtcp" >> >> > port="7714") > > *** lookup-table *** > { "version":1, >> "nomatch":"unk", >> >> > "type":"string", > "table":[ {"index":"10.3.5.4", "value":"typea" }, > >> >> > {"index":"10.2.2.1", "value":"typea" }, > {"index":"10.0.2.2", >> >> > "value":"typeb" }, > {"index":"10.2.2.3", "value":"typeb" } > ] > } > >> > > >> >> > best regards > Chris > > > > Gesendet: Dienstag, 24. März 2015 um >> >> 17:14 >> >> > Uhr > Von: [email protected] > An: [email protected] > >> >> > Betreff: Re: [rsyslog] Separation of actions based on log source - >> with >> >> > good performance > Hi David, > > Thanks sounds great, I will try this >> in >> >> > the next days :) > > Chris > > > > Gesendet: Montag, 23. März >> 2015 um >> >> > 17:44 Uhr > Von: "David Lang" > An: rsyslog-users > Betreff: Re: >> >> [rsyslog] >> >> > Separation of actions based on log source - with good performance > >> This >> >> is >> >> > the sort of thing that the table lookup functionality was designed >> for. > >> >> > It wasn't fully implemented to the design (funding fell through), but >> I >> >> > think it works for a single table. > you could use it to do the >> mapping >> >> > from your many hosts to a couple of values and then have your test be >> >> based >> >> > on the resulting value. > > David Lang On Mon, 23 Mar 2015 > [...] > >> >> > >> >> >> > >> _______________________________________________ >> >> > >> rsyslog mailing list >> >> > >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> > >> http://www.rsyslog.com/professional-services/ >> >> > >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> >> > >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> >> > myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >> POST if >> >> > you DON'T LIKE THAT. >> >> > > _______________________________________________ >> >> > > rsyslog mailing list >> >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> > > http://www.rsyslog.com/professional-services/ >> >> > > What's up with rsyslog? Follow https://twitter.com/rgerhards >> >> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> >> myriad >> >> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> >> > DON'T LIKE THAT. >> >> > >> >> > >> >> > >> >> > -- >> >> > Regards, >> >> > Janmejay >> >> > http://codehunk.wordpress.com >> >> > _______________________________________________ >> >> > rsyslog mailing list >> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> > http://www.rsyslog.com/professional-services/ >> >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> myriad >> >> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> >> > DON'T LIKE THAT. >> >> _______________________________________________ >> >> rsyslog mailing list >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> http://www.rsyslog.com/professional-services/ >> >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> >> DON'T LIKE THAT. >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com/professional-services/ >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. -- Regards, Janmejay http://codehunk.wordpress.com _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
