Sorry, folks, good intent always seems to find someone who turns it into negative. I was yesterday contacted by a distro maintainer who wants to turn this bug in the officially non-existant lookup table feature into a CVE and insists that it is a vuln even after the argument that the feature never oficially existed.
It looks like it was a bad idea to merge potentially useful yet incomplete code into the main branch (and documenting it to be not present). It looks like I need to re-think my stance on experimental features. Anyhow, I really don't want to support the argument that something non-existing can be a CVE. As such, I will create a new master-insecure branch, which will be a clone of the current master branch. Then I'll remove the lookup table code, so that the code base matches the documentation. I really don't want to create a general principle here that we need to create CVEs (and patched) for something that was just added as a convenience for a handful of folks who were ready to take a risk. If there is sufficient interest, we can consider officially adding this feature to the January 8.15 release iff it is ready by then. @janmejay: please let me know if you would like to continue with your work on lookup tables under this new situation. As soon as I have time, I'll check what else needs to be removed. Not sure about the ./contributed branch, because the project cannot guarantee at all this is bug-free. It's documented to be so, but if that is not sufficient, it should probably live only in the master-insecure branch. Rainer 2015-10-02 17:29 GMT+02:00 singh.janmejay <[email protected]>: > As of now it returns empty string for no-match. I guess we should go ahead > with it in its current form. We can add default value any time later > without breaking compatibility(it'd default to ""). > > I'll add a test for multiple tables on Monday. > > On Fri, Oct 2, 2015, 7:16 PM <[email protected]> wrote: > >> Hi, >> No, I didn't. I even didn't realize the patch. >> >> It seems to be exactly the related issue. So I don't expect any further >> issues. >> I will use the new version on 2 systems. If there is any other issue, I >> will let you know. >> Release data for next rsyslog version is quite far so enough time to >> test... ;) >> >> The missing implementation of "nomatch" (default) entry as described at >> http://www.rsyslog.com/doc/lookup_tables.html >> would from my opinion require changes: >> >> Arround line 132 of lookup.c file (save of value) >> Arround line 243 of lookup.c file (search in lookuptable fails, so return >> nomatch value. >> >> >> regards >> Chris >> >> >> > Gesendet: Donnerstag, 01. Oktober 2015 um 16:57 Uhr >> > Von: "singh.janmejay" <[email protected]> >> > An: rsyslog-users <[email protected]> >> > Betreff: Re: [rsyslog] Separation of actions based on log source - with >> good performance >> > >> > Yes, if you build off master, that problem should go away (if it was due >> to >> > lookup-table). >> > >> > On Thu, Oct 1, 2015, 7:00 PM Rainer Gerhards <[email protected]> >> > wrote: >> > >> > > 2015-10-01 15:14 GMT+02:00 singh.janmejay <[email protected]>: >> > > > If you can share output of all thread backtrace we can confirm if >> this >> > > > is the cause. >> > > >> > > let's first double-check: Christopher, did you use yesterday evening's >> > > master branch? Because that contains a patch from Janmejay that I >> > > think causes the problem for you. Or am I wrong, Janmejay? >> > > >> > > Rainer >> > > > >> > > > On Thu, Oct 1, 2015 at 2:30 PM, <[email protected]> wrote: >> > > >> Hi, >> > > >> Ups I was not detailed enough. >> > > >> The problem with rsyslog-die does not always occur. But sometimes >> > > unexpectedly. >> > > >> In my environments the files grow or reduce sometimes, so maybe this >> > > has something do with it (or the processing delay). >> > > >> >> > > >> regards >> > > >> Chris >> > > >> >> > > >> >> > > >> -----Ursprüngliche Nachricht----- >> > > >> Gesendet: Donnerstag, 01 Oktober 2015 um 10:57:37 Uhr >> > > >> Von: [email protected] >> > > >> An: "singh.janmejay" <[email protected]>,rsyslog-users < >> > > [email protected]> >> > > >> Betreff: Re: [rsyslog] Separation of actions based on log source - >> with >> > > good performance >> > > >> Hi, >> > > >> For my opinion it is really good to support looku-tables official. >> > > >> Thanks for the work on the implementation David & Rainer. >> > > >> >> > > >> I have some experiences using lookup-Tables with > 2500 Entries. >> > > >> >> > > >> There are 2 open issues: >> > > >> >> > > >> 1. There is a bug when sending SIGHUP and reprocessing big lists, >> which >> > > leads to die of rsyslogd. >> > > >> I spend some time to identify this bug, unfortunately I'm still not >> > > able to find the exact reason. >> > > >> The problem seems to occur not directly after sending SIGHUP, but >> > > later. Maybe this has something to do with Queues. >> > > >> >> > > >> 2. The "default" Value is not implemented. This should be mentioned >> in >> > > the documentation or implemented. >> > > >> I guess its quite less work, but I'm not sure how soon I find the >> time >> > > to do all the things arround the pure developement... ;) >> > > >> >> > > >> >> > > >> >> > > >> regards >> > > >> Chris >> > > >> >> > > >> -----Ursprüngliche Nachricht----- >> > > >> Gesendet: Donnerstag, 01 Oktober 2015 um 09:41:26 Uhr >> > > >> Von: "singh.janmejay" <[email protected]> >> > > >> An: rsyslog-users <[email protected]> >> > > >> Betreff: Re: [rsyslog] Separation of actions based on log source - >> with >> > > good performance >> > > >> OK, allow me a few days, I'll add one more test for multiple tables. >> > > Will >> > > >> make the doc change after that. >> > > >> >> > > >> -- >> > > >> Regards, >> > > >> Janmejay >> > > >> >> > > >> PS: Please blame the typos in this mail on my phone's uncivilized >> soft >> > > >> keyboard sporting it's not-so-smart-assist technology. >> > > >> >> > > >> On Oct 1, 2015 12:29 PM, "Rainer Gerhards" < >> [email protected]> >> > > wrote: >> > > >> >> > > >>> 2015-09-29 20:58 GMT+02:00 singh.janmejay < >> [email protected]>: >> > > >>> > Sweet, plan on playing with it tomorrow. >> > > >>> >> > > >>> If you have verified that the current functionality works fine >> after >> > > >>> your patch, I wouldn't object if you modify the doc to tell the >> world >> > > >>> that this part of lookup tables is now officially supported. we >> could >> > > >>> release with 8.14. I think what currently exists is already pretty >> > > >>> useful and if we feel confident enough it works, we should release >> it. >> > > >>> >> > > >>> Rainer >> > > >>> > >> > > >>> > -- >> > > >>> > Regards, >> > > >>> > Janmejay >> > > >>> > >> > > >>> > PS: Please blame the typos in this mail on my phone's uncivilized >> > > soft >> > > >>> > keyboard sporting it's not-so-smart-assist technology. >> > > >>> > >> > > >>> > On Sep 30, 2015 12:16 AM, "Rainer Gerhards" < >> > > [email protected]> >> > > >>> > wrote: >> > > >>> > >> > > >>> >> It's a long time since I implemented what currently is there. It >> > > should >> > > >>> be >> > > >>> >> relatively solid with probably some minor glitches. It provides >> the >> > > code >> > > >>> >> functionality as far as I remember. >> > > >>> >> >> > > >>> >> Rainer >> > > >>> >> >> > > >>> >> Sent from phone, thus brief. >> > > >>> >> Am 29.09.2015 20:07 schrieb "singh.janmejay" < >> > > [email protected] >> > > >>> >: >> > > >>> >> >> > > >>> >> > Rainer/David, >> > > >>> >> > >> > > >>> >> > Exactly how much of lookup_table functionality is implemented? >> > > >>> >> > >> > > >>> >> > What can I not do with it? (you mentioned something about >> single >> > > table >> > > >>> >> > in this thread, can you please elaborate?). >> > > >>> >> > >> > > >>> >> > On Tue, Mar 31, 2015 at 7:23 PM, Rainer Gerhards >> > > >>> >> > <[email protected]> wrote: >> > > >>> >> > > 2015-03-31 15:46 GMT+02:00 <[email protected]>: >> > > >>> >> > >> Hi, >> > > >>> >> > >> Do you have some experience how large Lookup-tables can be >> > > until >> > > >>> there >> > > >>> >> > are "negative" effects? >> > > >>> >> > >> 2400 entries seems to work fine :) >> > > >>> >> > > >> > > >>> >> > > IIRC the current partial implementation is O(log n), so no >> > > problem. >> > > >>> >> > > >> > > >>> >> > >> >> > > >>> >> > >> And another question, do I loose events, when doing a kill >> -HUP >> > > >>> (for >> > > >>> >> > update of lookup-table)? >> > > >>> >> > >> (e.g. client threads are hard "terminated"...) >> > > >>> >> > > >> > > >>> >> > > *should* not cause any issues. >> > > >>> >> > > >> > > >>> >> > > Rainer >> > > >>> >> > >> >> > > >>> >> > >> best regards >> > > >>> >> > >> Chris >> > > >>> >> > >> >> > > >>> >> > >> >> > > >>> >> > >> >> > > >>> >> > >> Gesendet: Mittwoch, 25. März 2015 um 19:28 Uhr >> > > >>> >> > >> Von: "David Lang" <[email protected]> >> > > >>> >> > >> An: rsyslog-users <[email protected]> >> > > >>> >> > >> Betreff: Re: [rsyslog] Separation of actions based on log >> > > source - >> > > >>> >> with >> > > >>> >> > good performance >> > > >>> >> > >> On Wed, 25 Mar 2015, [email protected] wrote: > >> Hi, > >> > > I was >> > > >>> >> > doing some experiments with the lookup-table. > Looks really >> nice >> > > and >> > > >>> the >> > > >>> >> > performance is promising. > (Unfortunately the evaluation of >> > > "nomatch" >> > > >>> >> > attribute is currently not implemented...) > > Never the >> less: > >> > > My >> > > >>> plan >> > > >>> >> > is, to do diffent actions based on the type of host, mapped >> in the >> > > >>> >> > lookup-list. > For testing purposes, I use alway omfile. > > >> > > >>> >> Unfortunately >> > > >>> >> > it does not work, to change the ruleset based on the >> variable. > >> > > Is >> > > >>> there >> > > >>> >> > any other option or is there any mistake? for omfile you can >> use >> > > the >> > > >>> >> > dynafile approach to use the return variable, for remote >> things >> > > you >> > > >>> would >> > > >>> >> > need to do an if then else approach for performance reasons >> many >> > > of >> > > >>> the >> > > >>> >> > fields in rsyslog do not accept variables. This allows them >> to be >> > > >>> >> > computed/parsed once at startup rather than having to be >> > > evaluated for >> > > >>> >> each >> > > >>> >> > log message. It's a bit of a hassle when you do want to do >> > > something >> > > >>> >> > dynamic, but even in cases where you have some dynamic >> things, you >> > > >>> tend >> > > >>> >> to >> > > >>> >> > have other static things that benefit from the speedup. David >> > > Lang > >> > > >>> *** >> > > >>> >> > syslog.conf *** > lookup_table(name="lookuptable" >> > > >>> >> > file="/etc/rsyslog.lookup") > set $!dst = >> lookup("lookuptable", >> > > >>> >> > $fromhost-ip); > ruleset(name="typea"){ > action(type="omfile" >> > > >>> >> > file="/var/log/file_typea.log") > } > ruleset(name="typea"){ > >> > > >>> >> > action(type="omfile" file="/var/log/file_typeb.log") > } > > # >> > > Change >> > > >>> set >> > > >>> >> > default ruleset, based on sourceip > $DefaultRuleset $!dst > > >> > > >>> >> > module(load="imtcp" KeepAlive="on" KeepAlive.Probes="1" >> > > >>> >> > KeepAlive.Interval="2" KeepAlive.Time="20") > >> input(type="imtcp" >> > > >>> >> > port="7714") > > *** lookup-table *** > { "version":1, >> > > >>> "nomatch":"unk", >> > > >>> >> > "type":"string", > "table":[ {"index":"10.3.5.4", >> "value":"typea" >> > > }, > >> > > >>> >> > {"index":"10.2.2.1", "value":"typea" }, > {"index":"10.0.2.2", >> > > >>> >> > "value":"typeb" }, > {"index":"10.2.2.3", "value":"typeb" } > >> ] > >> > > } > >> > > >>> > > >> > > >>> >> > best regards > Chris > > > > Gesendet: Dienstag, 24. März >> > > 2015 um >> > > >>> >> 17:14 >> > > >>> >> > Uhr > Von: [email protected] > An: >> > > [email protected] > >> > > >>> >> > Betreff: Re: [rsyslog] Separation of actions based on log >> source - >> > > >>> with >> > > >>> >> > good performance > Hi David, > > Thanks sounds great, I will >> try >> > > this >> > > >>> in >> > > >>> >> > the next days :) > > Chris > > > > Gesendet: Montag, 23. >> März >> > > >>> 2015 um >> > > >>> >> > 17:44 Uhr > Von: "David Lang" > An: rsyslog-users > Betreff: >> Re: >> > > >>> >> [rsyslog] >> > > >>> >> > Separation of actions based on log source - with good >> performance >> > > > >> > > >>> This >> > > >>> >> is >> > > >>> >> > the sort of thing that the table lookup functionality was >> designed >> > > >>> for. > >> > > >>> >> > It wasn't fully implemented to the design (funding fell >> through), >> > > but >> > > >>> I >> > > >>> >> > think it works for a single table. > you could use it to do >> the >> > > >>> mapping >> > > >>> >> > from your many hosts to a couple of values and then have your >> > > test be >> > > >>> >> based >> > > >>> >> > on the resulting value. > > David Lang On Mon, 23 Mar 2015 > >> > > [...] > >> > > >>> >> > >> >> > > >>> >> > >> _______________________________________________ >> > > >>> >> > >> rsyslog mailing list >> > > >>> >> > >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > >>> >> > >> http://www.rsyslog.com/professional-services/ >> > > >>> >> > >> What's up with rsyslog? Follow >> https://twitter.com/rgerhards >> > > >>> >> > >> NOTE WELL: This is a PUBLIC mailing list, posts are >> ARCHIVED >> > > by a >> > > >>> >> > myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO >> NOT >> > > >>> POST if >> > > >>> >> > you DON'T LIKE THAT. >> > > >>> >> > > _______________________________________________ >> > > >>> >> > > rsyslog mailing list >> > > >>> >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > >>> >> > > http://www.rsyslog.com/professional-services/ >> > > >>> >> > > What's up with rsyslog? Follow >> https://twitter.com/rgerhards >> > > >>> >> > > NOTE WELL: This is a PUBLIC mailing list, posts are >> ARCHIVED by >> > > a >> > > >>> >> myriad >> > > >>> >> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >> POST >> > > if you >> > > >>> >> > DON'T LIKE THAT. >> > > >>> >> > >> > > >>> >> > >> > > >>> >> > >> > > >>> >> > -- >> > > >>> >> > Regards, >> > > >>> >> > Janmejay >> > > >>> >> > http://codehunk.wordpress.com >> > > >>> >> > _______________________________________________ >> > > >>> >> > rsyslog mailing list >> > > >>> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > >>> >> > http://www.rsyslog.com/professional-services/ >> > > >>> >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > > >>> >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED >> by a >> > > >>> myriad >> > > >>> >> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >> POST >> > > if you >> > > >>> >> > DON'T LIKE THAT. >> > > >>> >> _______________________________________________ >> > > >>> >> rsyslog mailing list >> > > >>> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > >>> >> http://www.rsyslog.com/professional-services/ >> > > >>> >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> > > >>> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by >> a >> > > myriad >> > > >>> >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >> if >> > > you >> > > >>> >> DON'T LIKE THAT. >> > > >>> > _______________________________________________ >> > > >>> > rsyslog mailing list >> > > >>> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > >>> > http://www.rsyslog.com/professional-services/ >> > > >>> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > > >>> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> > > myriad >> > > >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >> you >> > > >>> DON'T LIKE THAT. >> > > >>> _______________________________________________ >> > > >>> rsyslog mailing list >> > > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > >>> http://www.rsyslog.com/professional-services/ >> > > >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >> > > >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> > > myriad >> > > >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >> you >> > > >>> DON'T LIKE THAT. >> > > >> _______________________________________________ >> > > >> rsyslog mailing list >> > > >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > >> http://www.rsyslog.com/professional-services/ >> > > >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> > > >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> > > myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >> if >> > > you DON'T LIKE THAT. >> > > >> _______________________________________________ >> > > >> rsyslog mailing list >> > > >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > >> http://www.rsyslog.com/professional-services/ >> > > >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> > > >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> > > myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >> if >> > > you DON'T LIKE THAT. >> > > > >> > > > >> > > > >> > > > -- >> > > > Regards, >> > > > Janmejay >> > > > http://codehunk.wordpress.com >> > > > _______________________________________________ >> > > > rsyslog mailing list >> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > > http://www.rsyslog.com/professional-services/ >> > > > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> myriad >> > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> > > DON'T LIKE THAT. >> > > _______________________________________________ >> > > rsyslog mailing list >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > http://www.rsyslog.com/professional-services/ >> > > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> myriad >> > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> > > DON'T LIKE THAT. >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com/professional-services/ >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
