Good point. I'll change it. It's not a big change, will do it whenever I can find a few hours in the coming week.
For clarity, we'll return to semantics proposed in original post. I'll make it async too. -- Regards, Janmejay PS: Please blame the typos in this mail on my phone's uncivilized soft keyboard sporting it's not-so-smart-assist technology. On Dec 19, 2015 5:08 AM, "David Lang" <[email protected]> wrote: > I meant to respond to this, but may have forgotten to. > > The table load can be quite lengthy[1], so we wanted it to be > asynchronous. The new two-part approach doesn't have that ability. > > I would have the load process generate a log message when it finishes > saying if it succeeds or fails and you can then detect and take action on > that. > > David Lang > > [1] the use case I was thinking of when I designed the sparse table type > is geoip lookup, using a table such as the one provided by MaxMind (some > for free, some at some costs), this can be a multi GB table that's being > loaded, so we don't want to halt log processing while the table is loading > > > On Fri, 18 Dec 2015, singh.janmejay wrote: > > Date: Fri, 18 Dec 2015 21:12:01 +0530 >> From: singh.janmejay <[email protected]> >> Reply-To: rsyslog-users <[email protected]> >> To: rsyslog-users <[email protected]> >> Subject: Re: [rsyslog] rsyslog next release and lookup-tables >> >> This change is now in PR. >> >> There is a minor change. Actually stub_lookup_table_value statement >> needs to know the lookup table as well, so it takes 2 arguments (not >> 1). >> So, the fragment would look like: >> if (not reload_lookup_table("foo")) then { >> stub_lookup_table_value("foo", "reload_failed") >> } >> >> On Thu, Nov 26, 2015 at 8:16 PM, singh.janmejay >> <[email protected]> wrote: >> >>> Resurrecting this thread for a discussing a deviation from old >>> lookup_table proposal. >>> >>> The PR doesn't include load_lookup_table (a rainerscript statement >>> that is supposed to reload lookup table). >>> >>> Original proposal link: http://www.rsyslog.com/doc/lookup_tables.html >>> >>> Im thinking of implementing it using a function >>> reload_lookup_table(name) and a statement >>> stub_lookup_table_value(value). >>> >>> It allows better composability and leaves room for better backward >>> compatibility. >>> >>> Its possible to get the same effect as the load_lookup_table(...) gets >>> in the proposed form using: >>> >>> if (not reload_lookup_table("foo")) then { >>> stub_lookup_table_value("reload_failed") >>> } >>> >>> However, it can be composed with other rainerscript controls. Eg. one >>> can log failiures to reload table to a separate log-file, increment a >>> metric which reports this failure, or send it to a remote destination. >>> >>> if (not reload_lookup_table("foo")) then { >>> action(type="omfile"...) >>> } >>> >>> Functionally, this is all that matters. >>> >>> Have a look at PR(comments) for details (backward compatibility issue >>> etc). >>> >>> >>> >>> On Tue, Nov 3, 2015 at 7:37 PM, singh.janmejay <[email protected]> >>> wrote: >>> >>>> Will cross-reference in the kill-feature PR. >>>> >>>> On Tue, Nov 3, 2015 at 7:37 PM, singh.janmejay < >>>> [email protected]> wrote: >>>> >>>>> Sorry for the delay. Here is the PR: >>>>> https://github.com/rsyslog/rsyslog/pull/578 >>>>> >>>>> On Tue, Nov 3, 2015 at 6:02 PM, Rainer Gerhards >>>>> <[email protected]> wrote: >>>>> >>>>>> 2015-11-03 12:54 GMT+01:00 <[email protected]>: >>>>>> >>>>>>> Hello, >>>>>>> As far as I see, today is the release data of the next rsyslog >>>>>>> version. >>>>>>> I did not see any changes about the lookup diffs, Janmejay promised, >>>>>>> so I'm quite nerverous that the new release will no longer contain the >>>>>>> lookup-tables. >>>>>>> >>>>>> >>>>>> Please have a look here for status updates: >>>>>> https://github.com/rsyslog/rsyslog/pull/544 >>>>>> >>>>>> In short: I won't remove it this release, as I have no longer been >>>>>> tortured with CVEs and I think we can let it stand as is - NOT >>>>>> officially existing - for a bit longer. I hope we can merge something >>>>>> solid into the december or january relaese. >>>>>> >>>>>> Rainer >>>>>> >>>>>>> >>>>>>> Please do not remove it, as it works fine (after the last patch) and >>>>>>> I (and possibly others) use it already in production. >>>>>>> If it is needed I will help to document the functionality as it >>>>>>> exists right now. >>>>>>> >>>>>>> Best regards, >>>>>>> Christopher >>>>>>> >>>>>>> >>>>>>> >>>>>>> ------------------------------------------------------------------------------------------------------------ >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Hello, >>>>>>> I have never heared such a nonsense. >>>>>>> Actually the number of applications that does not include features >>>>>>> that are not official documented shoult be extremly limited. >>>>>>> >>>>>>> The functionality is really usefull and already in big landscapes >>>>>>> productive. >>>>>>> Please, please do NOT remove the lookup-table from the main branch. >>>>>>> The functionaltiy works fine, I'm using this since march and I did >>>>>>> not have any issue since the latest patch of janmejay. >>>>>>> >>>>>>> Even the "concept" is not fully implemented (e.g. smaller things >>>>>>> like nomatch) the main part works fine. >>>>>>> >>>>>>> >>>>>>> My suggestion would be to document everything which is currently >>>>>>> implemented and keep the "conceptual documentation" as it is. >>>>>>> So the Maintainer should no longer have an issue with it. >>>>>>> >>>>>>> >>>>>>> If the main issue it the time to document the already implemented >>>>>>> features, I can create a patch. >>>>>>> >>>>>>> >>>>>>> Chris >>>>>>> >>>>>>> >>>>>>> >>>>>>> Gesendet: Dienstag, 06. Oktober 2015 um 07:36 Uhr >>>>>>>> Von: "David Lang" <[email protected]> >>>>>>>> An: rsyslog-users <[email protected]> >>>>>>>> Betreff: Re: [rsyslog] Separation of actions based on log source - >>>>>>>> with good performance >>>>>>>> >>>>>>>> a CVE for something that requires manually enabling an experimental >>>>>>>> feature??? >>>>>>>> >>>>>>>> it would be one thing if a default config had the problem, or if it >>>>>>>> was >>>>>>>> something entirely dependent on remote data. >>>>>>>> >>>>>>>> I would be very tempted to respond to the CVE with "don't enable >>>>>>>> this incomplete >>>>>>>> feature" as the solution. It's very common for incomplete features >>>>>>>> to be >>>>>>>> included in released versions >>>>>>>> >>>>>>>> grumble, we have enough real bugs to worry about. >>>>>>>> >>>>>>>> David Lang >>>>>>>> >>>>>>>> On Tue, 6 Oct 2015, Rainer Gerhards wrote: >>>>>>>> >>>>>>>> > Date: Tue, 6 Oct 2015 07:15:31 +0200 >>>>>>>> > From: Rainer Gerhards <[email protected]> >>>>>>>> > Reply-To: rsyslog-users <[email protected]> >>>>>>>> > To: rsyslog-users <[email protected]> >>>>>>>> > Subject: Re: [rsyslog] Separation of actions based on log source >>>>>>>> - with good >>>>>>>> > performance >>>>>>>> > >>>>>>>> > Sorry, folks, good intent always seems to find someone who turns >>>>>>>> it >>>>>>>> > into negative. I was yesterday contacted by a distro maintainer >>>>>>>> who >>>>>>>> > wants to turn this bug in the officially non-existant lookup table >>>>>>>> > feature into a CVE and insists that it is a vuln even after the >>>>>>>> > argument that the feature never oficially existed. >>>>>>>> > >>>>>>>> > It looks like it was a bad idea to merge potentially useful yet >>>>>>>> > incomplete code into the main branch (and documenting it to be not >>>>>>>> > present). It looks like I need to re-think my stance on >>>>>>>> experimental >>>>>>>> > features. >>>>>>>> > >>>>>>>> > Anyhow, I really don't want to support the argument that something >>>>>>>> > non-existing can be a CVE. As such, I will create a new >>>>>>>> > master-insecure branch, which will be a clone of the current >>>>>>>> master >>>>>>>> > branch. Then I'll remove the lookup table code, so that the code >>>>>>>> base >>>>>>>> > matches the documentation. I really don't want to create a general >>>>>>>> > principle here that we need to create CVEs (and patched) for >>>>>>>> something >>>>>>>> > that was just added as a convenience for a handful of folks who >>>>>>>> were >>>>>>>> > ready to take a risk. >>>>>>>> > >>>>>>>> > If there is sufficient interest, we can consider officially adding >>>>>>>> > this feature to the January 8.15 release iff it is ready by then. >>>>>>>> > @janmejay: please let me know if you would like to continue with >>>>>>>> your >>>>>>>> > work on lookup tables under this new situation. >>>>>>>> > >>>>>>>> > As soon as I have time, I'll check what else needs to be removed. >>>>>>>> Not >>>>>>>> > sure about the ./contributed branch, because the project cannot >>>>>>>> > guarantee at all this is bug-free. It's documented to be so, but >>>>>>>> if >>>>>>>> > that is not sufficient, it should probably live only in the >>>>>>>> > master-insecure branch. >>>>>>>> > >>>>>>>> > Rainer >>>>>>>> > >>>>>>>> > 2015-10-02 17:29 GMT+02:00 singh.janmejay < >>>>>>>> [email protected]>: >>>>>>>> >> As of now it returns empty string for no-match. I guess we >>>>>>>> should go ahead >>>>>>>> >> with it in its current form. We can add default value any time >>>>>>>> later >>>>>>>> >> without breaking compatibility(it'd default to ""). >>>>>>>> >> >>>>>>>> >> I'll add a test for multiple tables on Monday. >>>>>>>> >> >>>>>>>> >> On Fri, Oct 2, 2015, 7:16 PM <[email protected]> wrote: >>>>>>>> >> >>>>>>>> >>> Hi, >>>>>>>> >>> No, I didn't. I even didn't realize the patch. >>>>>>>> >>> >>>>>>>> >>> It seems to be exactly the related issue. So I don't expect any >>>>>>>> further >>>>>>>> >>> issues. >>>>>>>> >>> I will use the new version on 2 systems. If there is any other >>>>>>>> issue, I >>>>>>>> >>> will let you know. >>>>>>>> >>> Release data for next rsyslog version is quite far so enough >>>>>>>> time to >>>>>>>> >>> test... ;) >>>>>>>> >>> >>>>>>>> >>> The missing implementation of "nomatch" (default) entry as >>>>>>>> described at >>>>>>>> >>> http://www.rsyslog.com/doc/lookup_tables.html >>>>>>>> >>> would from my opinion require changes: >>>>>>>> >>> >>>>>>>> >>> Arround line 132 of lookup.c file (save of value) >>>>>>>> >>> Arround line 243 of lookup.c file (search in lookuptable fails, >>>>>>>> so return >>>>>>>> >>> nomatch value. >>>>>>>> >>> >>>>>>>> >>> >>>>>>>> >>> regards >>>>>>>> >>> Chris >>>>>>>> >>> >>>>>>>> >>> >>>>>>>> >>> > Gesendet: Donnerstag, 01. Oktober 2015 um 16:57 Uhr >>>>>>>> >>> > Von: "singh.janmejay" <[email protected]> >>>>>>>> >>> > An: rsyslog-users <[email protected]> >>>>>>>> >>> > Betreff: Re: [rsyslog] Separation of actions based on log >>>>>>>> source - with >>>>>>>> >>> good performance >>>>>>>> >>> > >>>>>>>> >>> > Yes, if you build off master, that problem should go away (if >>>>>>>> it was due >>>>>>>> >>> to >>>>>>>> >>> > lookup-table). >>>>>>>> >>> > >>>>>>>> >>> > On Thu, Oct 1, 2015, 7:00 PM Rainer Gerhards < >>>>>>>> [email protected]> >>>>>>>> >>> > wrote: >>>>>>>> >>> > >>>>>>>> >>> > > 2015-10-01 15:14 GMT+02:00 singh.janmejay < >>>>>>>> [email protected]>: >>>>>>>> >>> > > > If you can share output of all thread backtrace we can >>>>>>>> confirm if >>>>>>>> >>> this >>>>>>>> >>> > > > is the cause. >>>>>>>> >>> > > >>>>>>>> >>> > > let's first double-check: Christopher, did you use >>>>>>>> yesterday evening's >>>>>>>> >>> > > master branch? Because that contains a patch from Janmejay >>>>>>>> that I >>>>>>>> >>> > > think causes the problem for you. Or am I wrong, Janmejay? >>>>>>>> >>> > > >>>>>>>> >>> > > Rainer >>>>>>>> >>> > > > >>>>>>>> >>> > > > On Thu, Oct 1, 2015 at 2:30 PM, < >>>>>>>> [email protected]> wrote: >>>>>>>> >>> > > >> Hi, >>>>>>>> >>> > > >> Ups I was not detailed enough. >>>>>>>> >>> > > >> The problem with rsyslog-die does not always occur. But >>>>>>>> sometimes >>>>>>>> >>> > > unexpectedly. >>>>>>>> >>> > > >> In my environments the files grow or reduce sometimes, >>>>>>>> so maybe this >>>>>>>> >>> > > has something do with it (or the processing delay). >>>>>>>> >>> > > >> >>>>>>>> >>> > > >> regards >>>>>>>> >>> > > >> Chris >>>>>>>> >>> > > >> >>>>>>>> >>> > > >> >>>>>>>> >>> > > >> -----Ursprüngliche Nachricht----- >>>>>>>> >>> > > >> Gesendet: Donnerstag, 01 Oktober 2015 um 10:57:37 Uhr >>>>>>>> >>> > > >> Von: [email protected] >>>>>>>> >>> > > >> An: "singh.janmejay" >>>>>>>> >>> > > >> <[email protected]>,rsyslog-users >>>>>>>> < >>>>>>>> >>> > > [email protected]> >>>>>>>> >>> > > >> Betreff: Re: [rsyslog] Separation of actions based on >>>>>>>> log source - >>>>>>>> >>> with >>>>>>>> >>> > > good performance >>>>>>>> >>> > > >> Hi, >>>>>>>> >>> > > >> For my opinion it is really good to support looku-tables >>>>>>>> official. >>>>>>>> >>> > > >> Thanks for the work on the implementation David & Rainer. >>>>>>>> >>> > > >> >>>>>>>> >>> > > >> I have some experiences using lookup-Tables with > 2500 >>>>>>>> Entries. >>>>>>>> >>> > > >> >>>>>>>> >>> > > >> There are 2 open issues: >>>>>>>> >>> > > >> >>>>>>>> >>> > > >> 1. There is a bug when sending SIGHUP and reprocessing >>>>>>>> big lists, >>>>>>>> >>> which >>>>>>>> >>> > > leads to die of rsyslogd. >>>>>>>> >>> > > >> I spend some time to identify this bug, unfortunately >>>>>>>> I'm still not >>>>>>>> >>> > > able to find the exact reason. >>>>>>>> >>> > > >> The problem seems to occur not directly after sending >>>>>>>> SIGHUP, but >>>>>>>> >>> > > later. Maybe this has something to do with Queues. >>>>>>>> >>> > > >> >>>>>>>> >>> > > >> 2. The "default" Value is not implemented. This should >>>>>>>> be mentioned >>>>>>>> >>> in >>>>>>>> >>> > > the documentation or implemented. >>>>>>>> >>> > > >> I guess its quite less work, but I'm not sure how soon I >>>>>>>> find the >>>>>>>> >>> time >>>>>>>> >>> > > to do all the things arround the pure developement... ;) >>>>>>>> >>> > > >> >>>>>>>> >>> > > >> >>>>>>>> >>> > > >> >>>>>>>> >>> > > >> regards >>>>>>>> >>> > > >> Chris >>>>>>>> >>> > > >> >>>>>>>> >>> > > >> -----Ursprüngliche Nachricht----- >>>>>>>> >>> > > >> Gesendet: Donnerstag, 01 Oktober 2015 um 09:41:26 Uhr >>>>>>>> >>> > > >> Von: "singh.janmejay" <[email protected]> >>>>>>>> >>> > > >> An: rsyslog-users <[email protected]> >>>>>>>> >>> > > >> Betreff: Re: [rsyslog] Separation of actions based on >>>>>>>> log source - >>>>>>>> >>> with >>>>>>>> >>> > > good performance >>>>>>>> >>> > > >> OK, allow me a few days, I'll add one more test for >>>>>>>> multiple tables. >>>>>>>> >>> > > Will >>>>>>>> >>> > > >> make the doc change after that. >>>>>>>> >>> > > >> >>>>>>>> >>> > > >> -- >>>>>>>> >>> > > >> Regards, >>>>>>>> >>> > > >> Janmejay >>>>>>>> >>> > > >> >>>>>>>> >>> > > >> PS: Please blame the typos in this mail on my phone's >>>>>>>> uncivilized >>>>>>>> >>> soft >>>>>>>> >>> > > >> keyboard sporting it's not-so-smart-assist technology. >>>>>>>> >>> > > >> >>>>>>>> >>> > > >> On Oct 1, 2015 12:29 PM, "Rainer Gerhards" < >>>>>>>> >>> [email protected]> >>>>>>>> >>> > > wrote: >>>>>>>> >>> > > >> >>>>>>>> >>> > > >>> 2015-09-29 20:58 GMT+02:00 singh.janmejay < >>>>>>>> >>> [email protected]>: >>>>>>>> >>> > > >>> > Sweet, plan on playing with it tomorrow. >>>>>>>> >>> > > >>> >>>>>>>> >>> > > >>> If you have verified that the current functionality >>>>>>>> works fine >>>>>>>> >>> after >>>>>>>> >>> > > >>> your patch, I wouldn't object if you modify the doc to >>>>>>>> tell the >>>>>>>> >>> world >>>>>>>> >>> > > >>> that this part of lookup tables is now officially >>>>>>>> supported. we >>>>>>>> >>> could >>>>>>>> >>> > > >>> release with 8.14. I think what currently exists is >>>>>>>> already pretty >>>>>>>> >>> > > >>> useful and if we feel confident enough it works, we >>>>>>>> should release >>>>>>>> >>> it. >>>>>>>> >>> > > >>> >>>>>>>> >>> > > >>> Rainer >>>>>>>> >>> > > >>> > >>>>>>>> >>> > > >>> > -- >>>>>>>> >>> > > >>> > Regards, >>>>>>>> >>> > > >>> > Janmejay >>>>>>>> >>> > > >>> > >>>>>>>> >>> > > >>> > PS: Please blame the typos in this mail on my phone's >>>>>>>> uncivilized >>>>>>>> >>> > > soft >>>>>>>> >>> > > >>> > keyboard sporting it's not-so-smart-assist technology. >>>>>>>> >>> > > >>> > >>>>>>>> >>> > > >>> > On Sep 30, 2015 12:16 AM, "Rainer Gerhards" < >>>>>>>> >>> > > [email protected]> >>>>>>>> >>> > > >>> > wrote: >>>>>>>> >>> > > >>> > >>>>>>>> >>> > > >>> >> It's a long time since I implemented what currently >>>>>>>> is there. It >>>>>>>> >>> > > should >>>>>>>> >>> > > >>> be >>>>>>>> >>> > > >>> >> relatively solid with probably some minor glitches. >>>>>>>> It provides >>>>>>>> >>> the >>>>>>>> >>> > > code >>>>>>>> >>> > > >>> >> functionality as far as I remember. >>>>>>>> >>> > > >>> >> >>>>>>>> >>> > > >>> >> Rainer >>>>>>>> >>> > > >>> >> >>>>>>>> >>> > > >>> >> Sent from phone, thus brief. >>>>>>>> >>> > > >>> >> Am 29.09.2015 20:07 schrieb "singh.janmejay" < >>>>>>>> >>> > > [email protected] >>>>>>>> >>> > > >>> >: >>>>>>>> >>> > > >>> >> >>>>>>>> >>> > > >>> >> > Rainer/David, >>>>>>>> >>> > > >>> >> > >>>>>>>> >>> > > >>> >> > Exactly how much of lookup_table functionality is >>>>>>>> implemented? >>>>>>>> >>> > > >>> >> > >>>>>>>> >>> > > >>> >> > What can I not do with it? (you mentioned >>>>>>>> something about >>>>>>>> >>> single >>>>>>>> >>> > > table >>>>>>>> >>> > > >>> >> > in this thread, can you please elaborate?). >>>>>>>> >>> > > >>> >> > >>>>>>>> >>> > > >>> >> > On Tue, Mar 31, 2015 at 7:23 PM, Rainer Gerhards >>>>>>>> >>> > > >>> >> > <[email protected]> wrote: >>>>>>>> >>> > > >>> >> > > 2015-03-31 15:46 GMT+02:00 < >>>>>>>> [email protected]>: >>>>>>>> >>> > > >>> >> > >> Hi, >>>>>>>> >>> > > >>> >> > >> Do you have some experience how large >>>>>>>> Lookup-tables can be >>>>>>>> >>> > > until >>>>>>>> >>> > > >>> there >>>>>>>> >>> > > >>> >> > are "negative" effects? >>>>>>>> >>> > > >>> >> > >> 2400 entries seems to work fine :) >>>>>>>> >>> > > >>> >> > > >>>>>>>> >>> > > >>> >> > > IIRC the current partial implementation is O(log >>>>>>>> n), so no >>>>>>>> >>> > > problem. >>>>>>>> >>> > > >>> >> > > >>>>>>>> >>> > > >>> >> > >> >>>>>>>> >>> > > >>> >> > >> And another question, do I loose events, when >>>>>>>> doing a kill >>>>>>>> >>> -HUP >>>>>>>> >>> > > >>> (for >>>>>>>> >>> > > >>> >> > update of lookup-table)? >>>>>>>> >>> > > >>> >> > >> (e.g. client threads are hard "terminated"...) >>>>>>>> >>> > > >>> >> > > >>>>>>>> >>> > > >>> >> > > *should* not cause any issues. >>>>>>>> >>> > > >>> >> > > >>>>>>>> >>> > > >>> >> > > Rainer >>>>>>>> >>> > > >>> >> > >> >>>>>>>> >>> > > >>> >> > >> best regards >>>>>>>> >>> > > >>> >> > >> Chris >>>>>>>> >>> > > >>> >> > >> >>>>>>>> >>> > > >>> >> > >> >>>>>>>> >>> > > >>> >> > >> >>>>>>>> >>> > > >>> >> > >> Gesendet: Mittwoch, 25. März 2015 um 19:28 Uhr >>>>>>>> >>> > > >>> >> > >> Von: "David Lang" <[email protected]> >>>>>>>> >>> > > >>> >> > >> An: rsyslog-users <[email protected]> >>>>>>>> >>> > > >>> >> > >> Betreff: Re: [rsyslog] Separation of actions >>>>>>>> based on log >>>>>>>> >>> > > source - >>>>>>>> >>> > > >>> >> with >>>>>>>> >>> > > >>> >> > good performance >>>>>>>> >>> > > >>> >> > >> On Wed, 25 Mar 2015, [email protected] >>>>>>>> wrote: > >>>>>>>> >>> Hi, > >>>>>>>> >>> > > I was >>>>>>>> >>> > > >>> >> > doing some experiments with the lookup-table. > >>>>>>>> Looks really >>>>>>>> >>> nice >>>>>>>> >>> > > and >>>>>>>> >>> > > >>> the >>>>>>>> >>> > > >>> >> > performance is promising. > (Unfortunately the >>>>>>>> evaluation of >>>>>>>> >>> > > "nomatch" >>>>>>>> >>> > > >>> >> > attribute is currently not implemented...) > > >>>>>>>> Never the >>>>>>>> >>> less: > >>>>>>>> >>> > > My >>>>>>>> >>> > > >>> plan >>>>>>>> >>> > > >>> >> > is, to do diffent actions based on the type of >>>>>>>> host, mapped >>>>>>>> >>> in the >>>>>>>> >>> > > >>> >> > lookup-list. > For testing purposes, I use alway >>>>>>>> omfile. > > >>>>>>>> >>> > > >>> >> Unfortunately >>>>>>>> >>> > > >>> >> > it does not work, to change the ruleset based on >>>>>>>> the >>>>>>>> >>> variable. > >>>>>>>> >>> > > Is >>>>>>>> >>> > > >>> there >>>>>>>> >>> > > >>> >> > any other option or is there any mistake? for >>>>>>>> omfile you can >>>>>>>> >>> use >>>>>>>> >>> > > the >>>>>>>> >>> > > >>> >> > dynafile approach to use the return variable, for >>>>>>>> remote >>>>>>>> >>> things >>>>>>>> >>> > > you >>>>>>>> >>> > > >>> would >>>>>>>> >>> > > >>> >> > need to do an if then else approach for >>>>>>>> performance reasons >>>>>>>> >>> many >>>>>>>> >>> > > of >>>>>>>> >>> > > >>> the >>>>>>>> >>> > > >>> >> > fields in rsyslog do not accept variables. This >>>>>>>> allows them >>>>>>>> >>> to be >>>>>>>> >>> > > >>> >> > computed/parsed once at startup rather than having >>>>>>>> to be >>>>>>>> >>> > > evaluated for >>>>>>>> >>> > > >>> >> each >>>>>>>> >>> > > >>> >> > log message. It's a bit of a hassle when you do >>>>>>>> want to do >>>>>>>> >>> > > something >>>>>>>> >>> > > >>> >> > dynamic, but even in cases where you have some >>>>>>>> dynamic >>>>>>>> >>> things, you >>>>>>>> >>> > > >>> tend >>>>>>>> >>> > > >>> >> to >>>>>>>> >>> > > >>> >> > have other static things that benefit from the >>>>>>>> speedup. David >>>>>>>> >>> > > Lang > >>>>>>>> >>> > > >>> *** >>>>>>>> >>> > > >>> >> > syslog.conf *** > lookup_table(name="lookuptable" >>>>>>>> >>> > > >>> >> > file="/etc/rsyslog.lookup") > set $!dst = >>>>>>>> >>> lookup("lookuptable", >>>>>>>> >>> > > >>> >> > $fromhost-ip); > ruleset(name="typea"){ > >>>>>>>> action(type="omfile" >>>>>>>> >>> > > >>> >> > file="/var/log/file_typea.log") > } > >>>>>>>> ruleset(name="typea"){ > >>>>>>>> >>> > > >>> >> > action(type="omfile" >>>>>>>> file="/var/log/file_typeb.log") > } > > # >>>>>>>> >>> > > Change >>>>>>>> >>> > > >>> set >>>>>>>> >>> > > >>> >> > default ruleset, based on sourceip > >>>>>>>> $DefaultRuleset $!dst > > >>>>>>>> >>> > > >>> >> > module(load="imtcp" KeepAlive="on" >>>>>>>> KeepAlive.Probes="1" >>>>>>>> >>> > > >>> >> > KeepAlive.Interval="2" KeepAlive.Time="20") > >>>>>>>> >>> input(type="imtcp" >>>>>>>> >>> > > >>> >> > port="7714") > > *** lookup-table *** > { >>>>>>>> "version":1, >>>>>>>> >>> > > >>> "nomatch":"unk", >>>>>>>> >>> > > >>> >> > "type":"string", > "table":[ {"index":"10.3.5.4", >>>>>>>> >>> "value":"typea" >>>>>>>> >>> > > }, > >>>>>>>> >>> > > >>> >> > {"index":"10.2.2.1", "value":"typea" }, > >>>>>>>> {"index":"10.0.2.2", >>>>>>>> >>> > > >>> >> > "value":"typeb" }, > {"index":"10.2.2.3", >>>>>>>> "value":"typeb" } > >>>>>>>> >>> ] > >>>>>>>> >>> > > } > >>>>>>>> >>> > > >>> > > >>>>>>>> >>> > > >>> >> > best regards > Chris > > > > Gesendet: >>>>>>>> Dienstag, 24. März >>>>>>>> >>> > > 2015 um >>>>>>>> >>> > > >>> >> 17:14 >>>>>>>> >>> > > >>> >> > Uhr > Von: [email protected] > An: >>>>>>>> >>> > > [email protected] > >>>>>>>> >>> > > >>> >> > Betreff: Re: [rsyslog] Separation of actions based >>>>>>>> on log >>>>>>>> >>> source - >>>>>>>> >>> > > >>> with >>>>>>>> >>> > > >>> >> > good performance > Hi David, > > Thanks sounds >>>>>>>> great, I will >>>>>>>> >>> try >>>>>>>> >>> > > this >>>>>>>> >>> > > >>> in >>>>>>>> >>> > > >>> >> > the next days :) > > Chris > > > > Gesendet: >>>>>>>> Montag, 23. >>>>>>>> >>> März >>>>>>>> >>> > > >>> 2015 um >>>>>>>> >>> > > >>> >> > 17:44 Uhr > Von: "David Lang" > An: rsyslog-users >>>>>>>> > Betreff: >>>>>>>> >>> Re: >>>>>>>> >>> > > >>> >> [rsyslog] >>>>>>>> >>> > > >>> >> > Separation of actions based on log source - with >>>>>>>> good >>>>>>>> >>> performance >>>>>>>> >>> > > > >>>>>>>> >>> > > >>> This >>>>>>>> >>> > > >>> >> is >>>>>>>> >>> > > >>> >> > the sort of thing that the table lookup >>>>>>>> functionality was >>>>>>>> >>> designed >>>>>>>> >>> > > >>> for. > >>>>>>>> >>> > > >>> >> > It wasn't fully implemented to the design (funding >>>>>>>> fell >>>>>>>> >>> through), >>>>>>>> >>> > > but >>>>>>>> >>> > > >>> I >>>>>>>> >>> > > >>> >> > think it works for a single table. > you could use >>>>>>>> it to do >>>>>>>> >>> the >>>>>>>> >>> > > >>> mapping >>>>>>>> >>> > > >>> >> > from your many hosts to a couple of values and >>>>>>>> then have your >>>>>>>> >>> > > test be >>>>>>>> >>> > > >>> >> based >>>>>>>> >>> > > >>> >> > on the resulting value. > > David Lang On Mon, 23 >>>>>>>> Mar 2015 > >>>>>>>> >>> > > [...] > >>>>>>>> >>> > > >>> >> > >> >>>>>>>> >>> > > >>> >> > >> _______________________________________________ >>>>>>>> >>> > > >>> >> > >> rsyslog mailing list >>>>>>>> >>> > > >>> >> > >> >>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>> >>> > > >>> >> > >> http://www.rsyslog.com/professional-services/ >>>>>>>> >>> > > >>> >> > >> What's up with rsyslog? Follow >>>>>>>> >>> https://twitter.com/rgerhards >>>>>>>> >>> > > >>> >> > >> NOTE WELL: This is a PUBLIC mailing list, posts >>>>>>>> are >>>>>>>> >>> ARCHIVED >>>>>>>> >>> > > by a >>>>>>>> >>> > > >>> >> > myriad of sites beyond our control. PLEASE >>>>>>>> UNSUBSCRIBE and DO >>>>>>>> >>> NOT >>>>>>>> >>> > > >>> POST if >>>>>>>> >>> > > >>> >> > you DON'T LIKE THAT. >>>>>>>> >>> > > >>> >> > > _______________________________________________ >>>>>>>> >>> > > >>> >> > > rsyslog mailing list >>>>>>>> >>> > > >>> >> > > >>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>> >>> > > >>> >> > > http://www.rsyslog.com/professional-services/ >>>>>>>> >>> > > >>> >> > > What's up with rsyslog? Follow >>>>>>>> >>> https://twitter.com/rgerhards >>>>>>>> >>> > > >>> >> > > NOTE WELL: This is a PUBLIC mailing list, posts >>>>>>>> are >>>>>>>> >>> ARCHIVED by >>>>>>>> >>> > > a >>>>>>>> >>> > > >>> >> myriad >>>>>>>> >>> > > >>> >> > of sites beyond our control. PLEASE UNSUBSCRIBE >>>>>>>> and DO NOT >>>>>>>> >>> POST >>>>>>>> >>> > > if you >>>>>>>> >>> > > >>> >> > DON'T LIKE THAT. >>>>>>>> >>> > > >>> >> > >>>>>>>> >>> > > >>> >> > >>>>>>>> >>> > > >>> >> > >>>>>>>> >>> > > >>> >> > -- >>>>>>>> >>> > > >>> >> > Regards, >>>>>>>> >>> > > >>> >> > Janmejay >>>>>>>> >>> > > >>> >> > http://codehunk.wordpress.com >>>>>>>> >>> > > >>> >> > _______________________________________________ >>>>>>>> >>> > > >>> >> > rsyslog mailing list >>>>>>>> >>> > > >>> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>> >>> > > >>> >> > http://www.rsyslog.com/professional-services/ >>>>>>>> >>> > > >>> >> > What's up with rsyslog? Follow >>>>>>>> https://twitter.com/rgerhards >>>>>>>> >>> > > >>> >> > NOTE WELL: This is a PUBLIC mailing list, posts >>>>>>>> are ARCHIVED >>>>>>>> >>> by a >>>>>>>> >>> > > >>> myriad >>>>>>>> >>> > > >>> >> > of sites beyond our control. PLEASE UNSUBSCRIBE >>>>>>>> and DO NOT >>>>>>>> >>> POST >>>>>>>> >>> > > if you >>>>>>>> >>> > > >>> >> > DON'T LIKE THAT. >>>>>>>> >>> > > >>> >> _______________________________________________ >>>>>>>> >>> > > >>> >> rsyslog mailing list >>>>>>>> >>> > > >>> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>> >>> > > >>> >> http://www.rsyslog.com/professional-services/ >>>>>>>> >>> > > >>> >> What's up with rsyslog? Follow >>>>>>>> https://twitter.com/rgerhards >>>>>>>> >>> > > >>> >> NOTE WELL: This is a PUBLIC mailing list, posts are >>>>>>>> ARCHIVED by >>>>>>>> >>> a >>>>>>>> >>> > > myriad >>>>>>>> >>> > > >>> >> of sites beyond our control. PLEASE UNSUBSCRIBE and >>>>>>>> DO NOT POST >>>>>>>> >>> if >>>>>>>> >>> > > you >>>>>>>> >>> > > >>> >> DON'T LIKE THAT. >>>>>>>> >>> > > >>> > _______________________________________________ >>>>>>>> >>> > > >>> > rsyslog mailing list >>>>>>>> >>> > > >>> > http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>> >>> > > >>> > http://www.rsyslog.com/professional-services/ >>>>>>>> >>> > > >>> > What's up with rsyslog? Follow >>>>>>>> https://twitter.com/rgerhards >>>>>>>> >>> > > >>> > NOTE WELL: This is a PUBLIC mailing list, posts are >>>>>>>> ARCHIVED by a >>>>>>>> >>> > > myriad >>>>>>>> >>> > > >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO >>>>>>>> NOT POST if >>>>>>>> >>> you >>>>>>>> >>> > > >>> DON'T LIKE THAT. >>>>>>>> >>> > > >>> _______________________________________________ >>>>>>>> >>> > > >>> rsyslog mailing list >>>>>>>> >>> > > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>> >>> > > >>> http://www.rsyslog.com/professional-services/ >>>>>>>> >>> > > >>> What's up with rsyslog? Follow >>>>>>>> https://twitter.com/rgerhards >>>>>>>> >>> > > >>> NOTE WELL: This is a PUBLIC mailing list, posts are >>>>>>>> ARCHIVED by a >>>>>>>> >>> > > myriad >>>>>>>> >>> > > >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO >>>>>>>> NOT POST if >>>>>>>> >>> you >>>>>>>> >>> > > >>> DON'T LIKE THAT. >>>>>>>> >>> > > >> _______________________________________________ >>>>>>>> >>> > > >> rsyslog mailing list >>>>>>>> >>> > > >> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>> >>> > > >> http://www.rsyslog.com/professional-services/ >>>>>>>> >>> > > >> What's up with rsyslog? Follow >>>>>>>> https://twitter.com/rgerhards >>>>>>>> >>> > > >> NOTE WELL: This is a PUBLIC mailing list, posts are >>>>>>>> ARCHIVED by a >>>>>>>> >>> > > myriad of sites beyond our control. PLEASE UNSUBSCRIBE and >>>>>>>> DO NOT POST >>>>>>>> >>> if >>>>>>>> >>> > > you DON'T LIKE THAT. >>>>>>>> >>> > > >> _______________________________________________ >>>>>>>> >>> > > >> rsyslog mailing list >>>>>>>> >>> > > >> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>> >>> > > >> http://www.rsyslog.com/professional-services/ >>>>>>>> >>> > > >> What's up with rsyslog? Follow >>>>>>>> https://twitter.com/rgerhards >>>>>>>> >>> > > >> NOTE WELL: This is a PUBLIC mailing list, posts are >>>>>>>> ARCHIVED by a >>>>>>>> >>> > > myriad of sites beyond our control. PLEASE UNSUBSCRIBE and >>>>>>>> DO NOT POST >>>>>>>> >>> if >>>>>>>> >>> > > you DON'T LIKE THAT. >>>>>>>> >>> > > > >>>>>>>> >>> > > > >>>>>>>> >>> > > > >>>>>>>> >>> > > > -- >>>>>>>> >>> > > > Regards, >>>>>>>> >>> > > > Janmejay >>>>>>>> >>> > > > http://codehunk.wordpress.com >>>>>>>> >>> > > > _______________________________________________ >>>>>>>> >>> > > > rsyslog mailing list >>>>>>>> >>> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>> >>> > > > http://www.rsyslog.com/professional-services/ >>>>>>>> >>> > > > What's up with rsyslog? Follow >>>>>>>> https://twitter.com/rgerhards >>>>>>>> >>> > > > NOTE WELL: This is a PUBLIC mailing list, posts are >>>>>>>> ARCHIVED by a >>>>>>>> >>> myriad >>>>>>>> >>> > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >>>>>>>> POST if you >>>>>>>> >>> > > DON'T LIKE THAT. >>>>>>>> >>> > > _______________________________________________ >>>>>>>> >>> > > rsyslog mailing list >>>>>>>> >>> > > http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>> >>> > > http://www.rsyslog.com/professional-services/ >>>>>>>> >>> > > What's up with rsyslog? Follow >>>>>>>> https://twitter.com/rgerhards >>>>>>>> >>> > > NOTE WELL: This is a PUBLIC mailing list, posts are >>>>>>>> ARCHIVED by a >>>>>>>> >>> myriad >>>>>>>> >>> > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >>>>>>>> POST if you >>>>>>>> >>> > > DON'T LIKE THAT. >>>>>>>> >>> > _______________________________________________ >>>>>>>> >>> > rsyslog mailing list >>>>>>>> >>> > http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>> >>> > http://www.rsyslog.com/professional-services/ >>>>>>>> >>> > What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>> >>> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED >>>>>>>> by a myriad >>>>>>>> >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >>>>>>>> if you >>>>>>>> >>> DON'T LIKE THAT. >>>>>>>> >>> _______________________________________________ >>>>>>>> >>> rsyslog mailing list >>>>>>>> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>> >>> http://www.rsyslog.com/professional-services/ >>>>>>>> >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>> >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by >>>>>>>> a myriad >>>>>>>> >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >>>>>>>> if you >>>>>>>> >>> DON'T LIKE THAT. >>>>>>>> >> _______________________________________________ >>>>>>>> >> rsyslog mailing list >>>>>>>> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>> >> http://www.rsyslog.com/professional-services/ >>>>>>>> >> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by >>>>>>>> a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >>>>>>>> POST if >>>>>>>> you DON'T LIKE THAT. >>>>>>>> > _______________________________________________ >>>>>>>> > rsyslog mailing list >>>>>>>> > http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>> > http://www.rsyslog.com/professional-services/ >>>>>>>> > What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >>>>>>>> if >>>>>>>> you DON'T LIKE THAT._______________________________________________ >>>>>>>> rsyslog mailing list >>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>> http://www.rsyslog.com/professional-services/ >>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >>>>>>>> if >>>>>>>> you DON'T LIKE THAT. >>>>>>>> >>>>>>> _______________________________________________ >>>>>>> rsyslog mailing list >>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>> http://www.rsyslog.com/professional-services/ >>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >>>>>>> if >>>>>>> you DON'T LIKE THAT. >>>>>>> >>>>>> _______________________________________________ >>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>> http://www.rsyslog.com/professional-services/ >>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >>>>>> you DON'T LIKE THAT. >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Regards, >>>>> Janmejay >>>>> http://codehunk.wordpress.com >>>>> >>>> >>>> >>>> >>>> -- >>>> Regards, >>>> Janmejay >>>> http://codehunk.wordpress.com >>>> >>> >>> >>> >>> -- >>> Regards, >>> Janmejay >>> http://codehunk.wordpress.com >>> >> >> >> >> -- >> Regards, >> Janmejay >> http://codehunk.wordpress.com >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
