David, I understand your point and the complexity.
set $.configuredSeverity = "6"; if ($syslogseverity <= $.configuredSeverity) -- this is allowed and we can add to the documentation. I will pull such from my experience and add to the doc and get it reviewed. Appreciate the time and support. Thanks and Regards Lak. ________________________________ From: David Lang <[email protected]> Sent: Wednesday, April 25, 2018 5:20 AM To: putcha narayana Cc: Rainer Gerhards; rsyslog-users; David Lang Subject: Re: [rsyslog] Urgent, need help with correct usage of local variables in an action not every string in a rsyslog config can be replaced with a variable of any kind. you can compare two variables in an if..then statement, but you cannot use a variable in a facility/severity filter (you can't say *.$!foo) the rsyslog config gets compiled at startup, not interpreted with each log that's processed. Rsyslog keeps connections to remote systems open (as well as files) with fixed IP/port pairs, rsyslog can open the connection and keep it open. With these being variable, rsyslog will have to either close/open them each time (which is _extremely_ expensive), or will have to implement a mechanism to track a bunch of connectons, deal with them timing out and closing on you, etc. That's a lot of infrastructure to have to add and maintain over time. That's a lot of overhead you are asking for, with a large ask you need to provide a lot of justification, and so far that's been missing. David Lang On Wed, 25 Apr 2018, putcha narayana wrote: > Date: Wed, 25 Apr 2018 04:22:03 +0000 > From: putcha narayana <[email protected]> > To: Rainer Gerhards <[email protected]> > Cc: rsyslog-users <[email protected]>, David Lang <[email protected]> > Subject: Re: [rsyslog] Urgent, > need help with correct usage of local variables in an action > > Hi, > > > We are having parallel mail chains so i will paste David's input here and > share my inputs. > > > Rainer's mail added in "" > > 1) "Your contribution is happily accepted at > https://github.com/rsyslog/rsyslog-doc -" [https://avatars1.githubusercontent.com/u/6178456?s=400&v=4]<https://github.com/rsyslog/rsyslog-doc> GitHub - rsyslog/rsyslog-doc: documentation for the ...<https://github.com/rsyslog/rsyslog-doc> github.com README.md rsyslog-docs Documentation for the rsyslog project. Documentation for rsyslog is generated with the (Python) Sphinx documentation processor. > > [[LAK]]: I will take time and share my inputs for this. You guys are doing so > much, i really appreciate that. > > > 2) "I would really like to understand why you think global constants (there > are NOT variable, right?) would make sense. Maybe we are overlooking > something... " > > [[LAK]]: I called them variables because that is what they were called in > David's document. > https://www.usenix.org/publications/login/october-2013-volume-38-number-5/log-filtering-rsyslog > > We can call them as constants as they are not run-time modifiable. > > > 3) Regarding the backticks option. > > [[Lak]]: Does it support pipes and awk. Say i have rsyslogserver_config_file > with content as > > IPADDRESS1=10.40.60.50 > IPADDRESS2=10.50.70.60 > > `grep IPADDRESS1 rsyslogserver_config_file| awk -F= '{print $2}'` -- Does > this work? > > > 4) "I would really like to understand why you think global constants (there > are NOT variable, right?) would make sense. Maybe we are overlooking > something..." > > [[LAK]]: I felt that global constants could be part of one of the config > files loaded when rsyslogd starts or restarted. that way they are still part > of rsyslogd. May be an example of how and where global constants must be used > can enlighten me and others like me. > > > David's mail enclosed below and my responses are inline: > > 1) When you are already editing the config file and restarting syslog to > implement > your change, why should we suffer the cost of making this a variable? > [[Lak]]: We can merge bullet-3 and bullet-4 above. > > Backticks provides an alternate way. Current document opens up lot of > questions as to what all one can do with that. Is the current support > extended to everything that can be done using backticks. Note, i could write > whole bunch of script and run it using backticks. It could have ';' and is > that acceptable. > > It's not just the cost of a variable lookup, it's the cost of maintaining a > table of connectons, expiring them, figuring out how to re-use connections, > etc. > We do this with filenames using the dynafile mechanism, and it's a significant > enough overhead that it has to be explicitly opted in to each time it's used > (and misuse of this feature and it's settings is a very common cause of > horrifically bad performance) > > [[LAK]]: Recommended usage in the document and references to tickets which > caused bad performance due to bad usage can help users understand it better > and not repeat the mistakes. > > sending to a wide variety of destinations is just not that common a use case, > and when people are doing it (high volume systems distributing load), they > don't > want to pay the overhead. And even there, it's very unusual for there to be > more > than a handful of destinations, so simple if-then-else nested outputs handle > the > job. > > [[LAK]]: But configuring the remote syslog server, changing the severity of > the logs to be streamed should be and is configurable. Since we have > scripting option, I could use the following mechanism. > > set $.configuredSeverity = "6"; > > ($syslogseverity <= $.configuredSeverity) > > There can be multiple configuration files each filter certain set of logs > based on their properties (Severity, process-name, msg content, etc). With > separate configuration files I could achieve modularity by separating > application processes from platform processes. Allowing the user to have > different severity for each of them. > > One common variable among all the configuration files is the target IP and > port. So instead of setting it in each config file, i thought of using GLOBAL > variable. I will explore the Backticks option today. > > > > ________________________________ > From: Rainer Gerhards <[email protected]> > Sent: Tuesday, April 24, 2018 12:01 PM > To: putcha narayana > Cc: rsyslog-users; David Lang > Subject: Re: [rsyslog] Urgent, need help with correct usage of local > variables in an action > > 2018-04-24 13:38 GMT+02:00 putcha narayana > <[email protected]<mailto:[email protected]>>: > > 😊 wasn't me. so that makes 2 of us asking for same lol. > > > i think, using global variable would have kept the export and backtick > options out of the rsyslogd. But i am sure the support was provided after > careful consideration and requests from others. > > Actually, I still don't understand why this is helpful. After all, that means > you need to *edit* the rsyslog configuration. That's actually what your > script does when it modifies the local variables. So you need to mess up with > the config in any case. With the backticks approach, all is done outside of > the actual config, and actually this was the design goal - keep the config > read only, so you can ship a standard config for your application and have an > external file that just adds the special bits. > > I would really like to understand why you think global constants (there are > NOT variable, right?) would make sense. Maybe we are overlooking something... > > > the link > https://www.rsyslog.com/doc/master/rainerscript/constant_strings.html is > missing info on when to us them. > > https://github.com/rsyslog/rsyslog-docker/tree/master/appliance/alpine > provides one example. > > may be having references from action and other places where it can be used > could help. or a short explanation in the constant_strings section about > usage and examples will help too. > > > i felt that gap that each section is missing that short list of examples. i > bet not all scenarios can be covered with examples but a short list of > plausible alternatives can help to a great extent. > > Your contribution is happily accepted at > https://github.com/rsyslog/rsyslog-doc - really, not kidding. There is so > much to do, we depend on contributions (especially on the doc, as e.g. from > my perspective I really don't come up with the questions new users might > have)... > > Rainer > > > ________________________________ > From: Rainer Gerhards > <[email protected]<mailto:[email protected]>> > Sent: Tuesday, April 24, 2018 11:11 AM > To: putcha narayana > Cc: rsyslog-users; David Lang > Subject: Re: [rsyslog] Urgent, need help with correct usage of local > variables in an action > > You can also have a look here - came in co-incidentely (or was it you? > ;-)): > https://unix.stackexchange.com/questions/439286/how-to-set-and-use-constants-in-rsyslog-rainerscript/ > > [https://cdn.sstatic.net/Sites/unix/img/[email protected]?v=32fb07f7ce26]<https://unix.stackexchange.com/questions/439286/how-to-set-and-use-constants-in-rsyslog-rainerscript/> > > How to set and use constants in rsyslog > (RainerScript)?<https://unix.stackexchange.com/questions/439286/how-to-set-and-use-constants-in-rsyslog-rainerscript/> > unix.stackexchange.com<http://unix.stackexchange.com> > I'm writing an application which includes an rsyslog configuration which gets > placed in /etc/rsyslog.d/. Certain logmessages should be redirected to a > named pipe, like so: template (name="my_fmt"... > > > > > Rainer > > 2018-04-24 12:56 GMT+02:00 Rainer Gerhards > <[email protected]<mailto:[email protected]>>: >> Looking at your script, I wonder why you make it so complicated. You >> replace the local variable via sed anyway. Why not simply replacing >> the name inside the action target? >> >> Rainer >> >> 2018-04-24 10:47 GMT+02:00 putcha narayana >> <[email protected]<mailto:[email protected]>>: >>> Hello Rainer, >>> >>> >>> I shared an example of the mechanism i planned to employ. I understand that >>> my mechanism cannot be achieved with the current rsyslogd. There is no hard >>> and fast rule to only use the mechanism i shared. >>> >>> >>> I will be happy to employ an alternate working mechanism. Now that my end >>> goal is clear, please share some example for the alternate methods, i will >>> employ them. >>> >>> >>> Appreciate your help. >>> >>> >>> Thanks and Regards >>> >>> Lak. >>> >>> >>> ________________________________ >>> From: Rainer Gerhards >>> <[email protected]<mailto:[email protected]>> >>> Sent: Tuesday, April 24, 2018 8:43 AM >>> To: putcha narayana >>> Cc: rsyslog-users; David Lang >>> >>> Subject: Re: [rsyslog] Urgent, need help with correct usage of local >>> variables in an action >>> >>> Well, there are ways to do this, just not like you want ;-) >>> >>> Environment variables and files is the way to go. >>> >>> Rainer >>> >>> Sent from phone, thus brief. >>> >>> putcha narayana <[email protected]<mailto:[email protected]>> >>> schrieb am Di., 24. Apr. 2018, >>> 10:32: >>> >>> I believe it will be a good useful feature to allow users to define >>> local/global variables so that update in individual conf file or one place >>> update of global variables will enable configuring the external server IP >>> and Port. >>> >>> >>> I plan to use a static IP in the .66-MgmtMsgsLogging.conf. >>> >>> When IP and Port are updated then using the script i will replace them. >>> >>> When enable option is given then the comment in front of action will be >>> removed. >>> >>> To disable add the # back. >>> >>> >>> #action(type="omfwd" Target="10.10.10.10" Port="0" Protocol="udp" >>> Template="ForwardFormat") >>> #action(type="omfwd" Target="11.11.11.11" Port="0" Protocol="udp" >>> Template="ForwardFormat") >>> >>> >>> Thanks and Regards >>> >>> Lak. >>> >>> >>> >>> >>> >>> ________________________________ >>> From: Rainer Gerhards >>> <[email protected]<mailto:[email protected]>> >>> Sent: Tuesday, April 24, 2018 8:23 AM >>> To: rsyslog-users >>> Cc: David Lang; putcha narayana >>> Subject: Re: [rsyslog] Urgent, need help with correct usage of local >>> variables in an action >>> >>> Of you insist on using local variables, the answer is: no, this does not >>> work. >>> >>> Rainer >>> >>> Sent from phone, thus brief. >>> >>> putcha narayana via rsyslog >>> <[email protected]<mailto:[email protected]>> schrieb am >>> Di., 24. >>> Apr. 2018, 10:08: >>> >>> Hi, >>> >>> >>> I am not requesting for script to update the config file. I have already >>> implemented that. I have attached the shell script and a sample config file. >>> >>> >>> I am seeking help w.r.t the config file with a provision to use local >>> variables so that they can be updated/configurable by user. >>> >>> >>> Thanks and Regards >>> >>> Lak. >>> >>> >>> ________________________________ >>> From: David Lang <[email protected]<mailto:[email protected]>> >>> Sent: Tuesday, April 24, 2018 6:59 AM >>> To: putcha narayana >>> Cc: David Lang; putcha narayana via rsyslog >>> Subject: Re: [rsyslog] Urgent, need help with correct usage of local >>> variables in an action >>> >>> This cannot be done inside rsyslog, you would need to use some external >>> software >>> (puppet, chef, salt, ansible, or even just sed) to change the config file >>> and >>> then start/restart rsyslog >>> >>> On Tue, 24 Apr 2018, putcha narayana wrote: >>> >>>> Date: Tue, 24 Apr 2018 06:47:03 +0000 >>>> From: putcha narayana >>>> <[email protected]<mailto:[email protected]>> >>>> To: David Lang <[email protected]<mailto:[email protected]>>, >>>> putcha narayana via rsyslog >>>> <[email protected]<mailto:[email protected]>> >>>> Subject: Re: [rsyslog] Urgent, >>>> need help with correct usage of local variables in an action >>>> >>>> Hi, >>>> >>>> >>>> In our design we have a GUI where operator can configure the IP and Port >>>> of the remote syslog server. Operator can enable or disable streaming to >>>> the >>>> configured IP and Port using a flag. >>>> >>>> >>>> In the backend we will consume the configuration made by the operator and >>>> prepare the required configuration file to stream logs to the configured >>>> remote syslog server (Identified by IP and Port). >>>> >>>> >>>> 1) In the design i chose, I plan to have initial config files prefixed >>>> with '.' so that those config files will not be read by rsyslogd. >>>> >>>> Example config file is enclosed below. >>>> >>>> 2) When operator configures the IP, Port, then a script will be invoked in >>>> the background which will update the initial set of lines that define the >>>> local variables. Say set $.IPADDRESS1 = ""; will be updated as set >>>> $.IPADDRESS1 = "10.10.100.200"; >>>> >>>> 3) Config file updated will be saved without the '.' prefix. restarting >>>> rsyslod will/should apply the configuration (i.e., stream logs to remote >>>> syslog server 10.10.100.200. >>>> >>>> >>>> Example config file with 2 remote syslog servers for illustration purpose: >>>> >>>> set $.IPADDRESS1 = ""; >>>> set $.IPADDRESS2 = ""; >>>> set $.PORTNUMBER1 = ""; >>>> set $.PORTNUMBER2 = ""; >>>> >>>> if ($syslogseverity-text == 'info') >>>> then >>>> { >>>> action(type="omfwd" Target="$.IPADDRESS1" Port="$.PORTNUMBER1" >>>> Protocol="udp" Template="ForwardFormat") >>>> action(type="omfwd" Target="$.IPADDRESS2" Port="$.PORTNUMBER2" >>>> Protocol="udp" Template="ForwardFormat") >>>> stop >>>> } >>>> >>>> >>>> Please guide me if the same can be achieved using alternate mechanisms. >>>> >>>> >>>> thanks and regards >>>> >>>> Lak. >>>> >>>> >>>> >>>> ________________________________ >>>> From: David Lang <[email protected]<mailto:[email protected]>> >>>> Sent: Tuesday, April 24, 2018 5:36 AM >>>> To: putcha narayana via rsyslog >>>> Cc: Rainer Gerhards; putcha narayana >>>> Subject: Re: [rsyslog] Urgent, need help with correct usage of local >>>> variables in an action >>>> >>>> target and port cannot be variables, variables can only be used where they >>>> are >>>> explicitly allowed (almost entirely in templates) >>>> >>>> let's back up a bit and ask what you are trying to do rather than why this >>>> specific approach doesn't work. >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of >>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T >>> LIKE THAT. > > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
![https://cdn.sstatic.net/Sites/unix/img/[email protected]?v=32fb07f7ce26](https://cdn.sstatic.net/Sites/unix/img/apple-touch-i...@2.png?v=32fb07f7ce26){kind=link}