not every string in a rsyslog config can be replaced with a variable of any
kind.
you can compare two variables in an if..then statement, but you cannot use a
variable in a facility/severity filter (you can't say *.$!foo)
the rsyslog config gets compiled at startup, not interpreted with each log
that's processed. Rsyslog keeps connections to remote systems open (as well as
files)
with fixed IP/port pairs, rsyslog can open the connection and keep it open. With
these being variable, rsyslog will have to either close/open them each time
(which is _extremely_ expensive), or will have to implement a mechanism to track
a bunch of connectons, deal with them timing out and closing on you, etc. That's
a lot of infrastructure to have to add and maintain over time. That's a lot of
overhead you are asking for, with a large ask you need to provide a lot of
justification, and so far that's been missing.
David Lang
On Wed, 25 Apr 2018, putcha narayana wrote:
Date: Wed, 25 Apr 2018 04:22:03 +0000
From: putcha narayana <[email protected]>
To: Rainer Gerhards <[email protected]>
Cc: rsyslog-users <[email protected]>, David Lang <[email protected]>
Subject: Re: [rsyslog] Urgent,
need help with correct usage of local variables in an action
Hi,
We are having parallel mail chains so i will paste David's input here and share
my inputs.
Rainer's mail added in ""
1) "Your contribution is happily accepted at https://github.com/rsyslog/rsyslog-doc
-"
[[LAK]]: I will take time and share my inputs for this. You guys are doing so
much, i really appreciate that.
2) "I would really like to understand why you think global constants (there are NOT
variable, right?) would make sense. Maybe we are overlooking something... "
[[LAK]]: I called them variables because that is what they were called in
David's document.
https://www.usenix.org/publications/login/october-2013-volume-38-number-5/log-filtering-rsyslog
We can call them as constants as they are not run-time modifiable.
3) Regarding the backticks option.
[[Lak]]: Does it support pipes and awk. Say i have rsyslogserver_config_file
with content as
IPADDRESS1=10.40.60.50
IPADDRESS2=10.50.70.60
`grep IPADDRESS1 rsyslogserver_config_file| awk -F= '{print $2}'` -- Does
this work?
4) "I would really like to understand why you think global constants (there are NOT
variable, right?) would make sense. Maybe we are overlooking something..."
[[LAK]]: I felt that global constants could be part of one of the config files
loaded when rsyslogd starts or restarted. that way they are still part of
rsyslogd. May be an example of how and where global constants must be used can
enlighten me and others like me.
David's mail enclosed below and my responses are inline:
1) When you are already editing the config file and restarting syslog to
implement
your change, why should we suffer the cost of making this a variable?
[[Lak]]: We can merge bullet-3 and bullet-4 above.
Backticks provides an alternate way. Current document opens up lot of questions
as to what all one can do with that. Is the current support extended to
everything that can be done using backticks. Note, i could write whole bunch of
script and run it using backticks. It could have ';' and is that acceptable.
It's not just the cost of a variable lookup, it's the cost of maintaining a
table of connectons, expiring them, figuring out how to re-use connections, etc.
We do this with filenames using the dynafile mechanism, and it's a significant
enough overhead that it has to be explicitly opted in to each time it's used
(and misuse of this feature and it's settings is a very common cause of
horrifically bad performance)
[[LAK]]: Recommended usage in the document and references to tickets which
caused bad performance due to bad usage can help users understand it better and
not repeat the mistakes.
sending to a wide variety of destinations is just not that common a use case,
and when people are doing it (high volume systems distributing load), they don't
want to pay the overhead. And even there, it's very unusual for there to be more
than a handful of destinations, so simple if-then-else nested outputs handle the
job.
[[LAK]]: But configuring the remote syslog server, changing the severity of the
logs to be streamed should be and is configurable. Since we have scripting
option, I could use the following mechanism.
set $.configuredSeverity = "6";
($syslogseverity <= $.configuredSeverity)
There can be multiple configuration files each filter certain set of logs based
on their properties (Severity, process-name, msg content, etc). With separate
configuration files I could achieve modularity by separating application
processes from platform processes. Allowing the user to have different severity
for each of them.
One common variable among all the configuration files is the target IP and
port. So instead of setting it in each config file, i thought of using GLOBAL
variable. I will explore the Backticks option today.
________________________________
From: Rainer Gerhards <[email protected]>
Sent: Tuesday, April 24, 2018 12:01 PM
To: putcha narayana
Cc: rsyslog-users; David Lang
Subject: Re: [rsyslog] Urgent, need help with correct usage of local variables
in an action
2018-04-24 13:38 GMT+02:00 putcha narayana
<[email protected]<mailto:[email protected]>>:
😊 wasn't me. so that makes 2 of us asking for same lol.
i think, using global variable would have kept the export and backtick options
out of the rsyslogd. But i am sure the support was provided after careful
consideration and requests from others.
Actually, I still don't understand why this is helpful. After all, that means
you need to *edit* the rsyslog configuration. That's actually what your script
does when it modifies the local variables. So you need to mess up with the
config in any case. With the backticks approach, all is done outside of the
actual config, and actually this was the design goal - keep the config read
only, so you can ship a standard config for your application and have an
external file that just adds the special bits.
I would really like to understand why you think global constants (there are NOT
variable, right?) would make sense. Maybe we are overlooking something...
the link https://www.rsyslog.com/doc/master/rainerscript/constant_strings.html
is missing info on when to us them.
https://github.com/rsyslog/rsyslog-docker/tree/master/appliance/alpine provides
one example.
may be having references from action and other places where it can be used
could help. or a short explanation in the constant_strings section about usage
and examples will help too.
i felt that gap that each section is missing that short list of examples. i bet
not all scenarios can be covered with examples but a short list of plausible
alternatives can help to a great extent.
Your contribution is happily accepted at https://github.com/rsyslog/rsyslog-doc
- really, not kidding. There is so much to do, we depend on contributions
(especially on the doc, as e.g. from my perspective I really don't come up with
the questions new users might have)...
Rainer
________________________________
From: Rainer Gerhards
<[email protected]<mailto:[email protected]>>
Sent: Tuesday, April 24, 2018 11:11 AM
To: putcha narayana
Cc: rsyslog-users; David Lang
Subject: Re: [rsyslog] Urgent, need help with correct usage of local variables
in an action
You can also have a look here - came in co-incidentely (or was it you?
;-)):
https://unix.stackexchange.com/questions/439286/how-to-set-and-use-constants-in-rsyslog-rainerscript/
[https://cdn.sstatic.net/Sites/unix/img/[email protected]?v=32fb07f7ce26]<https://unix.stackexchange.com/questions/439286/how-to-set-and-use-constants-in-rsyslog-rainerscript/>
How to set and use constants in rsyslog
(RainerScript)?<https://unix.stackexchange.com/questions/439286/how-to-set-and-use-constants-in-rsyslog-rainerscript/>
unix.stackexchange.com<http://unix.stackexchange.com>
I'm writing an application which includes an rsyslog configuration which gets placed in
/etc/rsyslog.d/. Certain logmessages should be redirected to a named pipe, like so:
template (name="my_fmt"...
Rainer
2018-04-24 12:56 GMT+02:00 Rainer Gerhards
<[email protected]<mailto:[email protected]>>:
Looking at your script, I wonder why you make it so complicated. You
replace the local variable via sed anyway. Why not simply replacing
the name inside the action target?
Rainer
2018-04-24 10:47 GMT+02:00 putcha narayana
<[email protected]<mailto:[email protected]>>:
Hello Rainer,
I shared an example of the mechanism i planned to employ. I understand that
my mechanism cannot be achieved with the current rsyslogd. There is no hard
and fast rule to only use the mechanism i shared.
I will be happy to employ an alternate working mechanism. Now that my end
goal is clear, please share some example for the alternate methods, i will
employ them.
Appreciate your help.
Thanks and Regards
Lak.
________________________________
From: Rainer Gerhards
<[email protected]<mailto:[email protected]>>
Sent: Tuesday, April 24, 2018 8:43 AM
To: putcha narayana
Cc: rsyslog-users; David Lang
Subject: Re: [rsyslog] Urgent, need help with correct usage of local
variables in an action
Well, there are ways to do this, just not like you want ;-)
Environment variables and files is the way to go.
Rainer
Sent from phone, thus brief.
putcha narayana <[email protected]<mailto:[email protected]>>
schrieb am Di., 24. Apr. 2018,
10:32:
I believe it will be a good useful feature to allow users to define
local/global variables so that update in individual conf file or one place
update of global variables will enable configuring the external server IP
and Port.
I plan to use a static IP in the .66-MgmtMsgsLogging.conf.
When IP and Port are updated then using the script i will replace them.
When enable option is given then the comment in front of action will be
removed.
To disable add the # back.
#action(type="omfwd" Target="10.10.10.10" Port="0" Protocol="udp"
Template="ForwardFormat")
#action(type="omfwd" Target="11.11.11.11" Port="0" Protocol="udp"
Template="ForwardFormat")
Thanks and Regards
Lak.
________________________________
From: Rainer Gerhards
<[email protected]<mailto:[email protected]>>
Sent: Tuesday, April 24, 2018 8:23 AM
To: rsyslog-users
Cc: David Lang; putcha narayana
Subject: Re: [rsyslog] Urgent, need help with correct usage of local
variables in an action
Of you insist on using local variables, the answer is: no, this does not
work.
Rainer
Sent from phone, thus brief.
putcha narayana via rsyslog
<[email protected]<mailto:[email protected]>> schrieb am Di.,
24.
Apr. 2018, 10:08:
Hi,
I am not requesting for script to update the config file. I have already
implemented that. I have attached the shell script and a sample config file.
I am seeking help w.r.t the config file with a provision to use local
variables so that they can be updated/configurable by user.
Thanks and Regards
Lak.
________________________________
From: David Lang <[email protected]<mailto:[email protected]>>
Sent: Tuesday, April 24, 2018 6:59 AM
To: putcha narayana
Cc: David Lang; putcha narayana via rsyslog
Subject: Re: [rsyslog] Urgent, need help with correct usage of local
variables in an action
This cannot be done inside rsyslog, you would need to use some external
software
(puppet, chef, salt, ansible, or even just sed) to change the config file
and
then start/restart rsyslog
On Tue, 24 Apr 2018, putcha narayana wrote:
Date: Tue, 24 Apr 2018 06:47:03 +0000
From: putcha narayana <[email protected]<mailto:[email protected]>>
To: David Lang <[email protected]<mailto:[email protected]>>,
putcha narayana via rsyslog
<[email protected]<mailto:[email protected]>>
Subject: Re: [rsyslog] Urgent,
need help with correct usage of local variables in an action
Hi,
In our design we have a GUI where operator can configure the IP and Port
of the remote syslog server. Operator can enable or disable streaming to the
configured IP and Port using a flag.
In the backend we will consume the configuration made by the operator and
prepare the required configuration file to stream logs to the configured
remote syslog server (Identified by IP and Port).
1) In the design i chose, I plan to have initial config files prefixed
with '.' so that those config files will not be read by rsyslogd.
Example config file is enclosed below.
2) When operator configures the IP, Port, then a script will be invoked in
the background which will update the initial set of lines that define the
local variables. Say set $.IPADDRESS1 = ""; will be updated as set
$.IPADDRESS1 = "10.10.100.200";
3) Config file updated will be saved without the '.' prefix. restarting
rsyslod will/should apply the configuration (i.e., stream logs to remote
syslog server 10.10.100.200.
Example config file with 2 remote syslog servers for illustration purpose:
set $.IPADDRESS1 = "";
set $.IPADDRESS2 = "";
set $.PORTNUMBER1 = "";
set $.PORTNUMBER2 = "";
if ($syslogseverity-text == 'info')
then
{
action(type="omfwd" Target="$.IPADDRESS1" Port="$.PORTNUMBER1"
Protocol="udp" Template="ForwardFormat")
action(type="omfwd" Target="$.IPADDRESS2" Port="$.PORTNUMBER2"
Protocol="udp" Template="ForwardFormat")
stop
}
Please guide me if the same can be achieved using alternate mechanisms.
thanks and regards
Lak.
________________________________
From: David Lang <[email protected]<mailto:[email protected]>>
Sent: Tuesday, April 24, 2018 5:36 AM
To: putcha narayana via rsyslog
Cc: Rainer Gerhards; putcha narayana
Subject: Re: [rsyslog] Urgent, need help with correct usage of local
variables in an action
target and port cannot be variables, variables can only be used where they
are
explicitly allowed (almost entirely in templates)
let's back up a bit and ask what you are trying to do rather than why this
specific approach doesn't work.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
![https://cdn.sstatic.net/Sites/unix/img/[email protected]?v=32fb07f7ce26](https://cdn.sstatic.net/Sites/unix/img/apple-touch-i...@2.png?v=32fb07f7ce26){kind=link}