Re: [rsyslog] Sourcing Environment Variables for Use in Templates?
just set $\foo instead of $!foo
On Fri, 27 Oct 2017, Naftuli Kay via rsyslog wrote:
Can anyone shed any light on how to set global variables? Environment
variables won't change over the lifetime of the process so it would make
sense to not have to allocate for every log message.
Thanks,
- Naftuli Kay
On Wed, Oct 25, 2017 at 9:13 PM, David Lang <da...@lang.hm> wrote:
Rsyslog variables are JSON based, so everything can be represented in
aJSON structure. To address it, you separate keys with !
There are two 'normal' trees of variables $! and $.
Sincd $! existed first, and several things use it by default, it's
considered the 'normal' variables. But since there is a need to have
variables that don't show up by default, $. was introduced and is called
'local' variables.
Global variables exist to allow you to set something that lasts past the
current log message
so back to how variables work.
If you had the JSON object {"foo":"1","bar":"2"}
you could use $!foo or $!bar (or $.foo or $.bar if it's local variables)
If you have the JSON object {"foo":{"bar":"1","baz":"2"}}
you could use $!foo!bar or $!foo!baz (or $.foo.bar $.foo.baz)
but you can also just use $!foo, which would return the string
{"bar":"1","baz":"2"}
$! (or $.) return the entire structure.
If you write a log with the template RSYSLOG_DebugFormat, you will see all
the variables that exist, and the bottom three lines with be $!, $., and $\
David Lang
On Wed, 25 Oct 2017, deoren wrote:
Date: Wed, 25 Oct 2017 17:24:22 -0500
From: deoren <rsyslog-users-lists.adiscon@whyaskwhy.org>
Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
To: rsyslog@lists.adiscon.com
Subject: Re: [rsyslog] Sourcing Environment Variables for Use in
Templates?
On 10/25/2017 4:18 PM, Naftuli Kay via rsyslog wrote:
So would I do "set $deploy_env = getenv('DEPLOY_ENV')"? How would I then
reference this variable? I'm still trying to learn more about rsyslog
variables and how to use them in templates.
I'm still learning myself, so I completely understand.
Start here:
http://www.rsyslog.com/doc/v8-stable/rainerscript/variable_p
roperty_types.html
As I understand it, you have three types of variables/properties that you
can set: local variables, message JSON properties and global variables.
Global variables are not listed on that page, but probably should be. From
what I remember reading on the mailing list, they operate much like global
variables in programming languages: they're in the highest scope and are
independent of the log messages flowing through rsyslog.
For your use, you likely want to set either a local variable or a message
JSON property. The latter would work best if you need to pass that value
off to another system or include it alongside other values bundled within
that tree, otherwise if that isn't your goal, you likely will want to use a
local variable.
Here is how you set a local variable:
set $.deploy_env = getenv('DEPLOY_ENV');
For something like this, I would be tempted to use a global variable, but
since I don't have experience using them, I can't personally recommend that
approach.
Assuming you go with a local variable, you should be able to reference it
as '$.deploy_env' in your templates like any other local variable. Here is
an example of referencing two local variables I set earlier in my
configuration ('$.tag' and '$.msg') within a template:
template(name="Modified-StockRsyslogFileFormat" type="list") {
property(name="timestamp" dateFormat="rfc3339")
constant(value=" ")
property(name="hostname")
constant(value=" ")
property(name="$.tag")
property(name="$.msg" spifno1stsp="on" )
property(name="$.msg" droplastlf="on" )
constant(value="\n")
}
It's not the best named template and I'll likely update it at some point
to better describe WHY I opted to use it vs the standard template (in my
case I wanted the existing format, but wanted to drop in my local variables
in place of the default message properties referenced by the stock
template). AFAIK it's not possible to inherit an existing template and
tweak a value, though I could be wrong on that (the whole "still learning"
bit very much applies).
Lastly, I ended up saving myself a lot of frustration by generating an
epub file from the documentation and reading it on my devices over the
course of a month or so. PDFs and general HTML format are also available.
http://rsyslog.readthedocs.io/en/stable/
http://www.rsyslog.com/doc/v8-stable/
It takes some time to work through the content, but it was time well
spent.
___
rsyslog mailing list
http://lists.
Re: [rsyslog] Sourcing Environment Variables for Use in Templates?
On 10/27/2017 5:19 PM, Naftuli Kay via rsyslog wrote: Can anyone shed any light on how to set global variables? Environment variables won't change over the lifetime of the process so it would make sense to not have to allocate for every log message. Thanks, - Naftuli Kay I've not used them yet, but I've seen global variables referenced with either a '$/' [1][2] or a '$\' prefix[3]. I don't know which is correct, but I would assume based on the frequency where I found them listed that the first is correct. Once you determine that, you would use global variables in the same manner that David Lang described in his last reply to this thread. [1] https://www.usenix.org/system/files/login/articles/06_lang-online.pdf [2] http://lists.adiscon.net/pipermail/rsyslog/2013-October/034453.html [3] http://lists.adiscon.net/pipermail/rsyslog/2017-October/044716.html ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] Sourcing Environment Variables for Use in Templates?
Can anyone shed any light on how to set global variables? Environment
variables won't change over the lifetime of the process so it would make
sense to not have to allocate for every log message.
Thanks,
- Naftuli Kay
On Wed, Oct 25, 2017 at 9:13 PM, David Lang <da...@lang.hm> wrote:
> Rsyslog variables are JSON based, so everything can be represented in
> aJSON structure. To address it, you separate keys with !
>
> There are two 'normal' trees of variables $! and $.
>
> Sincd $! existed first, and several things use it by default, it's
> considered the 'normal' variables. But since there is a need to have
> variables that don't show up by default, $. was introduced and is called
> 'local' variables.
>
> Global variables exist to allow you to set something that lasts past the
> current log message
>
> so back to how variables work.
>
> If you had the JSON object {"foo":"1","bar":"2"}
>
> you could use $!foo or $!bar (or $.foo or $.bar if it's local variables)
>
> If you have the JSON object {"foo":{"bar":"1","baz":"2"}}
>
> you could use $!foo!bar or $!foo!baz (or $.foo.bar $.foo.baz)
>
> but you can also just use $!foo, which would return the string
> {"bar":"1","baz":"2"}
>
> $! (or $.) return the entire structure.
>
> If you write a log with the template RSYSLOG_DebugFormat, you will see all
> the variables that exist, and the bottom three lines with be $!, $., and $\
>
> David Lang
>
> On Wed, 25 Oct 2017, deoren wrote:
>
> Date: Wed, 25 Oct 2017 17:24:22 -0500
>> From: deoren <rsyslog-users-lists.adiscon@whyaskwhy.org>
>> Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
>> To: rsyslog@lists.adiscon.com
>> Subject: Re: [rsyslog] Sourcing Environment Variables for Use in
>> Templates?
>>
>>
>> On 10/25/2017 4:18 PM, Naftuli Kay via rsyslog wrote:
>>
>>> So would I do "set $deploy_env = getenv('DEPLOY_ENV')"? How would I then
>>> reference this variable? I'm still trying to learn more about rsyslog
>>> variables and how to use them in templates.
>>>
>>
>> I'm still learning myself, so I completely understand.
>>
>> Start here:
>>
>> http://www.rsyslog.com/doc/v8-stable/rainerscript/variable_p
>> roperty_types.html
>>
>> As I understand it, you have three types of variables/properties that you
>> can set: local variables, message JSON properties and global variables.
>> Global variables are not listed on that page, but probably should be. From
>> what I remember reading on the mailing list, they operate much like global
>> variables in programming languages: they're in the highest scope and are
>> independent of the log messages flowing through rsyslog.
>>
>> For your use, you likely want to set either a local variable or a message
>> JSON property. The latter would work best if you need to pass that value
>> off to another system or include it alongside other values bundled within
>> that tree, otherwise if that isn't your goal, you likely will want to use a
>> local variable.
>>
>> Here is how you set a local variable:
>>
>> set $.deploy_env = getenv('DEPLOY_ENV');
>>
>> For something like this, I would be tempted to use a global variable, but
>> since I don't have experience using them, I can't personally recommend that
>> approach.
>>
>> Assuming you go with a local variable, you should be able to reference it
>> as '$.deploy_env' in your templates like any other local variable. Here is
>> an example of referencing two local variables I set earlier in my
>> configuration ('$.tag' and '$.msg') within a template:
>>
>> template(name="Modified-StockRsyslogFileFormat" type="list") {
>>property(name="timestamp" dateFormat="rfc3339")
>>constant(value=" ")
>>property(name="hostname")
>>constant(value=" ")
>>property(name="$.tag")
>>property(name="$.msg" spifno1stsp="on" )
>>property(name="$.msg" droplastlf="on" )
>>constant(value="\n")
>> }
>>
>> It's not the best named template and I'll likely update it at some point
>> to better describe WHY I opted to use it vs the standard template (in my
>> case I wanted the existing format, but wanted to drop in my local variables
>> in place of the default message properties referenced by the stock
>> template). AFAIK it's not possible to inherit a
Re: [rsyslog] Sourcing Environment Variables for Use in Templates?
Rsyslog variables are JSON based, so everything can be represented in aJSON
structure. To address it, you separate keys with !
There are two 'normal' trees of variables $! and $.
Sincd $! existed first, and several things use it by default, it's considered
the 'normal' variables. But since there is a need to have variables that don't
show up by default, $. was introduced and is called 'local' variables.
Global variables exist to allow you to set something that lasts past the current
log message
so back to how variables work.
If you had the JSON object {"foo":"1","bar":"2"}
you could use $!foo or $!bar (or $.foo or $.bar if it's local variables)
If you have the JSON object {"foo":{"bar":"1","baz":"2"}}
you could use $!foo!bar or $!foo!baz (or $.foo.bar $.foo.baz)
but you can also just use $!foo, which would return the string
{"bar":"1","baz":"2"}
$! (or $.) return the entire structure.
If you write a log with the template RSYSLOG_DebugFormat, you will see all the
variables that exist, and the bottom three lines with be $!, $., and $\
David Lang
On Wed, 25 Oct 2017, deoren wrote:
Date: Wed, 25 Oct 2017 17:24:22 -0500
From: deoren <rsyslog-users-lists.adiscon....@whyaskwhy.org>
Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
To: rsyslog@lists.adiscon.com
Subject: Re: [rsyslog] Sourcing Environment Variables for Use in Templates?
On 10/25/2017 4:18 PM, Naftuli Kay via rsyslog wrote:
So would I do "set $deploy_env = getenv('DEPLOY_ENV')"? How would I then
reference this variable? I'm still trying to learn more about rsyslog
variables and how to use them in templates.
I'm still learning myself, so I completely understand.
Start here:
http://www.rsyslog.com/doc/v8-stable/rainerscript/variable_property_types.html
As I understand it, you have three types of variables/properties that you can
set: local variables, message JSON properties and global variables. Global
variables are not listed on that page, but probably should be. From what I
remember reading on the mailing list, they operate much like global variables
in programming languages: they're in the highest scope and are independent of
the log messages flowing through rsyslog.
For your use, you likely want to set either a local variable or a message
JSON property. The latter would work best if you need to pass that value off
to another system or include it alongside other values bundled within that
tree, otherwise if that isn't your goal, you likely will want to use a local
variable.
Here is how you set a local variable:
set $.deploy_env = getenv('DEPLOY_ENV');
For something like this, I would be tempted to use a global variable, but
since I don't have experience using them, I can't personally recommend that
approach.
Assuming you go with a local variable, you should be able to reference it as
'$.deploy_env' in your templates like any other local variable. Here is an
example of referencing two local variables I set earlier in my configuration
('$.tag' and '$.msg') within a template:
template(name="Modified-StockRsyslogFileFormat" type="list") {
property(name="timestamp" dateFormat="rfc3339")
constant(value=" ")
property(name="hostname")
constant(value=" ")
property(name="$.tag")
property(name="$.msg" spifno1stsp="on" )
property(name="$.msg" droplastlf="on" )
constant(value="\n")
}
It's not the best named template and I'll likely update it at some point to
better describe WHY I opted to use it vs the standard template (in my case I
wanted the existing format, but wanted to drop in my local variables in place
of the default message properties referenced by the stock template). AFAIK
it's not possible to inherit an existing template and tweak a value, though I
could be wrong on that (the whole "still learning" bit very much applies).
Lastly, I ended up saving myself a lot of frustration by generating an epub
file from the documentation and reading it on my devices over the course of a
month or so. PDFs and general HTML format are also available.
http://rsyslog.readthedocs.io/en/stable/
http://www.rsyslog.com/doc/v8-stable/
It takes some time to work through the content, but it was time well spent.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
Re: [rsyslog] Sourcing Environment Variables for Use in Templates?
On 10/25/2017 4:18 PM, Naftuli Kay via rsyslog wrote:
So would I do "set $deploy_env = getenv('DEPLOY_ENV')"? How would I then
reference this variable? I'm still trying to learn more about rsyslog
variables and how to use them in templates.
I'm still learning myself, so I completely understand.
Start here:
http://www.rsyslog.com/doc/v8-stable/rainerscript/variable_property_types.html
As I understand it, you have three types of variables/properties that
you can set: local variables, message JSON properties and global
variables. Global variables are not listed on that page, but probably
should be. From what I remember reading on the mailing list, they
operate much like global variables in programming languages: they're in
the highest scope and are independent of the log messages flowing
through rsyslog.
For your use, you likely want to set either a local variable or a
message JSON property. The latter would work best if you need to pass
that value off to another system or include it alongside other values
bundled within that tree, otherwise if that isn't your goal, you likely
will want to use a local variable.
Here is how you set a local variable:
set $.deploy_env = getenv('DEPLOY_ENV');
For something like this, I would be tempted to use a global variable,
but since I don't have experience using them, I can't personally
recommend that approach.
Assuming you go with a local variable, you should be able to reference
it as '$.deploy_env' in your templates like any other local variable.
Here is an example of referencing two local variables I set earlier in
my configuration ('$.tag' and '$.msg') within a template:
template(name="Modified-StockRsyslogFileFormat" type="list") {
property(name="timestamp" dateFormat="rfc3339")
constant(value=" ")
property(name="hostname")
constant(value=" ")
property(name="$.tag")
property(name="$.msg" spifno1stsp="on" )
property(name="$.msg" droplastlf="on" )
constant(value="\n")
}
It's not the best named template and I'll likely update it at some point
to better describe WHY I opted to use it vs the standard template (in my
case I wanted the existing format, but wanted to drop in my local
variables in place of the default message properties referenced by the
stock template). AFAIK it's not possible to inherit an existing template
and tweak a value, though I could be wrong on that (the whole "still
learning" bit very much applies).
Lastly, I ended up saving myself a lot of frustration by generating an
epub file from the documentation and reading it on my devices over the
course of a month or so. PDFs and general HTML format are also available.
http://rsyslog.readthedocs.io/en/stable/
http://www.rsyslog.com/doc/v8-stable/
It takes some time to work through the content, but it was time well spent.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
Re: [rsyslog] Sourcing Environment Variables for Use in Templates?
So would I do "set $deploy_env = getenv('DEPLOY_ENV')"? How would I then
reference this variable? I'm still trying to learn more about rsyslog
variables and how to use them in templates.
Thanks,
- Naftuli Kay
On Wed, Oct 25, 2017 at 2:05 PM, deoren <
rsyslog-users-lists.adiscon@whyaskwhy.org> wrote:
> On 10/25/2017 3:48 PM, Naftuli Kay via rsyslog wrote:
>
>> I have a few environment variables that I'd like to include in my log
>> messages that I'm formatting in JSON format. I have a service that runs on
>> boot which generates /etc/sysconfig/ec2 which contains variables like
>> EC2_INSTANCE_ID, EC2_AMI_ID, etc.
>>
>> If I expose these environment variables to the rsyslog daemon's execution
>> environment, is there a way for me to use them in my templates? The only
>> other alternative that I can see is using a script to template my rsyslog
>> configuration files before starting rsyslog, which seems less than ideal.
>>
>
> http://www.rsyslog.com/doc/v8-stable/rainerscript/functions.
> html#getenv-str
>
> I've not done it myself, but the support appears to be there.
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
Re: [rsyslog] Sourcing Environment Variables for Use in Templates?
On 10/25/2017 3:48 PM, Naftuli Kay via rsyslog wrote: I have a few environment variables that I'd like to include in my log messages that I'm formatting in JSON format. I have a service that runs on boot which generates /etc/sysconfig/ec2 which contains variables like EC2_INSTANCE_ID, EC2_AMI_ID, etc. If I expose these environment variables to the rsyslog daemon's execution environment, is there a way for me to use them in my templates? The only other alternative that I can see is using a script to template my rsyslog configuration files before starting rsyslog, which seems less than ideal. http://www.rsyslog.com/doc/v8-stable/rainerscript/functions.html#getenv-str I've not done it myself, but the support appears to be there. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
