What I do on my relay boxes
$template structured_forwarding,"<%pri%>%timereported% %hostname% %syslogtag%
@cee:%$!%\n"
module(load="imudp" timerequery="4")
module(load="imtcp" maxsessions="1000")
module(load="mmjsonparse")
input(type="imudp" port="514" ruleset="relay")
input(type="imtcp" port="514" ruleset="relay")
ruleset(name="relay"
parser=["rsyslog.ciscoios","rsyslog.rfc5424","rsyslog.rfc3164"]){
action(type="mmjsonparse")
# if the message we got was in JSON from the beginning, there won't be a $!msg
variable
if $!msg == "" then set $!msg = $msg;
set $!trusted!origserver = $fromhost-ip;
set $!trusted!edge!time = $timegenerated;
set $!trusted!edge!relay = $$myhostname;
set $!trusted!edge!input = $inputname;
set $!trusted!environment = "NonProd";
action(type="omfwd" Target="10.1.5.5" Port="514" Protocol="tcp" queue.type="FixedArray"
template="structured_forwarding" name="relay_remote")
}
I do the rest of the parsing on the central system (it's fast enough and it
avoids bloating the messages that are relayed)
David Lang
On Wed, 15 Apr 2015, Radu Gheorghe wrote:
Date: Wed, 15 Apr 2015 18:47:52 +0300
From: Radu Gheorghe <[email protected]>
Reply-To: rsyslog-users <[email protected]>
To: rsyslog-users <[email protected]>
Subject: Re: [rsyslog] Best practice for an application to get structured data
to rsyslog
On Wed, Apr 15, 2015 at 6:25 PM, Dave Caplinger <
[email protected]> wrote:
On Apr 14, 2015, at 11:43 PM, David Lang <[email protected]> wrote:
On Wed, 15 Apr 2015, Ezell, Matthew A. wrote:
[...]
what I do is to take whatever message was output and then run mmjsonparse
against it. If it's cee JSON (insert grumble about the requirement for
the cee
cookie ;-) I have all the variables, but no $!msg field. If I have a
$!msg
field, then I parse it using mmnormalize to extract variables from it.
If there
isn't a $!msg field, I set $!msg=$mesg so that I have something I can
spit out
when I'm doing a 'plain' logfile.
I also add metadata to the JSON (fromhost-ip, received time, hostname of
relay,
and an environment tag so that later on I can trivially tell the
difference
between dev and prod copies of the same software)
We do something very similar to this, and I suspect so do other
high-volume Rsyslog users such as Radu at Sematext.
Yes, we actually check whether parsing worked:
if $parsesuccess == "OK" then
...
and use different templates for JSON and non-JSON messages. For JSON ones
we use the $!all-json variable to get us all parsed properties. You could
also use the jsonmesg property to get everything (parsed + syslog
variables) but some info will be duplicated that way.
Best regards,
Radu
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
