On Tue, 11 Dec 2018, Lavanya Kanchanapalli via rsyslog wrote:
That tool is awesome! It confirmed my suspicion that mmnormalizer is doing
what it is supposed to. But my rsyslog conf is not able to retrieve the
parsed and structured text.
Below is rsyslog configuration I am using (mmnormalize property line
highlighted). According to this recipe example
<https://www.rsyslog.com/log-normalization-for-different-formats/>, values
parsed by mmnormalize are added under "usr" sub-tree. But that does not
seem to work.
What is a way to access log line parsed by mmnormalize?
if the parsing fails the variable original-msg and unparsed-data are created
instead of the items you are expecting.
output the log message with the template RSYSLOG_DebugFormat before and after
the mmnormalize call
did you fix the leading space problem?
David Lang
*rsyslog.conf*
# Load Modules
module (load="imfile" mode="inotify") # for text file with inotify
instead of pulling
module (load="builtin:omfile" dirCreateMode="0775" fileCreateMode="0664" )
module(load="mmnormalize") # text parsing
# RAW - don't format
template(name="ulog-raw-format" type="list") {
property(name="msg" droplastlf="on")
constant(value="\n")
}
# Template for plain text formatted log messages.
template(name="ulog-text-format" type="list") {
constant(value="{")
property(outname="appName" name="procid" format="jsonf")
constant(value=", ")
property(outname="hostname" name="hostname" format="jsonf")
constant(value=", ")
constant(value="\"event\":[{")
#property(outname="uuid" name="msgid" format="jsonf")
constant(value=", ")
constant(value="\"payload\":{")
property(outname="_log_" name="syslogtag" format="jsonf")
constant(value=", ")
#property(name="$!usr!rcvdat" outname="dt1" format="jsonf")
constant(value=", ")
#property(name="$!usr!company" outname="company"
format="jsonf") constant(value=", ")
* property(name="$!usr!field1" outname="field2" format="json")
constant(value=",")*
property(name="msg" format="jsonf" droplastlf="on" )
constant(value="}")
constant(value="}]} \n")
}
# the main attraction
ruleset(name="catalina-ruleset"
queue.type="fixedArray"
queue.size="250000"
) {
action(type="omfile" template="ulog-raw-format"
File="/apps/tomcat/logs/rawcatalina.out" ioBufferSize="64k"
flushOnTXEnd="off" asyncWriting="on")
action(type="mmnormalize" rulebase="/etc/rsyslog.d/txtrulebase.rb"
userawmsg="on")
action(type="omfile" template="ulog-text-format"
File="/apps/tomcat/logs/textp.out" ioBufferSize="64k" flushOnTXEnd="off"
asyncWriting="on")
}
input(type="imfile" File="/apps/tomcat/logs/catalina.out" Tag="catalina"
ruleset="catalina-ruleset" readMode="2" escapeLF="off"
reopenOnTruncate="on")
Lavanya K
On Tue, Dec 11, 2018 at 3:32 PM Lavanya Kanchanapalli <
[email protected]> wrote:
I vaguely remember reading about it. This is great! Let me give that
a try.
Thank you,
Lavanya K
On Tue, Dec 11, 2018 at 3:28 PM Rich Megginson via rsyslog <
[email protected]> wrote:
Indeed there is http://www.liblognorm.com/files/manual/lognormalizer.html
and it is incredibly useful
On 12/11/18 4:18 PM, Joe Blow via rsyslog wrote:
Isn't there a lognormalize command you can use to test this stuff
with? I
seem to remember being able to specify a rules file, and a log file,
and it
will walk the logs and let you know which characters are wrong for your
rules.
Cheers,
JB
On Tue, Dec 11, 2018 at 3:31 PM David Lang <[email protected]> wrote:
On Tue, 11 Dec 2018, Lavanya Kanchanapalli via rsyslog wrote:
Hi,
I am trying to get mmnormalize work with examples given in liblognorm
documentation
<
https://github.com/rsyslog/liblognorm/blob/master/doc/configuration.rst#json
.
I tried a couple but could not get it to work. What is missing or how
can I
get mmnormalize to parse text as expected?
in most cases $msg starts with a space, so in your rules you need a
space
after
the :
try that and then we can go from there
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
