That tool is awesome! It confirmed my suspicion that mmnormalizer is doing what it is supposed to. But my rsyslog conf is not able to retrieve the parsed and structured text.
Below is rsyslog configuration I am using (mmnormalize property line highlighted). According to this recipe example <https://www.rsyslog.com/log-normalization-for-different-formats/>, values parsed by mmnormalize are added under "usr" sub-tree. But that does not seem to work. What is a way to access log line parsed by mmnormalize? *rsyslog.conf* # Load Modules module (load="imfile" mode="inotify") # for text file with inotify instead of pulling module (load="builtin:omfile" dirCreateMode="0775" fileCreateMode="0664" ) module(load="mmnormalize") # text parsing # RAW - don't format template(name="ulog-raw-format" type="list") { property(name="msg" droplastlf="on") constant(value="\n") } # Template for plain text formatted log messages. template(name="ulog-text-format" type="list") { constant(value="{") property(outname="appName" name="procid" format="jsonf") constant(value=", ") property(outname="hostname" name="hostname" format="jsonf") constant(value=", ") constant(value="\"event\":[{") #property(outname="uuid" name="msgid" format="jsonf") constant(value=", ") constant(value="\"payload\":{") property(outname="_log_" name="syslogtag" format="jsonf") constant(value=", ") #property(name="$!usr!rcvdat" outname="dt1" format="jsonf") constant(value=", ") #property(name="$!usr!company" outname="company" format="jsonf") constant(value=", ") * property(name="$!usr!field1" outname="field2" format="json") constant(value=",")* property(name="msg" format="jsonf" droplastlf="on" ) constant(value="}") constant(value="}]} \n") } # the main attraction ruleset(name="catalina-ruleset" queue.type="fixedArray" queue.size="250000" ) { action(type="omfile" template="ulog-raw-format" File="/apps/tomcat/logs/rawcatalina.out" ioBufferSize="64k" flushOnTXEnd="off" asyncWriting="on") action(type="mmnormalize" rulebase="/etc/rsyslog.d/txtrulebase.rb" userawmsg="on") action(type="omfile" template="ulog-text-format" File="/apps/tomcat/logs/textp.out" ioBufferSize="64k" flushOnTXEnd="off" asyncWriting="on") } input(type="imfile" File="/apps/tomcat/logs/catalina.out" Tag="catalina" ruleset="catalina-ruleset" readMode="2" escapeLF="off" reopenOnTruncate="on") Lavanya K On Tue, Dec 11, 2018 at 3:32 PM Lavanya Kanchanapalli < [email protected]> wrote: > I vaguely remember reading about it. This is great! Let me give that > a try. > Thank you, > Lavanya K > > > On Tue, Dec 11, 2018 at 3:28 PM Rich Megginson via rsyslog < > [email protected]> wrote: > >> Indeed there is http://www.liblognorm.com/files/manual/lognormalizer.html >> and it is incredibly useful >> >> On 12/11/18 4:18 PM, Joe Blow via rsyslog wrote: >> > Isn't there a lognormalize command you can use to test this stuff >> with? I >> > seem to remember being able to specify a rules file, and a log file, >> and it >> > will walk the logs and let you know which characters are wrong for your >> > rules. >> > >> > Cheers, >> > >> > JB >> > >> > On Tue, Dec 11, 2018 at 3:31 PM David Lang <[email protected]> wrote: >> > >> >> On Tue, 11 Dec 2018, Lavanya Kanchanapalli via rsyslog wrote: >> >> >> >>> Hi, >> >>> I am trying to get mmnormalize work with examples given in liblognorm >> >>> documentation >> >>> < >> >> >> https://github.com/rsyslog/liblognorm/blob/master/doc/configuration.rst#json >> >>> . >> >>> I tried a couple but could not get it to work. What is missing or how >> >> can I >> >>> get mmnormalize to parse text as expected? >> >> in most cases $msg starts with a space, so in your rules you need a >> space >> >> after >> >> the : >> >> >> >> try that and then we can go from there >> >> >> >> David Lang >> >> _______________________________________________ >> >> rsyslog mailing list >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> http://www.rsyslog.com/professional-services/ >> >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> myriad >> >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> >> DON'T LIKE THAT. >> >> >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com/professional-services/ >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >> you DON'T LIKE THAT. >> >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
