Hi Rich: My apps and rsyslog are running on Ubuntu (Xenial). Installing rsyslog-mmnormalize worked. In my searched on Internet I did not find any reference to rsyslog-mmnormalize and so tried independently installing liblognorm and its dependencies which obviously did not work.
One other question, would you know how rsyslog determine which rule in rulebase to apply to a particular log line? My application logs can have logs from tomcat and a couple of other libraries with different formats. I am trying to figure out how to identify a log line is different format and handle it accordingly. Thanks a bunch for your help. Lavanya K On Wed, Dec 5, 2018 at 9:16 AM Rich Megginson via rsyslog < [email protected]> wrote: > On 12/5/18 10:11 AM, Lavanya Kanchanapalli via rsyslog wrote: > > Hi, > > > > I am trying to use rsyslog to parse and structure unstructured text > > application logs (tomcat logs). Based on the documentation and example > > rsyslog recipes > > <https://www.rsyslog.com/log-normalization-for-different-formats/> > mmnormalize > > seems to very well suited for this purpose. > > > > Both with rsyslog 8.16.x and with 8.39.0 when trying to use mmnormalize > as > > an action the module was not recognized. > > > What is your platform? On RHEL/CentOS/Fedora and similar platforms, the > rsyslog-mmnormalize is a separate RPM that must be installed separately e.g. > > yum install rsyslog rsyslog-mmnormalize .... > > > > > > How can I set up to use mmnormalize with rsyslog? Details and logs below. > > > > *syslog log showing issue with rsyslog config* > > > > Dec 3 11:33:55 sys1 systemd[1]: Starting System Logging Service... > > > > Dec 3 11:33:55 sys1 systemd[1]: Started System Logging Service. > > > > Dec 3 11:33:55 sys1 rsyslogd: could not load module 'mmnormalize', > errors: > > trying to load module /usr/lib/rsyslog/mmnormalize.so: > > /usr/lib/rsyslog/mmnormalize.so: > > cannot open shared object file: No such file or directory [v8.39.0 try > > http://www.rsyslog.com/e/2066 ] > > > > Dec 3 11:33:55 sys1 rsyslogd: module name 'mmnormalize' is unknown > [v8.39.0 > > try http://www.rsyslog.com/e/2209 ] > > > > Dec 3 11:33:55 sys1 rsyslogd: error during parsing file > > /etc/rsyslog.d/52-tomcat.conf, > > on or before line 52: errors occured in file > > '/etc/rsyslog.d/52-tomcat.conf' around line 52 [v8.39.0 try > > http://www.rsyslog.com/e/2207 ] > > > > *Remediation already tried* > > I tried the following but neither of them did not help solve the issue. > > - Installed separately liblognorm, libstr, json-c,libfastjson (not > sure if > > I linked these to rsyslog correctly) > > - Upgraded rsyslog from 8.16.x to 8.39.0 > > > > *rsyslog .conf* > > # Load Modules > > module (load="imfile" mode="inotify") > > module (load="builtin:omfile" dirCreateMode="0775" fileCreateMode="0664" > ) > > module(load="mmnormalize") # text parsing > > > > input(type="imfile" File="/apps/tomcat/logs/catalina.out" Tag="catalina" > > ruleset="catalina-ruleset" readMode="2" escapeLF="off" > > reopenOnTruncate="on") > > > > ruleset(name="catalina-ruleset" ) { > > action(type="omfile" template="ulog-raw-format" > > File="/apps/tomcat/logs/rawcatalina.out") > > action(type="mmnormalize" rulebase="/etc/txtrulebase.rb" > userawmsg="on") > > action(type="omfile" template="text-format" > > File="/apps/tomcat/logs/textp.out") > > } > > > > template(name="ulog-text-format" type="list") { > > constant(value="{") > > property(outname="appName" name="procid" format="jsonf") > > constant(value=", ") > > property(outname="hostname" name="hostname" format="jsonf") > > constant(value=", ") > > > > constant(value="\"event\":[{") > > constant(value="\"payload\":{") > > property(name="structured-data") constant(value=", ") > > property(name="$!usr!rcvdat" format="jsonf") > constant(value=", > > ") > > #property(name="$!usr!lvl2" format="jsonf") > constant(value=", ") > > property(name="msg" format="jsonf" droplastlf="on" ) > > constant(value="}") > > constant(value="}]} \n") > > } > > > > *rulebase.rb* > > version=2 > > rule=rfc5424:%rcvdat:date-rfc5424% %lvl2:word% %msg2:rest% > > > > Thank you, > > Lavanya K > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
