Ciprian and David, thank you for the great responses. I'll give your recommendations a try and post back if I run into problems.
Thanks, Alec On Thu, Nov 19, 2015 at 11:57 AM, David Lang <[email protected]> wrote: > On Thu, 19 Nov 2015, Alec Swan wrote: > > I am assuming your first suggestion was to use a Tag in imfile input and >> then filter on that Tag in mmnormalize action. How do I access the tag >> that >> was assigned in imfile? >> > > the tag in the imfile becomes $programname > > if $programname='foo' then {} > > I don't understand how I can chain multiple mmnormalize rules if some of >> them can match logs from different files, e.g. each file contains three >> words but each word has different semantics? >> > > in almost all cases, there are other words around the fields that you need > to extract, but if not, there are a couple things you can do. > > first, write some logs with the template RSYSLOG_DebugFormat so that you > see all the variables that exist. Specifically look at the rawmsg variable. > If it contains additional information that makes the logs unambiguous, you > can tell mmnormalize to work against the rawmsg instead of msg variable. > > The other thing you can do is to create your own variable and have > mmnormalize parse from that. > > For example, I deliver my logs to my central server encapsulated in JSON. > > so an example log that arrives would be: > > <46>Nov 19 10:17:01 linux-dev01 rsyslogd: @cee:{ "msg": "[origin > software=\"rsyslogd\" swVersion=\"8.10.0.master\" x-pid=\"417\" > x-info=\"http:\/\/www.rsyslog.com\"] rsyslogd was HUPed", "trusted": { > "origserver": "10.1.2.2", "edge": { "syslogtag": "rsyslogd:", "hostname": > "linux-dev01", "facility": "syslog", "severity": "info", "time": "Nov 19 > 10:17:01", "relay": "ISEC27", "input": "imudp", "event.tags": null }, > "environment": "CJP-Prod" } } > > I then have a template that extracts and recreates the 'original' log > message into a variable $.stdlog > > $template stdmsg,"%timereported% %hostname% > %syslogtag%%$!msg:::sp-if-no-1st-sp%%$!msg:::escape-cc%" > set $.stdmsg = exec_template("stdmsg"); > action(type="mmnormalize" path="$.extracted" variable="$!msg" > ruleBase="/root/rsyslog.rulebase.json" name="norm-json") > > $.stdlog looks like: > > Nov 19 10:17:01 linux-dev01 rsyslogd: [origin software="rsyslogd" > swVersion="8.10.0.master" x-pid="417" x-info="http://www.rsyslog.com";] > rsyslogd was HUPed > > and I then can have rules in my rulebase that work against it. > > you can create a variable yourself with whatever context you need and do > something similar. > > David Lang > > > Thanks, >> >> Alec >> >> On Thu, Nov 19, 2015 at 11:00 AM, David Lang <[email protected]> wrote: >> >> On Thu, 19 Nov 2015, Alec Swan wrote: >>> >>> Hello, >>> >>>> >>>> I would like to parse multiple log files with mmnormalize actions. >>>> However, >>>> it is not clear how to specify different parsing rules for different >>>> files >>>> in a single rsyslog .conf file. If this is possible could anybody >>>> provide >>>> a >>>> sample configuration? >>>> >>>> >>> well, you could do it by having a series of if statements that check the >>> input or tag. >>> >>> But you really shouldn't need to do this. If you just combine the >>> definitions you should be able to have things work. >>> >>> mmnormalize is designed to be fast and work with very large rulesets >>> (unlike most regex based systems, which need to have you split up the >>> rules). I currently have >1300 rules in my rulebase for example. >>> >>> David Lang >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>> DON'T LIKE THAT. >>> >>> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> >> _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
