David, I had hope to get to this before departing, but looks bad now. It's a more complex request, as probably the complete workflow around parsers needs to be refactored. Currently, the design is that the parser stage is run *once* *before* message processing begins.
It would be great if you could open a github issue tracker, so that we don't forget about this case (piles of mail tend to be not good for tracking, at least for me ;)). Rainer 2014-12-06 0:27 GMT+01:00 David Lang <[email protected]>: > > On Sat, 6 Dec 2014, singh.janmejay wrote: > > David, can you please elaborate using an pseudo message and pseudo config? >> I think I get what you are saying, and was wondering about this a little >> while ago myself, but this will ensure all of us are on the same page. >> > > so, for the example from today's messages > > If the format sent is: > > <pri>syslogtag message > > this will get parsed in interesting ways depending on if syslogtag looks > like a valid hostname or not. > > the fix I suggested was > > $format, fixup1="<%pri%>%fromhost% %hostname% %syslogtag%%msg%\n" > if $fromhost-ip == '1.2.3.4' then { > @remote;fixup1 > stop > } > > if you want to output to a local file as well, this becomes: > > $format, fixup1="<%pri%>%timestamp% %fromhost% %hostname% > %syslogtag%%msg%\n" > $format, fixup2="%timestamp% %fromhost% %hostname% %syslogtag%%msg%\n" > if $fromhost-ip == '1.2.3.4' then { > @remote;fixup1 > /var/log/messages;fixup2 > stop > } > > If you have multiple different types of fixups, are doing complicated > things ith the logs this gets _really_ ugly. > > And if you can't have multiple outputs to the same thing you end up > needing to do something like: > > set $.mymessage = format(); > > in many places, then > > $format myformat="$.mymessage" > > and then use myformat in your outputs > > Instead I am suggesting allowing > > if $fromhost-ip == '1.2.3.4' then { > set $.myraw = format("<%pri%>%timestamp% %fromhost% %hostname% > %syslogtag%%msg%\n"); > reparse($.myraw) > } > > This would take the contents of $.myraw (crafted by the admin), put it in > $raw, clear $. and $! and then run the parser stack against the 'new' raw > message (correctly populating the derived properties) > > so after the reparse() call, $hostname would have the data previously in > $fromhost, $syslogtag would have the data previously in $hostname, etc. > > is this clearer? > > David Lang > > > > On Fri, Dec 5, 2014 at 11:35 PM, David Lang <[email protected]> wrote: >> >> the question about how to fix up a message prompted a thought. This is a >>> pretty common problem, and it can be dealt with by creating a custom >>> parsing module, or a custom message modification module, but most of the >>> time the fixups that are needed are pretty simple. >>> >>> so how about adding a reparse($!var) function that would take the >>> contents >>> of $!var, put it in $rawmsg and run the parser stack against it? >>> >>> This would allow people to do a lot of the common fixups with a few >>> normal >>> rsyslog commands and then let the normal parsers populate all the >>> variables. >>> >>> With this approach, there would be a fixup section at the top of the >>> config that would clean up the messages, and then clean logic to output >>> the >>> messages. Currently when you have this sort of thing, you end up with a >>> bunch of sections to handle individual broken types of messages with a >>> bunch of custom templates, so outputs end up getting specified many >>> times. >>> >>> David Lang >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>> DON'T LIKE THAT. >>> >>> >> >> >> >> _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
