you are talking about different parsers
ruleset parsers take the data arriving over the wire, figure out where the end
of the message is, and what part is the timestamp, hostname, etc. This populates
the normal properties in rsyslog.
you then can use mmnormalize/mmjsonparse to do additional parsing of the
message.
Until some other parser extracts meaning out of the bytes arriving over the
wire, there is nothing for mmnormalize to work from, you can't even tell where
one message finishes and the next one starts.
The reason why it's strongly recommended to leave the RFC3164 parser in the list
is that it's designed to extract as much meaning as possible from complete
garbage, there is no message that the rfc3164 parser will say cannot be handled.
So if you don't list the rfc3164 last, the other parsers will never be applied
(the rfc3164 parser will parse the message and that will be that). All the other
parsers are very picky, and will only successfully parse messages that exactly
comply with their specs.
does this clarify things a bit.
David Lang
On Thu, 10 Dec 2015, Alec Swan wrote:
That should work. But then I don't understand why it's always recommended
to add RFC3164 parser at the end (
http://www.rsyslog.com/doc/master/configuration/ruleset/rsconf1_rulesetparser.html)
in order to avoid dropping messages? With your approach messages are always
parsed either by mmnormalize or <parse failed stuff>.
On Thu, Dec 10, 2015 at 1:16 PM, David Lang <[email protected]> wrote:
On Thu, 10 Dec 2015, Alec Swan wrote:
I've read a several articles about the benefits of parser chains for
rulesets but couldn't find any examples for the new style of
configuration.
Here is my scenario. I need to parse logs from my-app.log file written by
an app. Recently, the format of this file changed slightly, e.g. extra
tabs
removed between parts of each log message. I am using mmnormalize to parse
the old format using old.rb liblognorm rule. I would like to be able to
use
new.rb rule to parse the log file in case old.rb rule fails.
What do I need to do in the following to accomplish this?
input(type = "imfile" File = "my-app.logs" Ruleset = "my-app")
ruleset(name = "my-app") {
action(type = "mmnormalize" rulebase = "/etc/rsyslog.d/rules/old.rb")
# here I would like to use new.rb in case the action above failed to
parse log file
action(type = "omfile" template="es-payload"
file="/tmp/my-app-parsed.log")
}
what I've resorted to doing is:
action(type="mmnormalize" ruleBase="/etc/rsyslog.rb")
if $!originalmsg != "" then {
<parse failed stuff>
}
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
