hi all,
my apologies if this question has already been asked before and there is a
solution which I failed to find from rsyslog mailing list archive.
I have to parse syslog messages with the message field consisting of
keyword=value pairs, where values may be enclosed in double-quotes. Also,
values may contain spaces (in that case, surrounding double-quotes are
mandatory). In addition, a double-quote may appear within the value,
provided it has been masked with the backslash.
Here is an example of a valid message:
firstkey=value secondkey="several words" thirdkey="doublequote\"present"
There is an additional important requirement -- when parsing such messages,
field names must be created from keyword names of the event. In other
words, parsing result for the example event should look like this:
{ "logdata": { "firstkey": "value", "secondkey":"several words",
"thirdkey":"doublequote\"present" } }
or perhaps:
{ "logdata": { "firstkey": "value", "secondkey":"\"several words\"",
"thirdkey":"\"doublequote\\\"present\"" } }
>From the mailing list archive, I have found a recommendation to use the
'name-value-list' parser, and here is a simple rule I have created:
rule=:%logdata:name-value-list%
However, when lognormalizer is used for testing this rule against above
example message, parsing will fail. When testing this rule against each
individual keyword=value pair, results are the following:
cat examples.log | lognormalizer -r test.rb -e json
{ "logdata": { "firstkey": "value" } }
{ "originalmsg": "secondkey=\"several words\"", "unparsed-data":
"secondkey=\"several words\"" }
{ "logdata": { "thirdkey": "\"doublequote\\\"present\"" } }
In other words, the first and third keyword=value pair are parsed
successfully, while parsing fails for the second pair (probably because the
value contains a space character).
I have also tried v2-iptables parser, but it fails for all pairs. According
to documentation, all keywords have to be uppercase and spaces are not
permitted in values, so v2-iptables is clearly not applicable for my
scenario.
And here comes my question -- can this parsing problem be solved with
mmnormalize or some other module, and if so, can you provide relevant
examples or recommendations?
kind regards,
risto
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
