Hello.
Is it OK to discuss liblognorm here?
This approach to log parsing seems attractive to me. In it's current
state, though, it not very usable for highload, and if there is no high
load, then one can use regexps to do the same. So I would like to extend
the idea to something fitting our purposes, instead of writing custom
parsing module.
There are a few shortcomings now:
1. Liblognorm is using libee for parsing and event handling, then the
event gets converted to json and imported to json-c structures. It is
declared as inefficient. I'll do my own tests of how inefficient it is
in reality. Then, what is preferred way of overcoming it? Liblognorm
could be extended to support json-c natively, or it could present some
callback interface to populate fields in mmnormalize. It is questionable
if we should continue to use libee, then. Or libee could be rewritten to
use json-c, maybe...
2. Liblognorm is unable to match last part of a string in some cases.
There is no field type which could fit anything till the end of string.
This quirk maybe arise from some ideology, but it makes impossible, for
example, to parse common CSV format, unless last field fits some of
predefined field types by accident. Currently, parsers are defined in
libee, and there is no interface to add one, which presents us with a
choice: extend libee or use own parsers. There can be other useful field
types, as well.
For the latter, what is the reason under these two restrictions in
char-to parser:
It is considered a format error if
* a) the to-be-parsed buffer is already positioned on the terminator
character
* b) there is no terminator until the end of the buffer
Both break CSV parsing.
--
Pavel Levshin
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
