Hello,
I have been looking at the liblognorm manual, trying liblognorm-utils
2.0.3 from Ubuntu 18.04, and I have a few questions.
1. Is there an epsilon ("match nothing") transition, or a simple way to
mark a part as optional?
Example: let's say a log line may be prefixed by an optional sequence
number, followed by colon space. I can write it explicitly as two
rules, matching without and with prefix, and that works:
==> test.rb <==
version=2
prefix=
rule=:sshd[%pid:number%]: Login failed
prefix=%sequence:number%:\x20
rule=:sshd[%pid:number%]: Login failed
==> test.log <==
sshd[1234]: Login failed
12345: sshd[1234]: Login failed
I just wondered if there was a cleaner way to do this, e.g. using
'alternative'.
(Note: the DAG generated from test.rb above does not merge the tails.
There are two branches which separately match the sshd[...]... part)
2. I wondered what the best practice is for recording which rule
matched, e.g. for metrics ("this pattern matched N times").
-oaddRuleLocation is not stable if the rulebase changes. I can think of
a couple of options:
a. for every rule of interest, add a unique tag. It could be of the form
'id=123'
b. use -oaddRule; hash the rule which matches and take the first ~8 hex
digits (like a git commit identifier)
I find option (b) appealing since it's quite stable - e.g. it doesn't
care about the name of user-generated types - and is zero-touch. Any
other suggestions or experience to share?
Aside: I tried looking for downstream projects which use liblognorm to
see how they deal with this. I found Sagan, but AFAICS its lognorm
rulebase
<https://github.com/beave/sagan-rules/blob/master/normalization.rulebase>
is only used for extracting fields (with hard-coded names like 'src-ip
<https://github.com/beave/sagan/blob/master/src/liblognormalize.c#L154>')
and it doesn't care or remember which rule matched. It seems to use its
own PCRE matching to trigger actual alerts, and it doesn't set any
LN_CTXOPT options.
Finally, a couple of minor notes about implementation:
3. If a rulebase contains a field of an unknown type, a warning is
generated but processing continues anyway with a partial/broken rule.
Is that intentional? Is there a way to run a check over a rulebase and
get a pass/fail status?
Example: in the test.rb above, in the prefix line change \x20 to
%-:space% (which is wrong: it should be %-:whitespace%). The results
are particularly confusing if the broken entry is a prefix line, because
part of the prefix is still consumed:
{ "originalmsg": "12345: sshd[1234]: Login failed", "unparsed-data": "
sshd[1234]: Login failed" }
4. I think there is an error in lognormalize optarg parser or help text,
which says:
-d Print DOT file to stdout and exit
-d<filename> Save DOT file to the filename
However, if you try to use -d without an argument, you get an error
saying the argument is required. (It works with -d "" though).
Regards,
Brian.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
