here is a paper on the parsing performance
https://www.fernuni-hagen.de/rechnerarchitektur/docs/rainer_gerhards.pdf
On Fri, 9 Jul 2021, David Lang via rsyslog wrote:
Date: Fri, 9 Jul 2021 15:40:19 -0700 (PDT)
From: David Lang via rsyslog <[email protected]>
To: Jim Van Meggelen via rsyslog <[email protected]>
Cc: David Lang <[email protected]>
Subject: Re: [rsyslog] using Kibana / OpenSearch Dashboards to analyze logs
during development
also, dig into mmnormalize (liblognorm), it's a very efficient parse engine
for extracting values out of logs. The Dyn_stats() feature in rsyslog ends up
being a rather powerful tool for summarizing things (SEC is more powerful,
but you can do a lot with just dyn_stats() )
David Lang
On Fri, 9 Jul 2021, Jim Van Meggelen via rsyslog wrote:
Date: Fri, 9 Jul 2021 07:42:28 -0500 (CDT)
From: Jim Van Meggelen via rsyslog <[email protected]>
To: rsyslog-users <[email protected]>
Cc: Jim Van Meggelen <[email protected]>
Subject: Re: [rsyslog] using Kibana / OpenSearch Dashboards to analyze logs
during development
Daniel,
I'm pretty sure you and I have had at least one yap at some conference or
another. Could be I just attended a talk of yours.
I saw your name here and thought "I'm pretty sure I've met him somewhere",
and that was somewhat of a pleasant shock, because I've been digging into
rsyslog for some stuff I've been thinking about, and it's in a similar vein
to what you're talking about here (feeling multi-line data into analytics
to help make some sense of it), and frankly it's nice to hear someone else
in the same line of work is thinking similar things with respect to these
log files (which are chock full of detailed data).
I don't know if what we're after is in fact the same (most folks seem to
use logging for error handling, whereas I'm thinking more about gleaning
business analytics from the data).
It feels like there's gold in all those log files. It'd be interesting to
see how it could be mined.
Regards,
Jim
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
