Daniel, I'm pretty sure you and I have had at least one yap at some conference or another. Could be I just attended a talk of yours.
I saw your name here and thought "I'm pretty sure I've met him somewhere", and that was somewhat of a pleasant shock, because I've been digging into rsyslog for some stuff I've been thinking about, and it's in a similar vein to what you're talking about here (feeling multi-line data into analytics to help make some sense of it), and frankly it's nice to hear someone else in the same line of work is thinking similar things with respect to these log files (which are chock full of detailed data). I don't know if what we're after is in fact the same (most folks seem to use logging for error handling, whereas I'm thinking more about gleaning business analytics from the data). It feels like there's gold in all those log files. It'd be interesting to see how it could be mined. Regards, Jim -- Jim Van Meggelen ClearlyCore Inc. +1-416-639-6001 (DID) +1-877-253-2716 (Canada) +1-866-644-7729 (USA) +1-416-425-6111 x6001 [email protected] [ http://www.clearlycore.com/ | http://www.clearlycore.com ] Asterisk: The Definitive Guide FIFTH EDITION NOW AVAILABLE TO DOWNLOAD: [ https://cdn.oreillystatic.com/pdf/Asterisk_The_Definitive_Guide.pdf | https://cdn.oreillystatic.com/pdf/Asterisk_The_Definitive_Guide.pdf ] ----- Original Message ----- > From: "Daniel Pocock via rsyslog" <[email protected]> > To: [email protected] > Cc: "Daniel Pocock" <[email protected]> > Sent: Friday, 9 July, 2021 06:52:16 > Subject: [rsyslog] using Kibana / OpenSearch Dashboards to analyze logs > during development > One of my reasons for looking at rsyslog->OpenSearch integration is for > the analysis of logs during development of other free software projects, > especially in the VoIP and WebRTC space. > > Here is a quick summary: > > - we often have multi-line log messages, such as SIP or SDP message > bodies, JSON structured logging is very appropriate for these > > - modern WebRTC solutions always involve multiple processes (HTTP, TURN, > SIP, media), so there is a lot of benefit from merging all the logs in > some sort of server > > - we also have some other dedicated tools that can log to OpenSearch, > for example, the HOMER software[1] captures SIP as well as the RTCP > reports and lets us visualize them > > > I made some contributions to a few projects (pull requests GStreamer[2], > Kurento[3], committed in reSIProcate[4]) to log as JSON CEE. This means > all logs from the whole WebRTC stack can be aggregated and analyzed. > > Can anybody make any comments about how to get the best value out of the > data being captured? > > For example, apart from HOMER, are there any reports or front ends to > ElasticSearch / OpenSearch that are particularly useful in cases like > this, Syslog data in general? > > > > > 1. http://sipcapture.org/ > > 2. https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/847 > > 3. https://github.com/Kurento/kurento-media-server/pull/17 > > 4. > https://github.com/resiprocate/resiprocate/commit/d1df9aeb956be79253560fe3628b3f37a4ef94f2 > > _______________________________________________ > rsyslog mailing list > https://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE > THAT. _______________________________________________ rsyslog mailing list https://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
