This is why I love opensource software, comment on a need and someone will point
at a tool.
Thanks!
wq
On Wed, 15 Apr 2015, Brian Knox wrote:
We keep our logs in JSON format and don't find it to be a drawback. We
have logs searchable in elasticsearch - and for working with logs on disk,
have a small program that logs can be piped through that strips out
everything but the json which makes it very easy to pipe logs to jq (a
command line json processor - see https://stedolan.github.io/jq/ ).
On Wed, Apr 15, 2015 at 12:48 AM, David Lang <[email protected]> wrote:
On Wed, 15 Apr 2015, Ezell, Matthew A. wrote:
Sure, as a system administrator it's pretty clear how best to handle this.
If there's CEE JSON data coming over the wire, use mmjsonparse. If it's
unstructured traditional syslog(3) data, use mmnormalize to try to extract
relevant fields based on rules I setup. Write the traditional "message"
field to /var/log/messages and send the structured data to ElasticSearch.
But I'm a system administrator who cares about structured logging, so I
would have a custom rsyslog setup to handle this seamlessly.
The question is really from the application developer's point of view.
How do you log structured data in a way that doesn't change the format of
/var/log/messages for most users, but provides additional information for
those system administrators who choose to handle the structured data?
Imagine going to the developers of OpenSSH and requesting that they start
logging structured data. If they simply changed all their syslog(3) calls
to output CEE JSON instead instead of plain strings, it's going to break
just about every brute-force login detection system out there. That's
unacceptable. What is the *right* thing for them to do?
do like ossec does and have a config option that switches to JSON output.
since they have to have their software work everywhere that it's working
today, they can't change it's output at all. anything they do will break
parsers.
but with a config switch (which a distro could turn on by default), they
can output a different format, and that format could be JSON with the old
log text in a msg field (again though, which is the source of truth if they
differ)
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
