We keep our logs in JSON format and don't find it to be a drawback. We have logs searchable in elasticsearch - and for working with logs on disk, have a small program that logs can be piped through that strips out everything but the json which makes it very easy to pipe logs to jq (a command line json processor - see https://stedolan.github.io/jq/ ).
On Wed, Apr 15, 2015 at 12:48 AM, David Lang <[email protected]> wrote: > On Wed, 15 Apr 2015, Ezell, Matthew A. wrote: > > Sure, as a system administrator it's pretty clear how best to handle this. >> If there's CEE JSON data coming over the wire, use mmjsonparse. If it's >> unstructured traditional syslog(3) data, use mmnormalize to try to extract >> relevant fields based on rules I setup. Write the traditional "message" >> field to /var/log/messages and send the structured data to ElasticSearch. >> But I'm a system administrator who cares about structured logging, so I >> would have a custom rsyslog setup to handle this seamlessly. >> >> The question is really from the application developer's point of view. >> How do you log structured data in a way that doesn't change the format of >> /var/log/messages for most users, but provides additional information for >> those system administrators who choose to handle the structured data? >> >> Imagine going to the developers of OpenSSH and requesting that they start >> logging structured data. If they simply changed all their syslog(3) calls >> to output CEE JSON instead instead of plain strings, it's going to break >> just about every brute-force login detection system out there. That's >> unacceptable. What is the *right* thing for them to do? >> > > do like ossec does and have a config option that switches to JSON output. > > since they have to have their software work everywhere that it's working > today, they can't change it's output at all. anything they do will break > parsers. > > but with a config switch (which a distro could turn on by default), they > can output a different format, and that format could be JSON with the old > log text in a msg field (again though, which is the source of truth if they > differ) > > David Lang > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
