Sent from phone, thus brief. Am 02.04.2015 22:30 schrieb "David Lang" <[email protected]>: > > I have a couple systems that are sending raw JSON, but I'm also looking at interacting with logstash and nxlog systems, and they like to use raw JSON on a TCP port
Do you know about the framing? Is it all one line? > > an example of such messages is at http://bladesdev.com/samplejson.htm > > I can send you some of the ones I've run into in the field from other sources directly. That would be good. A handful different ones would be sufficient. Just want to see the actual details. > > David Lang > > On Thu, 2 Apr 2015, Rainer Gerhards wrote: > >> Date: Thu, 2 Apr 2015 22:15:25 +0200 >> From: Rainer Gerhards <[email protected]> >> Reply-To: rsyslog-users <[email protected]> >> To: rsyslog-users <[email protected]> >> Subject: Re: [rsyslog] JSON in rsyslog >> >> >> Side-note: do you have some examples? >> >> Sent from phone, thus brief. >> Am 02.04.2015 22:09 schrieb "Rainer Gerhards" <[email protected]>: >> >>> The non obvious one sounds interesting. But bear a bit with me, the >>> backlog is currently really bad and thus week had a lot of surprises;) >>> >>> Rainer >>> >>> Sent from phone, thus brief. >>> Am 02.04.2015 20:24 schrieb "David Lang" <[email protected]>: >>> >>>> cee appears to be defunct, but I am seeing more use of JSON in logs, but >>>> the way it's being used isn't meshing well with stock rsyslog. >>>> >>>> I'm seeing several tools that are sending raw JSON to the log port (no >>>> pri, timestamp, hostname, syslogtag or cee cookie, just {stuff} >>>> >>>> I see two obvious approaches to better handling of this sort of thing and >>>> one not-so-obvious approach that I think may actually be a better long-term >>>> option. >>>> >>>> 1. modify mmjsonparse to have the @cee cookie be optional, and gain the >>>> source and destination options that mmlognormalize has, then have rules in >>>> rsyslog to detect that $rawmsg startswith "{" and invoke the mmjsonparse >>>> against $rawmsg >>>> >>>> 2. create a new pmjson module that invokes json parsing against the raw >>>> message and populates the other standard properties >>>> >>>> then the non-obvious solution is in two parts (both of which I think are >>>> useful independently, and potentially extremely powerful when combined) >>>> >>>> two parts. >>>> >>>> 1. add an extension to liblognorm to let it be able to parse JSON into a >>>> tree (the way it does with descent, but without the current bug there) >>>> >>>> 2. allow a parser() definition to be used to create a parser that uses >>>> liblognorm (in addition to populating the rawmsg, fromhost-ip, fromhost, >>>> inputname, and timegenerated properties) and allow that to be bound to a >>>> ruleset/input for parsing. >>>> >>>> thoughts? >>>> >>>> David Lang >>>> >>>> P.S. I'm also wondering if it may be that in the longer term, it could be >>>> a win to replace some of the existing internal parsers with liblognorm >>>> definitions, since the parse tree is such an efficient parser. >>>> >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com/professional-services/ >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>> DON'T LIKE THAT. >>>> >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
