Hi David,
thanks for the tip. I can now understand what Rsyslog is doing when it
receives the data :)
Since the appliance is sending the JSON in a multi-line format, the last line
isn't being interpreted as the message, but rather it's being interpreted as
all the other fields.
Since I can't modify the way the appliance sends the logs, is there a way in
Rsyslog to get around this?
Cheers,
Duarte
On Monday 02 June 2014 10:56:51 David Lang wrote:
> could you write a log with the format RSYSLOG_DebugFormat? that will show
> the raw message that arrives.
>
> Also, when you say the data in the file is missing the closing }, is there
> any chance that it's just buffered and not written to disk yet? or are
> there other things in the log after this?
>
> David Lang
>
> On Mon, 2 Jun 2014, Duarte Silva wrote:
> > Date: Mon, 02 Jun 2014 16:22:29 +0100
> > From: Duarte Silva <[email protected]>
> > Reply-To: rsyslog-users <[email protected]>
> > To: [email protected]
> > Subject: [rsyslog] Problem with rsyslog deleating
> >
> > Hi,
> >
> > when sending JSON formated data to rsyslog, the closing "}" character in
> > the last line of the sent data isn't written into file. In tcpdump I can
> > clearly see the full message arriving:
> >
> > # tcpdump -A -n -vvv -s2048 -i eth0 "tcp port 514"
> > (...)
> > =.2.....<164>appliance-892.alert: {
> >
> > "alert": {
> >
> > "action": "notified",
> > "explanation": "something"
> > },
> > "id": "892",
> > "occurred": "2014-06-02T15:00:24Z",
> >
> > },
> > "appliance": "appliance",
> > "product": "product",
> > "version": "version"
> >
> > }
> > .
> >
> > But when rsyslog writes the message in the file, the output is:
> >
> > # tail -n 50 /var/log/test.log
> > (...)
> > {
> >
> > "alert": {
> >
> > "action": "notified",
> > "explanation": "something"
> > },
> > "id": "892",
> > "occurred": "2014-06-02T15:00:24Z",
> >
> > },
> > "appliance": "appliance",
> > "product": "product",
> > "version": "version"
> >
> > The configuration is as follows:
> >
> > # cat /etc/rsyslog.d/test.conf
> >
> > $template msgonlytemplate,"%msg%\n"
> >
> > if $fromhost-ip == '1.1.1.1' then /var/log/test.log;msgonlytemplate
> > & ~
> >
> >
> > I'm currently running rsyslog-5.8.10-8.el6.x86_64. Any help will be
> > appreciated.
> >
> > Thanks in advance,
> > Duarte Silva
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> > DON'T LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> LIKE THAT.
Debug line with all properties:
FROMHOST: 'appliance', fromhost-ip: '2.2.2.2', HOSTNAME: 'appliance', PRI: 164,
syslogtag 'appliance-898.alert:', programname: 'appliance-898.alert', APP-NAME:
'appliance-898.alert', PROCID: '', MSGID: '-',
TIMESTAMP: 'Jun 3 11:28:36', STRUCTURED-DATA: '-',
msg: ' {'
escaped msg: ' {'
inputname: imtcp rawmsg: '<164>appliance-898.alert: {'
Debug line with all properties:
FROMHOST: 'appliance', fromhost-ip: '2.2.2.2', HOSTNAME: 'appliance', PRI: 13,
syslogtag '', programname: '', APP-NAME: '', PROCID: '', MSGID: '-',
TIMESTAMP: 'Jun 3 11:28:36', STRUCTURED-DATA: '-',
msg: ' "alert": {'
escaped msg: ' "alert": {'
inputname: imtcp rawmsg: ' "alert": {'
Debug line with all properties:
FROMHOST: 'appliance', fromhost-ip: '2.2.2.2', HOSTNAME: 'appliance', PRI: 13,
syslogtag '', programname: '', APP-NAME: '', PROCID: '', MSGID: '-',
TIMESTAMP: 'Jun 3 11:28:36', STRUCTURED-DATA: '-',
msg: ' "action": "notified", '
escaped msg: ' "action": "notified", '
inputname: imtcp rawmsg: ' "action": "notified", '
Debug line with all properties:
FROMHOST: 'appliance', fromhost-ip: '2.2.2.2', HOSTNAME: 'appliance', PRI: 13,
syslogtag '', programname: '', APP-NAME: '', PROCID: '', MSGID: '-',
TIMESTAMP: 'Jun 3 11:28:36', STRUCTURED-DATA: '-',
msg: ' "explanation": "something"'
escaped msg: ' "explanation": "something'
inputname: imtcp rawmsg: ' "explanation": "something'
Debug line with all properties:
FROMHOST: 'appliance', fromhost-ip: '2.2.2.2', HOSTNAME: 'appliance', PRI: 13,
syslogtag '', programname: '', APP-NAME: '', PROCID: '', MSGID: '-',
TIMESTAMP: 'Jun 3 11:28:36', STRUCTURED-DATA: '-',
msg: ' }, '
escaped msg: ' }, '
inputname: imtcp rawmsg: ' }, '
Debug line with all properties:
FROMHOST: 'appliance', fromhost-ip: '2.2.2.2', HOSTNAME: 'appliance', PRI: 13,
syslogtag '', programname: '', APP-NAME: '', PROCID: '', MSGID: '-',
TIMESTAMP: 'Jun 3 11:28:36', STRUCTURED-DATA: '-',
msg: ' "id": "898", '
escaped msg: ' "id": "898", '
inputname: imtcp rawmsg: ' "id": "898", '
Debug line with all properties:
FROMHOST: 'appliance', fromhost-ip: '2.2.2.2', HOSTNAME: 'appliance', PRI: 13,
syslogtag '', programname: '', APP-NAME: '', PROCID: '', MSGID: '-',
TIMESTAMP: 'Jun 3 11:28:36', STRUCTURED-DATA: '-',
msg: ' "appliance": "appliance", '
escaped msg: ' "appliance": "appliance", '
inputname: imtcp rawmsg: ' "appliance": "appliance", '
Debug line with all properties:
FROMHOST: 'appliance', fromhost-ip: '2.2.2.2', HOSTNAME: 'appliance', PRI: 13,
syslogtag '', programname: '', APP-NAME: '', PROCID: '', MSGID: '-',
TIMESTAMP: 'Jun 3 11:28:36', STRUCTURED-DATA: '-',
msg: ' "version": "version"'
escaped msg: ' "version": "version"'
inputname: imtcp rawmsg: ' "version": "version"'
Debug line with all properties:
FROMHOST: 'appliance', fromhost-ip: '2.2.2.2', HOSTNAME: 'appliance', PRI: 13,
syslogtag '}', programname: '}', APP-NAME: '}', PROCID: '', MSGID: '-',
TIMESTAMP: 'Jun 3 11:28:36', STRUCTURED-DATA: '-',
msg: ''
escaped msg: ''
inputname: imtcp rawmsg: '}'_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
