Hi,
Perhaps an option could be added to xm_syslog not to replace linebreaks
but most users want it to work out of the box and line breaks will cause
the syslog to be malformed on the receiver, unless it is UDP syslog.
The Syslog_TLS style framing is is even more uncommon.
Regards,
Botond
On Fri, 29 Jan 2016 15:59:34 +0100
Milan Krčmář <milan.krc...@gmail.com> wrote:
> Hello,
>
> the RFC5424 is quite liberal to changes of control characters in MSG
> part, probably for historical reasons, however, it does not allow for
> such a replacement (or avoidance or escaping) in PARAM-VALUE, which is
> arbitrary UTF-8 string.
>
> I think this is why RFC5425 introduced the MSG-LEN framing for
> stream-oriented TLS transports, we use the same framing for plain TCP
> as well - "InputType Syslog_TLS" in im_tcp etc.
>
> Note this is not just my excessive requirement to be RFC5424 compliant
> in every detail:
>
> - im_mseventlog and/or im_msvistalog already produce structured fields
> with line breaks in their values (only in a fraction of events, but
> from 20 servers in total, we are receiving such an event several times
> an hour),
>
> - software written in languages with exception handling often also
> generate events with line breaks in MSG part: the stack trace -- some
> syslog systems erroneously split such an event into several messages
> line by line - I am trying to avoid this.
>
> So what is our replacement for line breaks?
>
> - in datagram transport (UDP, Unix datagram socket) - the events are
> framed by the datagram,
>
> - in stream transport (TCP, TLS) - MSG-LEN framing,
>
> - in files - we are still using '\n' for the files to be well readable
> by humans (and to be similar to plain old syslog) -- when reading back
> such a file into NXLog, we use NXLog's xm_multiline module with
> "HeaderLine /^<\d{1,3}>1 (- |\d{4}-)/" - this splits the input into
> events with "high probability only", but more complicated regexps
> would not increase the probability anyway.
>
> For files, I have considered to escape line breaks in MSG as "\n" or
> "\r", but 1) it is not allowed to do such an escape in PARAM-VALUE, 2)
> it makes the multi-line events human-unreadable. So I ended up with
> the xm_multiline and "high probability". Much better than replacing
> line breaks with spaces, isn't it? :-)
>
> Milan
>
>
> On Wed, Jan 27, 2016 at 1:11 PM, Botond Botyanszki <b...@nxlog.org> wrote:
> > Hi,
> >
> > On Tue, 26 Jan 2016 20:35:53 +0100
> > Milan Krčmář <milan.krc...@gmail.com> wrote:
> >
> >> why does the to_syslog_ietf() replace line breaks ('\r','\n') in
> >> message text with space (' ')? [in nx_logdata_to_syslog_rfc5424(),
> >> almost at the end]
> > That's done because linebreaks represent the end-of-record when syslog is
> > transferred over TCP or TLS/SSL.
> >
> >> In RFC5424, the message text is allowed to contain any octets.
> > This is what RFC5424 says:
> > "6.4. MSG
> > The syslog application SHOULD avoid octet values below 32 (the
> > traditional US-ASCII control character range except DEL). These
> > values are legal, but a syslog application MAY modify these
> > characters upon reception. For example, it might change them into an
> > escape sequence (e.g., value 0 may be changed to "\0"). "
> >
> >> The IETF standard
> >> allows the line breaks to appear in structured data fields as well -
> >> these are preserved. So, in output, you might get line breaks in
> >> middle of whole event anyway.
> > This might need to be reviewed to protect against such issues.
> >
> > Regards,
> > Botond
> >
> > ------------------------------------------------------------------------------
> > Site24x7 APM Insight: Get Deep Visibility into Application Performance
> > APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> > Monitor end-to-end web transactions and take corrective actions now
> > Troubleshoot faster and improve end-user experience. Signup Now!
> > http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
> > _______________________________________________
> > nxlog-ce-users mailing list
> > nxlog-ce-users@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/nxlog-ce-users
>
> ------------------------------------------------------------------------------
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
> _______________________________________________
> nxlog-ce-users mailing list
> nxlog-ce-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nxlog-ce-users
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
nxlog-ce-users mailing list
nxlog-ce-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nxlog-ce-users
