2013/6/12 Rainer Gerhards <[email protected]> > On Wed, Jun 12, 2013 at 3:21 PM, Radu Gheorghe <[email protected] > >wrote: > > > Hi Rainer, > > > > Thanks a lot for your reply! And quick hints are all I need, as I can > > continue with researching and testing. > > > > Regarding tools: this is exactly why I decided to ask instead of testing, > > because if a test fails, I'm not sure whether it's the tools fault or > > rsyslog's. > > > > Tools aside, it seems like my options are: > > - use UDP with RFC5424 > > - stay away from multiline messages. I don't think this would be an > option > > for me, as I'd want to be able to send stacktraces over syslog. > > > > IMHO multiline messages in syslog (or logging in general) are broken > design. You get into so much trouble at so many places. I would concentrate > on fixing them up as early as possible, doing escaping in order to provide > them. There is no real standard to quote, but RFC5424 somewhat recommends > (only via example) #ooo with o being octal digits - or the well-known \n. > But... just my 2cts here. >
Yeah, I get what your saying. Unfortunately, getting rid of \n is beyond my control. Nor can I control how the logs are displayed or searched. So I need to accept multi-line logs, and store them as proper multi-line in Elasticsearch (that's where they end up for now). I can use escaping, but only as far as transporting logs is concerned. In other words, I would have to escape them and un-escape them in rsyslog. And I think it's better to actually handle multi-line logs than do that. > > > > > > Not so sure about the following: > > - syslog over TCP (5424 or not) is a no-no for multi-lined messages, > > because you need a delimiter. I could change the delimiter from \n to > > something else, but that might confuse a lot of things > > > > well... rsyslog supports the same octet-counted framing that RFC5425 > requires. With it, multiline messages *can* be transmitted without any > problems (as far as the network is concerned). > Aha! that's good to know! > > > > - what about TLS (RFC 5425)? I see nice keywords there like "message > lenth" > > and "syslog frame". This should mean \n delimiters are not required, so > one > > could use this for sending multi-lined messages. > > > > > yup > > You may also want to have a look at RFC 6587, where we elaborate about the > multiline problem and framing. > Cool! Thanks! I've looked it up. > > But again - even if you manage to transfer the messages without problems, > almost all log processing tools expect a single log record to be on one > line. So unless you have a total custom solution, you really, really will > get into troubles. > > Rainer > > I am expecting some issues, yes. So far, the tools I'm working with can handle multi-line messages, but I'm expecting stuff to pop up :) Thanks a lot for taking the time to reply! I think I have all the information I need now - I just need to go on testing, setting up and documenting. And I'll report any bugs I see along the way. Best regards, Radu _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
