Anton: If we agree this is a valid use case, then we must reconsider the UTF-8 requirement/recommendation. Because binary data will not result in proper UTF-8 encoding. I have to admit that I strongly doubt this is something that should be in syslog-protocol - maybe something for a later add-on using structured data (base-64 encoded binary structured-data would be an option)...
Please also check my implementation report on other difficulties with UTF-8 encoding, only. Rainer > -----Original Message----- > From: Anton Okmianski (aokmians) [mailto:[EMAIL PROTECTED] > Sent: Monday, November 28, 2005 4:59 PM > To: Rainer Gerhards > Cc: [EMAIL PROTECTED] > Subject: RE: [Syslog] New direction and proposed charter > > > Rainer: > > They are valid use-cases. I believe Cisco IOS logs binary > diagnostic messages today, and for good reasons. I am not > sure how they are encoded (you can of course encode binary in > ASCII). I am not fresh on details, but it seems to be a valid > use-case to me. > > Is not it a problem if logging library has to allocate memory > (for possible character substitutions) and has to scan the > message in search of invalid characters to escape? When you > are running out of memory and are trying to log a critical > message, this could be the difference between being able to > send it or not. > > Thanks, > Anton. > > > -----Original Message----- > > From: [EMAIL PROTECTED] > > [mailto:[EMAIL PROTECTED] On Behalf Of Rainer Gerhards > > Sent: Thursday, November 24, 2005 3:41 PM > > Cc: [EMAIL PROTECTED] > > Subject: RE: [Syslog] New direction and proposed charter > > > > Steve: > > > > Sorry to be blunt: this is *faaaaaaaaaaar* bejond what we > > currently try to acomplish. I guess this should be defined in > > a smime to syslog mapping document. I am just a bit concerend > > about your proposed limit of > > 9999 octets. I am not sure how much voice will fit into it... > > > > Honestly - this is no payload for syslog. I also see no point > > in SNMP backup. > > > > Some people might find that some of the samples are actual > > valid use cases, and I may become convinced over time. But - > > to say it with Darren > > - we have smaller fish to fry today ;) > > > > I also do not find it very consistent to vote in a meeting > > for backwards compatibility and then ask on list for a syslog > > revolution... > > > > Rainer > > > > > -----Original Message----- > > > From: Steve Chang (schang99) [mailto:[EMAIL PROTECTED] > > > Sent: Thursday, November 24, 2005 8:33 PM > > > To: Rainer Gerhards; [EMAIL PROTECTED] > > > Subject: RE: [Syslog] New direction and proposed charter > > > > > > Rainer: > > > > > > OK. Here are a few of my imagined uses of the MSG with > > mixed text and > > > binary data. > > > > > > 1.) With the option to include binary data in the MSG, a > new set of > > > diagnostic messages can include some system internal data > structure > > > values of concern for analysis. The beginning text in > > MSG will state > > > detailed description of the following binary data structure > > > values. With proper tools provided, the binary data structure > > > values can be > > > examined in great details at the receiving end. > > > > > > 2.) Or the MSG can be used to capture the original incoming packet > > > data > > > detected containing certain pattern. > > > > > > 3.) We can also include some voice data (binary) in MSG > for certain > > > situations. > > > > > > 4.) It can become backup route for SNMP traps because it > > can now carry > > > text and binary data in MSG. > > > > > > The possible usage is basically up to device vendor's > > imaginations. > > > But if you only allow for text data because of the concern at hand > > > over NULL character, then the door is shut. > > > > > > With potentially rich feature sets to be imagined and to come with > > > this binary-data-capable MSG, the syslog message receiver > > will have to > > > be upgraded to be able to enjoy the benefits. I will leave it to > > > sales/marketing people to convince the customers so NULL > sensitive > > > syslog message receiver will not be in the way. > > > > > > > > > Thanks, > > > > > > Steve > > > > > > > -----Original Message----- > > > > From: [EMAIL PROTECTED] > > > [mailto:[EMAIL PROTECTED] > > > > On Behalf Of Rainer Gerhards > > > > Sent: Thursday, November 24, 2005 3:50 AM > > > > To: [EMAIL PROTECTED] > > > > Subject: RE: [Syslog] New direction and proposed charter > > > > > > > > Steve, > > > > > > > > no reply, but a question very important to me. What do you > > > consider a > > > > valid use case for the US-ASCII NUL character inside MSG? > > If I had a > > > > good sample, I'd probably have a better time understanding > > > why this is > > > > important... > > > > > > > > Rainer > > > > > > > > On Thu, 2005-11-24 at 11:48, Steve Chang (schang99) wrote: > > > > > Rainer: > > > > > > > > > > > -----Original Message----- > > > > > > From: [EMAIL PROTECTED] > > > > > [mailto:[EMAIL PROTECTED] > > > > > > On Behalf Of Rainer Gerhards > > > > > > Sent: Thursday, November 24, 2005 1:25 AM > > > > > > To: Balazs Scheidler > > > > > > Cc: [EMAIL PROTECTED] > > > > > > Subject: RE: [Syslog] New direction and proposed charter > > > > > > > > > > > > > On Thu, 2005-11-24 at 09:36 +0100, Rainer Gerhards wrote: > > > > > > > > Anton: > > > > > > > > > > > > > > > So I wonder if it wouldn't be wiser to accept > > that CLR here > > > > > > > and disallow > > > > > > > > NUL. After all, I can not see a valid use case for it > > > > > > > either... (in the > > > > > > > > sample you provided it honestly believe the > sender should > > > > > > > not send a NUL > > > > > > > > octet but a "\u0000" string). > > > > > > > > > > > > > > > > > > > > > > \u0000 is a NUL octet at least in its canonical encoding. > > > > > > > > > > > > OK, looks like I need to dig up the ASCII table. My > > > fault I didn't > > > do > > > > > it > > > > > > in the first place. > > > > > > > > > > > > Anston's sample was > > > > > > > > > > > > ...MyApp pid123 MALFORMED_INPUT: Bad input received > > > from client: > > > > > > [aaa\u0000] > > > > > > > > > > > > For simplicity, let me strip the rest and just look > > at that part > > > > > > > > > > > > \u0000 > > > > > > > > > > > > I think the sender of the sample message should not > > encode it as > > > > > > > > > > > > NUL (0x00) > > > > > > > > > > > > but instead as > > > > > > > > > > > > 0x5c-0x75-0x30-0x30-0x30-0x30 (\-u-0-0-0-0) > > > > > > > > > > > > Encoding it as NUL in an otherwise human-readable > > > message makes no > > > > > sense > > > > > > to me. > > > > > > > > > > > > > > > > From the perspective of formatting a piece of device > > internal raw > > > data > > > > > into an outgoing syslog message and considering the CPU cycles > > > needed to > > > > > encode a NULL character into some other form to keep > > the whole MSG > > > > > intact, I would say prepend the length of MSG (or say, add a > > > fixed 4 digits > > > byte > > > > > length of the MSG between SD-IDs and MSG) will be much > > easier for > > > the > > > > > sender. With this, I think the new IETF syslog > > compliant receiver > > > > > should also have easier time fetching in the complete MSG, > > > regardless > > > > > it's a plain text, XML-formated, or even binary data. > > > > > [** side note: a fixed 4 digit MSG length can handle > up to 9999 > > > bytes. > > > > > Big enough for foreseeable future. Also, the first MSG can be > > > appended > > > > > with other feature specific information before it gets > > > sent out. And > > > > > hence updating the final MSG length can be easier.] > > > > > > > > > > As to what will happen to the old syslog receivers > receiving the > > > message > > > > > will NULL character? The message will probably be cut > > short right > > > at > > > > > reading the NULL character as you have explained. And I > > > don't think > > > > > there will be a good compromised solution for it. > > > > > > > > > > At the sender side, the overhead in checking for a NULL > > > character in > > > any > > > > > of the incoming raw message and encode it to other form > > > is just not > > > > > practical. I would rather not have to pay that overhead > > > if possible. > > > > > Besides, if mandated, you will be limiting how MSG is > > to be used. > > > > > > > > > > I would say it should be sufficient to just explicitly > > > point out the > > > > > danger of having the NUL in the MSG part in the new > IETF syslog > > > > > protocol. > > > > > And you can be kind to add some suggestions as to how > the syslog > > > message > > > > > initial sending device can choose to do to handle this > > situation > > > > > themselves. > > > > > Vendors should make their own decisions on this issue. > > > > > > > > > > With this caveat, we get to keep the flexibility of > > having the MSG > > > part > > > > > in plain text, or in binary for the IETF syslog protocol. > > > > > > > > > > It's tradeoff for WG to debate. > > > > > > > > > > > > > > > > I am questioning if there is any legitimate case > > where an actual > > > NUL > > > > > > needs to be part of MSG. > > > > > > > > > > > > > UTF-8 allows > > > > > > > the same character to be encoded in multiple ways, see a > > > > > > > description on the encoding format: > > > > > > > > > > > > > > http://www1.tip.nl/~t876506/utf8tbl.html > > > > > > > > > > > > > > And I think it would be wise to require the > > canonical encoding > > > to be > > > > > > > used and treat all other encodings of the same > characters as > > > errors. > > > > > > > this is a large can of worms, lot's of security > > > problems arised > > > from > > > > > > > this issue. (see the IIS problems couple of years ago) > > > > > > > > > > > > "Shortest possible form" thanksfully is already required in > > > > > > syslog-protocol-15. > > > > > > > > > > > > Rainer > > > > > > > > > > > > _______________________________________________ > > > > > > Syslog mailing list > > > > > > Syslog@lists.ietf.org > > > > > > https://www1.ietf.org/mailman/listinfo/syslog > > > > > > > > > > > > _______________________________________________ > > > > Syslog mailing list > > > > Syslog@lists.ietf.org > > > > https://www1.ietf.org/mailman/listinfo/syslog > > > > > > > _______________________________________________ > > Syslog mailing list > > Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog > > > _______________________________________________ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog
