Hi, I recommend we stop talking about RFC3164-compliant, since RFC3164 is only an Informational document and not standards-track.
dbh > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Rainer Gerhards > Sent: Thursday, November 24, 2005 8:01 AM > To: Glenn Mansfield Keeni > Cc: [EMAIL PROTECTED] > Subject: RE: [Syslog] New direction and proposed charter > > Glenn, > > > Now the question is : are there any RFC3164 compliant devices > > (relays and syslogd's) and applications. > > I have to admit that I have written at least one compliant to > RFC 3164. > But my feeling is that I am very lonley with that. In one project, we > had to make RFC 3164 not only optional but to disable it by default, > because as soon as you enable it, everything is broken. > > Also, I have done a lot of testing with different syslogds yesterday. > None of this quite popular systems is compliant. I neither know any > compliant device. Anton has backed this point of view (for example, he > adds Solaris as being non-compliant). > > RFC 3164 is an informational document whichs utility was primarily to > describe security issues. It is good in that. However, it > does not even > remotely describe what BSD syslog is about (a good indication of that > might be that BSD syslog works totally different from RFC 3164...). > > I question that it makes any sense to modify our efforts to > take care of > an informational RFC that doesn't even describe what is current > technology. Don't take me wrong: I myself based a lot of my > decisions on > RFC 3164. Now, after extended testing, I thinkt that was a great > failure. When it comes to the observed format, RFC 3164 should tell > about <PRI> only. Other than that, there are no similarities > in observed > syslog behaviour. Sorry for the harsh words, but if you go to the lab, > that's where you end up (notice my own disappointment yesterday...). > > > If there are then we gain some and lose very little. [ Correct > > me here if am wrong in saying that we lose very little. I may > > have missed something] > > > > No sure on "lose very little". I am tired of taklking theory. I will > implement something and then tell. I guess we will loose some > things, at > least in ease of parsing. And I still think we do not gain anything at > all... > > > I will agree that many implementations are not RFC3164 compliant. > > But, it will be difficult to convince IESG folks or, anyone for > > that matter, that there are no RFC3164 compliant devices or > > applications at all. [ Note that the applications matter too!] > > OK, let's poll the list: who has implemented RFC 3164 including relay > behaviour described there? > > I have two candidates, MonitorWare/WinSyslog (some core component) on > Windows and rsyslogd on *nix. MonitorWare is the one with > default-disabled rfc3164. rsyslogd uses a much enhanced parser (non > compliant) to get its work done. rsyslogd would probably benefit from > your suggestion. However, I do not think this to be worthwhile enough, > because a) there are currently too few deployed to make it > matter and b) > the next version will natively support-syslog protocol and is an > extremely easy upgrade. > > > My suggestion is- avoid changes wherever we can, > > I agree on this, but in that specific case I really think its > not worth > it. RFC 3164 creates a false impression of format understanding. Other > than <PRI>, there is noting in common about existing syslog > implementations. > > Rainer > > > > Cheers > > > > Glenn > > > > Rainer Gerhards wrote: > > > This is really disappointing... > > > > > > I have done further testing with more syslogds to confirm > > the initial > > > findings. The more different programs / versions I try, the > > more a mess > > > this gets. OK, we knew FreeBSD syslogd does not include the > > hostname. > > > Next I tested with some Windows products, which looked > > promising. Then I > > > looked at the sysklogd source. It is the standard Linux > > package, so if > > > it works, a lot would be won. The source tells me that at > least the > > > timestamp should correctly be extracted. Then I actually > > tried it out > > > with a Debian system - the timestamp was ignored. Checking > > that source > > > tree I see that *that* sysklogd does deliberately ignore > > the data and > > > always pulls the local date (or ist it a bug - who > > cares...). When it > > > comes to relaying it is even stranger: sysklogd does > > neither relay the > > > timestamp nor the host part. At that point, I have stopped further > > > analysis, because I think I would be able to find another > > good number of > > > variants of syslog behaviour. > > > > > > Conclusion: > > > > > > 1. There is no point in trying to preserve backward > > > compatibility besides sticking with the <PRI>. Everything other > > > than <PRI> is handled differently by different implementations > > > and/or versions. > > > > > > 2. We can NOT expect that relaying over existing syslog > > > implementations will ever work. Please note that this would > > > also have broken syslog-sign without specifically implemented > > > daemons. > > > > > > As such, I revert back to proposing > > > > > > <PRI>VERSION SP TIMESTAMP SP HOSTNAME SP APP-NAME SP PROCID > > SP MSGID SP > > > [SD-ID]s SP MSG > > > > > > or, somewhat incorrect but shorter: > > > > > > <PRI>VERSION TIMESTAMP HOSTNAME APP-NAME PROCID MSGID [SD-ID]s MSG > > > > > > Please note that I have added the MSGID to the header. > > > > > > Rainer > > > > > > > > >>-----Original Message----- > > >>From: [EMAIL PROTECTED] > > >>[mailto:[EMAIL PROTECTED] On Behalf Of > Rainer Gerhards > > >>Sent: Wednesday, November 23, 2005 3:04 PM > > >>To: Glenn Mansfield Keeni; [EMAIL PROTECTED] > > >>Subject: RE: [Syslog] New direction and proposed charter > > >> > > >>Glenn, > > >> > > >>very interesting approach with the timestamp. I think your > > >>ideas can be > > >>the key to maintaining a lot of backwards compatibility by still > > >>retaining new functionality. > > >> > > >>First some bad news: I am not sure if by "BSD syslog" you > > are refering > > >>to RFC 3164 or a specific distribution of BSD. I have > > created a small > > >>script to test out your recommendation. I used FreeBSD stock > > >>syslogd as > > >>the receiver. > > >> > > >>It did NOT work as expected. There are two reasons > > >> > > >>a) (that) BSD syslogd takes the sender always from the system > > >>that send > > >>it > > >>b) even worse, when relaying, it puts "Forwarded from > > >><hostname>: " into > > >> > > >> the hostname part (yes, with all that spaces) > > >> > > >>So while the idea sounds excellent, it does not work with > > >>stock FreeBSD > > >>syslogd. I am not sure about other BSD variants, nor have I > > >>checked the > > >>sysklogd package. I believe it will have less issues in > this regard. > > >> > > >>This was the message I sent (via perl script): > > >> > > >>"<148>Oct 11 22:14:15 mymachine.example.com 1 ID47 2003T.003Z > > >> 'su root' > > >>failed for lonvick on /dev/pts/8" > > >> > > >>this was the raw message received after being relayed once > > by FreeBSD > > >>stock syslogd: > > >> > > >>"<148>Oct 11 22:14:15 Forwarded from 172.19.2.7: > > >>mymachine.example.com 1 > > >>ID47 2003T.003Z 'su root' failed for lonvick on /dev/pts/8" > > >> > > >>As you can see, the message is somewhat distorted - > > definitely enough > > >>for digital signatures to be broken. [Implementor's > > >>side-note: This can > > >>be fixed on a syslog-application layer level, far beyond the > > >>IETF scope. > > >>It's straightforward and easy to do, so it will probably happen.] > > >> > > >>Even though this actual sample seems not to work, it paves > > >>the way to a > > >>very elegant compatibility solution. The key is to add the extra > > >>information (e.g. Timestamp) in a different place. At > least I was so > > >>focussed on fields at whole that I did not notice this > > possibility. I > > >>have experiemented a bit more with Glenn's proposal, > shuffeling some > > >>fields. The result was this: > > >> > > >><PRI>BSD_syslog_timestamp FQDN TAG "@#"VERSION MSGID > > >>Remainder_Timestamp > > >>[SD-ID]s MSG > > >> > > >>or in an actual sample: > > >> > > >>"<148>Oct 11 22:14:15 mymachine.example.com su[4711]: @#1 ID47 > > >>2003T.003Z [SD-IDs] 'su root' failed for lonvick on /dev/pts/8" > > >> > > >>I have used the BSD timestamp and FAQN as Glenn suggested. > > >>Then, I have > > >>added the "TAG" again. If we think in the spirit of my mail > > >>this morning > > >>on syslog & non-IETF standards, it would not really hurt if we > > >>standardize TAG instead of two fields. If I would like to > retain the > > >>APP-NAME and PROCESSID, I could do the following ABNF: > > >> > > >>TAG = APP-NAME ["[" PROCESSID "]"] ":" > > >> > > >>The side-effect of this is that almost all > syslog-messages currently > > >>emited comply with that format. So I suggest that we > > strongly consider > > >>joining these two again. Out of the sudden, we have the > > "old" header, > > >>but it is parsable by a hyptothetical new syslogd. Next, I > > have used a > > >>trick from syslog-sign. I have changed the VERSION from a > > number to a > > >>number plus a cookie. The version would now be "@#1". I do > > not care if > > >>the cookie is "@#" or something else. The key point is that > > >>it, together > > >>with the version should be very unlikely to exist at that place in > > >>old-style syslog. That would allow a "new" receiver to > differentiate > > >>between old and new style syslog messages. The rest of the > > message is > > >>just applying Glenn's proposal again: it has the MSG, the > > >>missing parts > > >>of the timestamp, SD-IDs and MSG. The Remainder_Timestamp > > >>looks strange. > > >>We might like it, we might like something else. That's easy > > to change > > >>and discuss. It's the concept that matters right now, not > the exact > > >>format. > > >> > > >>If we take the outlined route, we would be able to extend > the syslog > > >>protocol with as much backward compatibility as is possible in a > > >>not-yet-standardized world. I find this very desirable. I > > >>think we even > > >>have good chances that many existing "old" syslogds would > relay such > > >>messages without changing them, thus keeping digital > > >>signatures intact. > > >>The required text changes for syslog-protocol should be moderate. > > >> > > >>I strongly propose we go in that direction. > > >> > > >>Rainer > > >> > > >>>-----Original Message----- > > >>>From: [EMAIL PROTECTED] > > >>>[mailto:[EMAIL PROTECTED] On Behalf Of Glenn > > >>>Mansfield Keeni > > >>>Sent: Wednesday, November 23, 2005 1:39 PM > > >>>To: [EMAIL PROTECTED] > > >>>Subject: Re: [Syslog] New direction and proposed charter > > >>> > > >>>Chris/Rainer, > > >>> > > >>> > > >>>>we continue to use <PRI>... at the start of syslog > > >>> > > >>>messages. This will > > >>> > > >>>>allow current receivers to continue to receive messages and > > >>> > > >>>put them in > > >>> > > >>>>the right bins. Does anyone disagree with this? > > >>> > > >>>Complete agreement. > > >>> > > >>>> > > >>>>The WG has agreed to use the timestamp Rainer has in the current > > >>>>syslog-protocol. > > >>> > > >>>In principle I agree with the timestamp format. It is good. > > >>>I may have missed the discussion on this matter, in that > > >> > > >>case please > > >> > > >>>accept my apologies and ignore the rest of the mail. > > >>> > > >>>To get existing BSD syslog devices specifically relays into > > >>>the compatibility > > >>>fold it WILL be good idea to keep the timestamp in two parts > > >>> > > >>> RainerTimestamp = BSD_syslog_timestamp + > > RemainderTimestamp > > >>> > > >>> > > >>>>One possibility would be to assemble a syslog message as: > > >>>> > > >>>><PRI>TIMESTAMP FQDN VERSION MSGID [SD-ID]s MSG > > >>> > > >>>In the context of what has been said above about the > > >>>timestamp. I would > > >>>suggest > > >>> <PRI>BSD_syslog_timestamp FQDN VERSION MSGID > > >>>Remainder_Timestamp [SD-ID]s MSG > > >>> > > >>>That would allow existing BSD-syslog relays to handle the new > > >>>syslog protocol > > >>>in a transparent manner. Everything from VERSION to the end > > >>>is treated as "message". > > >>> > > >>>We do not lose information. > > >>> The Remainder_timestamp carries it - in a slightly less > > >>>convenient place > > >>> though. > > >>> > > >>>On the other hand if we insist on using RainerTimestamp, > > >>>existing BSD_syslog > > >>>relays will relay the message as > > >>> > > >>><PRI>BSD_syslog_timestamp FQDN RainerTimestamp FQDN VERSION > > >>>MSGID [SD-ID]s MSG > > >>> > > >>>The message does get distorted to some extent. > > >>> > > >>> > > >>>>If we can agree to this then I suspect that we can have > a working > > >>>>document within Rainer's timeframe. I'll propose the > > >>> > > >>>following charter > > >>> > > >>>>to keep us focused. > > >>>> > > >>>>-------- Proposed Charter -------------- > > >>>> > > >>>>Syslog is a de-facto standard for logging system events. > > >>> > > >>>However, the > > >>> > > >>>>protocol component of this event logging system has not > > >>> > > >>>been formally > > >>> > > >>>>documented. While the protocol has been very useful and > > >>> > > >>>scalable, it > > >>> > > >>>>has some known security problems which were documented in > > >> > > >>RFC 3164. > > >> > > >>>>The goal of this working group is to address the security and > > >>>>integrity problems of the existing syslog mechanism while > > >>> > > >>>not breaking > > >>> > > >>>>backwards compatibility. The most obvious problems that > > >> > > >>need to be > > >> > > >>>>addressed in the syslog protocol are the timestamp, > which has not > > >>>>formally included a means to indicate the year, and the > > >>> > > >>>identification > > >>> > > >>>>of the source which has been a hostname without a > qualified domain > > >>>>name. Additionally, a version, some type of message > > >>> > > >>>indicator, and a > > >>> > > >>>>means to convey structured data will be included in the > protocol. > > >>>> > > >>>>syslog has traditionally been transported over UDP and > this WG has > > >>>>already defined RFC 3195 for the reliable transport for > the syslog > > >>>>messages. The WG will separate the UDP transport from the > > >>> > > >>>protocol so > > >>> > > >>>>that others may define additional transports in the future. > > >>>> > > >>>>- A standard will be produced that formally documents the syslog > > >>>>protocol. A mechanism will also be defined in this > specification > > >>>>that will provide a means to convey structured data. > > >>>> > > >>>>- A standard will be produced that documents the UDP > transport for > > >>>>syslog. > > >>>> > > >>>>- A standard will be produced that documents a mechanism to sign > > >>>>syslog messages to provide integrity checking and source > > >>>>authentication. > > >>>> > > >>>>- A MIB definition for syslog will be produced. > > >>>> > > >>>>------------------------------------------- > > >>>> > > >>>> > > >>>>PLEASE review this and respond. > > >>>> > > >>> > > >>>Glenn > > >>> > > >>> > > >>>_______________________________________________ > > >>>Syslog mailing list > > >>>Syslog@lists.ietf.org > > >>>https://www1.ietf.org/mailman/listinfo/syslog > > >>> > > >> > > >>_______________________________________________ > > >>Syslog mailing list > > >>Syslog@lists.ietf.org > > >>https://www1.ietf.org/mailman/listinfo/syslog > > >> > > > > > > > > > > > > > > > > _______________________________________________ > > Syslog mailing list > > Syslog@lists.ietf.org > > https://www1.ietf.org/mailman/listinfo/syslog > > > > _______________________________________________ > Syslog mailing list > Syslog@lists.ietf.org > https://www1.ietf.org/mailman/listinfo/syslog > _______________________________________________ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog
