For what it is worth... I have tried to create a send-template for rsyslogd, my *nix syslogd. It covers:
> <PRI>VERSION SP TIMESTAMP SP HOSTNAME SP APP-NAME SP PROCID SP MSGID SP > [SD-ID]s SP MSG > > or, somewhat incorrect but shorter: > > <PRI>VERSION TIMESTAMP HOSTNAME APP-NAME PROCID MSGID [SD-ID]s MSG > > Please note that I have added the MSGID to the header. It was easy to do, actually a 5 minute job. Of course, sending a message is another beast then parsing it, but I think this will also be sufficiently simple. As far as I know (some) other syslogds have similar template systems, so this point eventually proves that the modifications required will not break anything at all. The only tricky issue that remains is the NUL octect. The more I think about it, the more I think the CLR to disallow it is less evil than to make it stay... Rainer On Wed, 2005-11-23 at 16:29, Rainer Gerhards wrote: > This is really disappointing... > > I have done further testing with more syslogds to confirm the initial > findings. The more different programs / versions I try, the more a mess > this gets. OK, we knew FreeBSD syslogd does not include the hostname. > Next I tested with some Windows products, which looked promising. Then I > looked at the sysklogd source. It is the standard Linux package, so if > it works, a lot would be won. The source tells me that at least the > timestamp should correctly be extracted. Then I actually tried it out > with a Debian system - the timestamp was ignored. Checking that source > tree I see that *that* sysklogd does deliberately ignore the data and > always pulls the local date (or ist it a bug - who cares...). When it > comes to relaying it is even stranger: sysklogd does neither relay the > timestamp nor the host part. At that point, I have stopped further > analysis, because I think I would be able to find another good number of > variants of syslog behaviour. > > Conclusion: > > 1. There is no point in trying to preserve backward > compatibility besides sticking with the <PRI>. Everything other > than <PRI> is handled differently by different implementations > and/or versions. > > 2. We can NOT expect that relaying over existing syslog > implementations will ever work. Please note that this would > also have broken syslog-sign without specifically implemented > daemons. > > As such, I revert back to proposing > > <PRI>VERSION SP TIMESTAMP SP HOSTNAME SP APP-NAME SP PROCID SP MSGID SP > [SD-ID]s SP MSG > > or, somewhat incorrect but shorter: > > <PRI>VERSION TIMESTAMP HOSTNAME APP-NAME PROCID MSGID [SD-ID]s MSG > > Please note that I have added the MSGID to the header. > > Rainer > > > -----Original Message----- > > From: [EMAIL PROTECTED] > > [mailto:[EMAIL PROTECTED] On Behalf Of Rainer Gerhards > > Sent: Wednesday, November 23, 2005 3:04 PM > > To: Glenn Mansfield Keeni; [EMAIL PROTECTED] > > Subject: RE: [Syslog] New direction and proposed charter > > > > Glenn, > > > > very interesting approach with the timestamp. I think your > > ideas can be > > the key to maintaining a lot of backwards compatibility by still > > retaining new functionality. > > > > First some bad news: I am not sure if by "BSD syslog" you are refering > > to RFC 3164 or a specific distribution of BSD. I have created a small > > script to test out your recommendation. I used FreeBSD stock > > syslogd as > > the receiver. > > > > It did NOT work as expected. There are two reasons > > > > a) (that) BSD syslogd takes the sender always from the system > > that send > > it > > b) even worse, when relaying, it puts "Forwarded from > > <hostname>: " into > > > > the hostname part (yes, with all that spaces) > > > > So while the idea sounds excellent, it does not work with > > stock FreeBSD > > syslogd. I am not sure about other BSD variants, nor have I > > checked the > > sysklogd package. I believe it will have less issues in this regard. > > > > This was the message I sent (via perl script): > > > > "<148>Oct 11 22:14:15 mymachine.example.com 1 ID47 2003T.003Z > > 'su root' > > failed for lonvick on /dev/pts/8" > > > > this was the raw message received after being relayed once by FreeBSD > > stock syslogd: > > > > "<148>Oct 11 22:14:15 Forwarded from 172.19.2.7: > > mymachine.example.com 1 > > ID47 2003T.003Z 'su root' failed for lonvick on /dev/pts/8" > > > > As you can see, the message is somewhat distorted - definitely enough > > for digital signatures to be broken. [Implementor's > > side-note: This can > > be fixed on a syslog-application layer level, far beyond the > > IETF scope. > > It's straightforward and easy to do, so it will probably happen.] > > > > Even though this actual sample seems not to work, it paves > > the way to a > > very elegant compatibility solution. The key is to add the extra > > information (e.g. Timestamp) in a different place. At least I was so > > focussed on fields at whole that I did not notice this possibility. I > > have experiemented a bit more with Glenn's proposal, shuffeling some > > fields. The result was this: > > > > <PRI>BSD_syslog_timestamp FQDN TAG "@#"VERSION MSGID > > Remainder_Timestamp > > [SD-ID]s MSG > > > > or in an actual sample: > > > > "<148>Oct 11 22:14:15 mymachine.example.com su[4711]: @#1 ID47 > > 2003T.003Z [SD-IDs] 'su root' failed for lonvick on /dev/pts/8" > > > > I have used the BSD timestamp and FAQN as Glenn suggested. > > Then, I have > > added the "TAG" again. If we think in the spirit of my mail > > this morning > > on syslog & non-IETF standards, it would not really hurt if we > > standardize TAG instead of two fields. If I would like to retain the > > APP-NAME and PROCESSID, I could do the following ABNF: > > > > TAG = APP-NAME ["[" PROCESSID "]"] ":" > > > > The side-effect of this is that almost all syslog-messages currently > > emited comply with that format. So I suggest that we strongly consider > > joining these two again. Out of the sudden, we have the "old" header, > > but it is parsable by a hyptothetical new syslogd. Next, I have used a > > trick from syslog-sign. I have changed the VERSION from a number to a > > number plus a cookie. The version would now be "@#1". I do not care if > > the cookie is "@#" or something else. The key point is that > > it, together > > with the version should be very unlikely to exist at that place in > > old-style syslog. That would allow a "new" receiver to differentiate > > between old and new style syslog messages. The rest of the message is > > just applying Glenn's proposal again: it has the MSG, the > > missing parts > > of the timestamp, SD-IDs and MSG. The Remainder_Timestamp > > looks strange. > > We might like it, we might like something else. That's easy to change > > and discuss. It's the concept that matters right now, not the exact > > format. > > > > If we take the outlined route, we would be able to extend the syslog > > protocol with as much backward compatibility as is possible in a > > not-yet-standardized world. I find this very desirable. I > > think we even > > have good chances that many existing "old" syslogds would relay such > > messages without changing them, thus keeping digital > > signatures intact. > > The required text changes for syslog-protocol should be moderate. > > > > I strongly propose we go in that direction. > > > > Rainer > > > -----Original Message----- > > > From: [EMAIL PROTECTED] > > > [mailto:[EMAIL PROTECTED] On Behalf Of Glenn > > > Mansfield Keeni > > > Sent: Wednesday, November 23, 2005 1:39 PM > > > To: [EMAIL PROTECTED] > > > Subject: Re: [Syslog] New direction and proposed charter > > > > > > Chris/Rainer, > > > > > > > we continue to use <PRI>... at the start of syslog > > > messages. This will > > > > allow current receivers to continue to receive messages and > > > put them in > > > > the right bins. Does anyone disagree with this? > > > > > > Complete agreement. > > > > > > > > > > > > The WG has agreed to use the timestamp Rainer has in the current > > > > syslog-protocol. > > > In principle I agree with the timestamp format. It is good. > > > I may have missed the discussion on this matter, in that > > case please > > > accept my apologies and ignore the rest of the mail. > > > > > > To get existing BSD syslog devices specifically relays into > > > the compatibility > > > fold it WILL be good idea to keep the timestamp in two parts > > > > > > RainerTimestamp = BSD_syslog_timestamp + RemainderTimestamp > > > > > > > One possibility would be to assemble a syslog message as: > > > > > > > > <PRI>TIMESTAMP FQDN VERSION MSGID [SD-ID]s MSG > > > > > > In the context of what has been said above about the > > > timestamp. I would > > > suggest > > > <PRI>BSD_syslog_timestamp FQDN VERSION MSGID > > > Remainder_Timestamp [SD-ID]s MSG > > > > > > That would allow existing BSD-syslog relays to handle the new > > > syslog protocol > > > in a transparent manner. Everything from VERSION to the end > > > is treated as "message". > > > > > > We do not lose information. > > > The Remainder_timestamp carries it - in a slightly less > > > convenient place > > > though. > > > > > > On the other hand if we insist on using RainerTimestamp, > > > existing BSD_syslog > > > relays will relay the message as > > > > > > <PRI>BSD_syslog_timestamp FQDN RainerTimestamp FQDN VERSION > > > MSGID [SD-ID]s MSG > > > > > > The message does get distorted to some extent. > > > > > > > If we can agree to this then I suspect that we can have a working > > > > document within Rainer's timeframe. I'll propose the > > > following charter > > > > to keep us focused. > > > > > > > > -------- Proposed Charter -------------- > > > > > > > > Syslog is a de-facto standard for logging system events. > > > However, the > > > > protocol component of this event logging system has not > > > been formally > > > > documented. While the protocol has been very useful and > > > scalable, it > > > > has some known security problems which were documented in > > RFC 3164. > > > > > > > > The goal of this working group is to address the security and > > > > integrity problems of the existing syslog mechanism while > > > not breaking > > > > backwards compatibility. The most obvious problems that > > need to be > > > > addressed in the syslog protocol are the timestamp, which has not > > > > formally included a means to indicate the year, and the > > > identification > > > > of the source which has been a hostname without a qualified domain > > > > name. Additionally, a version, some type of message > > > indicator, and a > > > > means to convey structured data will be included in the protocol. > > > > > > > > syslog has traditionally been transported over UDP and this WG has > > > > already defined RFC 3195 for the reliable transport for the syslog > > > > messages. The WG will separate the UDP transport from the > > > protocol so > > > > that others may define additional transports in the future. > > > > > > > > - A standard will be produced that formally documents the syslog > > > > protocol. A mechanism will also be defined in this specification > > > > that will provide a means to convey structured data. > > > > > > > > - A standard will be produced that documents the UDP transport for > > > > syslog. > > > > > > > > - A standard will be produced that documents a mechanism to sign > > > > syslog messages to provide integrity checking and source > > > > authentication. > > > > > > > > - A MIB definition for syslog will be produced. > > > > > > > > ------------------------------------------- > > > > > > > > > > > > PLEASE review this and respond. > > > > > > > > > > Glenn > > > > > > > > > _______________________________________________ > > > Syslog mailing list > > > Syslog@lists.ietf.org > > > https://www1.ietf.org/mailman/listinfo/syslog > > > > > > > _______________________________________________ > > Syslog mailing list > > Syslog@lists.ietf.org > > https://www1.ietf.org/mailman/listinfo/syslog > > _______________________________________________ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog
