> -----Original Message----- > From: [email protected] [mailto:rsyslog- > [email protected]] On Behalf Of [email protected] > Sent: Wednesday, February 24, 2010 1:57 AM > To: Rory Toma > Cc: rsyslog-users > Subject: Re: [rsyslog] amazon s3 as logstore > > On Tue, 23 Feb 2010, [email protected] wrote: > > By the way, if keeping the logs in order is critical to you, you need > to > be aware that rsyslog may not process the logs in order under all > conditions. > > For example: if you have multiple worker threads, they each grab some > logs > off the queue and attempt to deliver/write them. This can cause the > logs > to be delivered/written in a different order than rsyslog received > them. > > Since there are so many other things that can re-order logs slightly, > this > is not viewed as a significant problem. >
David is very right here. If you rely on the sequence as recorded in the log file, you have a big problem. That problem is not caused by rsyslog's threading, but rather by the fact that syslog is usually used inside a remote system. In such a system, there is no guaranteed sequence: UDP get's reordered, TCP messages sit for different timespans in interim routers, syslog relay chains may delay messages even for hours (if a relay is down). I thought I had blogged about this, but can't find the post myself right now ;) But in short: you need to be very careful about your sequence expectation. Timestamps are much better, but of course not perfect. RFC5424 provides the plumbing for a better solution, but none is standardized yet and I know of no such implementation. I have Lamport clocks on my mind as the "ultimate" solution (and maybe I get time to implement that in a distant future...). Rainer _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
