Did you read my reply? Sent from phone, thus brief. Am 13.07.2014 09:53 schrieb "Justin Lee" <[email protected]>:
> 2014-07-13 3:31 GMT+08:00 David Lang <[email protected]>: > > On Sat, 12 Jul 2014, Justin Lee wrote: > > > >> Hello List, > >> > >> I noticed that local messages are passed by socket rather than > >> directly written to log files. The evidence is as follows: > > > > > > Yes, this is how syslog works. > > > > > >> Since log mechanism is an important infrastructure of many programs, > >> it is supposed to be as fast as possible. So do we have some > >> configurations or settings in rsyslog which make local messages > >> written directly into log files without passing through socket? > > > > > > actually, writing to disk can be slower than writing to a socket, so it's > > not clear that it's a performance win to write to disk directly. > > I think that writing to disk is supposed to be the final step of > rsyslogd in processing local messages. After all, we couldn't see the > messages in log files if rsyslogd didn't write to disk at all, right? > But in addition to disk writing, use of socket by syslog to transfer > local log messages introduces extra CPU time or memory to do the > following things: > > 1. To handle socket related stuff, such as copying log messages in and > out of socket buffer. Buffer copying introduces overhead. > > 2. We need a thread or daemon process (rsyslogd) to listen to and > receive log messages from the socket. It increases the memory > footprint of our log infrastructure because of the daemon process > itself. > > 3. At least one context switch is required to transfer a log message > from user program to rsyslog daemon, because user program and the > syslog are in two different processes. Most context switches flush the > CPU pipeline and hence affecting efficiency. > > 4. Including a daemon process in mediating log affairs makes message > logging dominated by kernel's scheduling policy and number of running > processes in the system. Because CPU resources and CPU time are shared > by all the processes running in the system, the more running processes > in the system the less CPU time a single process (rsyslogd in our > case) can get to accomplish its job. > > 5. Because all running processes in the system share the same CPU, the > kernel must execute each process in turn (let each process get a > chance to be executed) such that CPU time are divided and distributed > among all running processes. The order of process execution is > determined by kernel's scheduling policy. The execution order could be > like the steps below: > > (1) a user program writes a log message to the syslog socket. > > (2) the kernel decides to execute two other processes in the > system first before executing the rsyslog daemon since they have > higher priority or due to the scheduling policy. > > (3) the kernel than finally execute rsyslog daemon so that the log > message sent by user program at step (1) is received by the daemon > from syslog socket. > > as more and more processes run in the system, the number of processes > get executed at step (2) could be increased. So the time of a log > message getting propagated from user program to rsyslog daemon becomes > longer and longer. And the latency to transfer a log message becomes > higher and higher. I think this kills the performance the most. > > 6. Log messages may need to stay in socket buffer for longer time due > the high latency which implies a bigger socket buffer is required > which implies a greater memory footprint. The case gets worse when > system loading is high and increase the risk of socket buffer overflow > and message loss. > > > > > then you have the problem of what happens to the log. If you just write > to > > disk, you throw away a tremendous amount of flexibility in determining > what > > happens to the log. > > > > If you send the log through the syslog infrastructure, there are a lot of > > things that you can end up doing with it. > > > > You can: > > > > combine the logs from differnet programs into one file > > > > split the logs from one program into multiple files > > > > filter the logs > > > > send the logs to a remote machine > > > > put the logs into a database > > > > etc. > > > > > > > > so having your program write directly to disk is less work overall for > the > > system to do, but it can be slower for your application, and it's far > less > > flexible. > > > > David Lang > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of > > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > > LIKE THAT. > > Justin Lee > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
