2014-07-13 16:21 GMT+08:00 Rainer Gerhards <[email protected]>: > Did you read my reply?
Yep, thank you for your suggestion. It looked like liblogging doing exactly what I want it to be. According to man page SYSLOG(3), the fuction syslog() are specified in SUSv2, POSIX.1-2001, and POSIX.1-2008. It's a little sad that POSIX didn't designed it (syslog() function) as a local message tool. It seems that syslog() must work with a socket, which is not an ideal way to handle local log messages IMO ... As I can see, nevertheless, there are currently plenty of applications using syslog as just a local log message mechanism. But in it's heart, syslog didn't seem to be designed for local log at all... Which made me try to discover another logger. > > Sent from phone, thus brief. > Am 13.07.2014 09:53 schrieb "Justin Lee" <[email protected]>: > >> 2014-07-13 3:31 GMT+08:00 David Lang <[email protected]>: >> > On Sat, 12 Jul 2014, Justin Lee wrote: >> > >> >> Hello List, >> >> >> >> I noticed that local messages are passed by socket rather than >> >> directly written to log files. The evidence is as follows: >> > >> > >> > Yes, this is how syslog works. >> > >> > >> >> Since log mechanism is an important infrastructure of many programs, >> >> it is supposed to be as fast as possible. So do we have some >> >> configurations or settings in rsyslog which make local messages >> >> written directly into log files without passing through socket? >> > >> > >> > actually, writing to disk can be slower than writing to a socket, so it's >> > not clear that it's a performance win to write to disk directly. >> >> I think that writing to disk is supposed to be the final step of >> rsyslogd in processing local messages. After all, we couldn't see the >> messages in log files if rsyslogd didn't write to disk at all, right? >> But in addition to disk writing, use of socket by syslog to transfer >> local log messages introduces extra CPU time or memory to do the >> following things: >> >> 1. To handle socket related stuff, such as copying log messages in and >> out of socket buffer. Buffer copying introduces overhead. >> >> 2. We need a thread or daemon process (rsyslogd) to listen to and >> receive log messages from the socket. It increases the memory >> footprint of our log infrastructure because of the daemon process >> itself. >> >> 3. At least one context switch is required to transfer a log message >> from user program to rsyslog daemon, because user program and the >> syslog are in two different processes. Most context switches flush the >> CPU pipeline and hence affecting efficiency. >> >> 4. Including a daemon process in mediating log affairs makes message >> logging dominated by kernel's scheduling policy and number of running >> processes in the system. Because CPU resources and CPU time are shared >> by all the processes running in the system, the more running processes >> in the system the less CPU time a single process (rsyslogd in our >> case) can get to accomplish its job. >> >> 5. Because all running processes in the system share the same CPU, the >> kernel must execute each process in turn (let each process get a >> chance to be executed) such that CPU time are divided and distributed >> among all running processes. The order of process execution is >> determined by kernel's scheduling policy. The execution order could be >> like the steps below: >> >> (1) a user program writes a log message to the syslog socket. >> >> (2) the kernel decides to execute two other processes in the >> system first before executing the rsyslog daemon since they have >> higher priority or due to the scheduling policy. >> >> (3) the kernel than finally execute rsyslog daemon so that the log >> message sent by user program at step (1) is received by the daemon >> from syslog socket. >> >> as more and more processes run in the system, the number of processes >> get executed at step (2) could be increased. So the time of a log >> message getting propagated from user program to rsyslog daemon becomes >> longer and longer. And the latency to transfer a log message becomes >> higher and higher. I think this kills the performance the most. >> >> 6. Log messages may need to stay in socket buffer for longer time due >> the high latency which implies a bigger socket buffer is required >> which implies a greater memory footprint. The case gets worse when >> system loading is high and increase the risk of socket buffer overflow >> and message loss. >> >> > >> > then you have the problem of what happens to the log. If you just write >> to >> > disk, you throw away a tremendous amount of flexibility in determining >> what >> > happens to the log. >> > >> > If you send the log through the syslog infrastructure, there are a lot of >> > things that you can end up doing with it. >> > >> > You can: >> > >> > combine the logs from differnet programs into one file >> > >> > split the logs from one program into multiple files >> > >> > filter the logs >> > >> > send the logs to a remote machine >> > >> > put the logs into a database >> > >> > etc. >> > >> > >> > >> > so having your program write directly to disk is less work overall for >> the >> > system to do, but it can be slower for your application, and it's far >> less >> > flexible. >> > >> > David Lang >> > >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com/professional-services/ >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of >> > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T >> > LIKE THAT. >> >> Justin Lee >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
