On Sun 05 Apr 2009, E R wrote: > My concern is about what happens in this scenario: > > 1. Set everything up (named pipe, logger process, etc.) > 2. Now suppose the logger process dies (it crashes or I need to > restart it) 3. Any writes through existing connections to the named > pipe will just fail, and I'll lose those writes. > > I'm pretty sure about #3, but I might be wrong.
A (named) pipe is a ring buffer within the kernel. A writer can write to it up to a certain level. Then it is "full" and the writer blocks or gets an EAGAIN for nonblocking writes. So there is a certain period of time when the data resides in the kernel buffer. All that data is lost when the kernel panics, loses power or similar. Then the reader reads a piece of data from the pipe and does something with it. If that process crashes between reading the data from the pipe and finishing its processing that piece is lost. So there are at least 2 periods of time when data can get lost. But is that all? It depends on your definition of "sending to stderr". For example lets look at httpd. The event it wants to log happens at a certain "point" (in quotes because it is a short period in reality) in time. Before it goes out of stderr there is a period of time when a string is made from the event in ap_log_error(), log_error_core() or somewhere else. That perhaps requires some malloc()ing to extend a pool etc. And of course there can be a stray SIGKILL on its way to the process that arrives just in the moment before the formatted string hits stderr. That means there is a period of time when the event that is to be logged has happend but has not arrived on stderr. All I want to say here is you can't make 100% sure that every event that has to be logged is really logged. You can achieve quite a good level but never 100%. The good thing about a named pipe is that it can be connected by unrelated processes. So if the reader dies only the piece of data it is currently processing is lost. The writer can still write until the pipe is full. So, have your reader restarted fast enough. That's why I mentioned /etc/inittab. > Perhaps I can't do what I want to do at the application level and I > need to go to the kernel level - like write my own device driver or > something, especially since I want to capture all output sent through > stderr (ie. from Apache and from mod_perl and from any XS modules.) As I said before you can't achieve 100% even at kernel level. And imagine you'd want to write to a data base from kernel space. I don't say its impossible but you'd have to reimplement the whole protocol and the protocol specs may be not even published. Happy hacking! On Linux the ErrorLog of the base server is always opened on file descriptor 2 that means stderr. So, a reader that reads this stream gets everything (under normal conditions) that Apache, Perl or any XS module writes to stderr. Torsten -- Need professional mod_perl support? Just hire me: torsten.foert...@gmx.net
