On Tue, 23 Feb 2010 15:47:44 -0700, Alex Rousskov <[email protected]> wrote: > On 02/23/2010 02:57 PM, Amos Jeffries wrote: >> On Tue, 23 Feb 2010 09:03:51 -0700, Alex Rousskov >> <[email protected]> wrote: >>> On 02/23/2010 07:44 AM, Kinkie wrote: >>>> 2010/2/23 Tsantilas Christos <[email protected]>: >>>>>> Log rotation will probably need special care to avoid rotating the >> same >>>>>> log many times. It is relatively easy to designate one process to do >>>>>> the >>>>>> rotation (e.g., squid1), but that alone is not enough to synchronize >>>>>> everything. Ideally, I would prefer to avoid locking. Any clever >> tricks >>>>>> applicable here? >>>>> Maybe something like the following: >>>>> - there is a master process >>>>> - master process communicates with one direction pipes with kids, >>>>> sending >>>>> various commands >>>>> >>>>> For logs rotation: >>>>> - master process closes the old file and rename it >>>>> - master process opens a new log file >>>>> - kids still writing to the old file (because they did not close the >>>>> file descriptor yet) >>>>> - master process send a command to all kids saying that the log file >>>>> rotated >>>>> - The kids close the log file and reopen it. >>>>> >>>>> The above will work at least for unix systems (I do not know about >>>>> MS-Windows) >>>>> Master/kids communication using pipes is used by the apache worker >>>>> server. >>>>> I am also using it in my icap server. I am not seeing any problem. >>>> Why not just having a log writing helper to whom clients write log >>>> lines via some kind of sockets? >>>> That would make the log rotation problem just go away - especially the >>>> part where there may be race conditions among workers writing together >>>> to the same file. >>> The logging daemon does not make log rotation problem go away. You still >>> need to implement log rotation. It just becomes slightly easier to do >> so. >>> The O_APPEND solution, if it works, does not have race conditions among >>> workers writing together to the same file. Or, more precisely, we trust >>> the file system layer to handle those race conditions. >>> >>> Please see my earlier email with the same subject for the >>> daemon-vs-append comparison summary. I would not be surprised if there >>> are more pros and cons to consider here, but so far I do not see enough >>> advantages to implement a logging daemon as the default SMP logging >>> mechanism. >> >> We already have a logging daemon producing stdin/stdout/stderr sockets to >> a master process. We have a TCP multi-input daemon coming. The file >> daemon >> is certainly working swimmingly well here and elsewhere. >> >> All thats needed is to make it the default and... >> a) pass its sockets a little further afield (same as for all other >> helpers if we share them). > > If the stdin socket does not have an equivalent of O_APPEND, can > multiple processes write to it and still avoid corrupting each other > entries? > >> or >> b) use O_APPEND internally to the daemon so each process sub-child >> daemon >> can write the same log. > > Sure. And this would especially make sense if the daemon is doing more > than just logging (and Squid cannot do that internally). Otherwise, each > process can simply log with O_APPEND to a single file. The choice would > be up to the administrator, with the simplest option being the default. > > >> The hard bit is cache.log. Though if we go the O_APPEND route thats easy, >> if we want everything daemon logged, it should not be too hard to start a >> daemon for it and convert debugs() to log there. > > Ah, I did not realize cache.log daemon logging is not supported yet. One > more reason to start with simple O_APPEND. As a side effect, we would be > able to debug daemon log starting problems as well :-). >
Yay. Definitely +3 then. :) Amos
