Re: logfile helper
fre 2007-06-01 klockan 22:29 +0100 skrev Robin Bowes: I need something like this for squid-2.6.STABLE2 (built from the FC6 SRPM) Has anyone applied this to squid-2.6? Are there any other log-to-pipe solutions ? There is the Using an external helper to perform Logfile IO project.. http://devel.squid-cache.org/projects.html#s26_logfile_daemon Note: Based on Squid-2.HEAD and not 2.6 as the notes say.. Regards Henrik signature.asc Description: Detta är en digitalt signerad meddelandedel
Re: logfile helper
On Sat, Jun 02, 2007, Henrik Nordstrom wrote: fre 2007-06-01 klockan 22:29 +0100 skrev Robin Bowes: I need something like this for squid-2.6.STABLE2 (built from the FC6 SRPM) Has anyone applied this to squid-2.6? Are there any other log-to-pipe solutions ? There is the Using an external helper to perform Logfile IO project.. http://devel.squid-cache.org/projects.html#s26_logfile_daemon Note: Based on Squid-2.HEAD and not 2.6 as the notes say.. And I'll be happy to port the pipe patch or my logfile stuff to earlier versions of Squid-2.6 if people are willing to make a donation of a few hundred dollars to the project; or it'll have to wait for Squid-3.0 to be released. Adrian
Re: logfile helper
Adrian Chadd wrote: On Sat, Jun 02, 2007, Henrik Nordstrom wrote: fre 2007-06-01 klockan 22:29 +0100 skrev Robin Bowes: I need something like this for squid-2.6.STABLE2 (built from the FC6 SRPM) Has anyone applied this to squid-2.6? Are there any other log-to-pipe solutions ? There is the Using an external helper to perform Logfile IO project.. http://devel.squid-cache.org/projects.html#s26_logfile_daemon Note: Based on Squid-2.HEAD and not 2.6 as the notes say.. And I'll be happy to port the pipe patch or my logfile stuff to earlier versions of Squid-2.6 if people are willing to make a donation of a few hundred dollars to the project; or it'll have to wait for Squid-3.0 to be released. Henrik - thanks for the pointer. Adrian, I'm happy to attempt to port the Logfile IO stuff to 2.6 if I can get some sort of overview of how it works. Do you have any docs describing how your logfile stuff works? Thanks, R.
Re: logfile helper
On Sat, Jun 02, 2007, Robin Bowes wrote: Adrian, I'm happy to attempt to port the Logfile IO stuff to 2.6 if I can get some sort of overview of how it works. Do you have any docs describing how your logfile stuff works? Its pretty simple. the s26_logfile_daemon code abstracts out the logfile processing stuff a little. There's a few calls you need to provide in the logfile struct: * a call which says this is the beginning of the line * a call which adds data to the line * a call which says this is the end of the line * a call which handles rotation * a call which handles flushing the current data to disk (a sync call) It should be straightforward if you look at logfile.c and the logfile struct layout. Its slightly different from how the codebase worked before (where flush seemed to be the end of line and sync call) so don't try to look at the current squid-2 or squid-2.6 codebase. It'll all be incorporated into Squid-3 as I've got a seperate branch which implements a Logfile class to do the above. Implementing a pipe helper will be pretty trivial in Squid-3. Adrian
Re: logfile helper
Adrian Chadd wrote: On Sat, Jun 02, 2007, Robin Bowes wrote: Adrian, I'm happy to attempt to port the Logfile IO stuff to 2.6 if I can get some sort of overview of how it works. Do you have any docs describing how your logfile stuff works? Its pretty simple. the s26_logfile_daemon code abstracts out the logfile processing stuff a little. There's a few calls you need to provide in the logfile struct: * a call which says this is the beginning of the line * a call which adds data to the line * a call which says this is the end of the line * a call which handles rotation * a call which handles flushing the current data to disk (a sync call) It should be straightforward if you look at logfile.c and the logfile struct layout. Its slightly different from how the codebase worked before (where flush seemed to be the end of line and sync call) so don't try to look at the current squid-2 or squid-2.6 codebase. Hmmm, I've had a quick look and it's not clear. But maybe I need to study it in more detail. I don't really grok C/C++ - I can read and understand with effort but it's not something I've worked with much. It sure would be nice to get it in 2.6.STABLE2 ;) It'll all be incorporated into Squid-3 as I've got a seperate branch which implements a Logfile class to do the above. Implementing a pipe helper will be pretty trivial in Squid-3. I doubt we'd jump to squid3 for a while yet - we'll let that settle down for a while after the release. Thanks, R.
Re: logfile helper
On Sat, Jun 02, 2007, Robin Bowes wrote: It should be straightforward if you look at logfile.c and the logfile struct layout. Its slightly different from how the codebase worked before (where flush seemed to be the end of line and sync call) so don't try to look at the current squid-2 or squid-2.6 codebase. Hmmm, I've had a quick look and it's not clear. But maybe I need to study it in more detail. I don't really grok C/C++ - I can read and understand with effort but it's not something I've worked with much. It sure would be nice to get it in 2.6.STABLE2 ;) Its C in s26_logfile_daemon. Just read logfile.c and struct logfile in structs.h, then see how these function pointers are set: LOGLINESTART * f_linestart; LOGWRITE * f_linewrite; LOGLINEEND * f_lineend; LOGFLUSH * f_flush; LOGROTATE * f_rotate; LOGCLOSE * f_close; Adrian
Re: logfile helper
lör 2007-06-02 klockan 12:03 +0100 skrev Robin Bowes: Henrik - thanks for the pointer. Adrian, I'm happy to attempt to port the Logfile IO stuff to 2.6 if I can get some sort of overview of how it works. Do you have any docs describing how your logfile stuff works? s26_logfile_daemon should apply quite cleanly to 2.6 still I think. Maybe some minor edit is needed. If not the older version of the s26_logfile_daemon patch from before Squid-2.6 branched from Squid-2 should.. http://squid.cvs.sourceforge.net/*checkout*/squid/website/patches/squid-s26_logfile_daemon-HEAD.patch?revision=1.4 Regards Henrik signature.asc Description: Detta är en digitalt signerad meddelandedel
Re: logfile helper
On Sun, 11 Sep 2005, Andrey Chichak wrote: I have patched Squid-2.5.STABLE10 to write logs into pipe. Write in config file: cache_access_log |some script and access log would be catched with script. During logrotate pipe is reopened. I think this is useful for realtime log analize. Interesting. There is some areas where this can be improved however. 1. The writes to the pipe should be non-blocking to avoid halting Squid if the script can not keep up. 2. To deal with the pipe being non-blocking you need buffering of the log writes. And a (perhaps configurable) limit of how much log data can be buffered. And then the obvious but less fun, for these changes to make it into the main Squid code tree it needs to be made within the Squid-3 tree. Squid-2.5 is feature frozen since many years back.. Regards Henrik
