On Thu, Oct 06, 2005 at 03:49:10PM -0400, Brian Akins wrote:
Colm MacCarthaigh wrote:
If httpd writes a complete line, to any kind of a file descriptor,
anything beyond that is out of our control and becomes a question of the
quality of the piped logger, filesystem or whatever else is on the
My problem with piped loggers is there is no fast way to make sure you
have a complete line. This is especially hard when you buffer the logs.
Think of a situation where the piped logger is supposed to write to a
socket, for example. If the piped logger has the ability to fail over
to
On Thu, Oct 06, 2005 at 02:06:24PM -0400, Brian Akins wrote:
My problem with piped loggers is there is no fast way to make sure you
have a complete line. This is especially hard when you buffer the logs.
If httpd writes a complete line, to any kind of a file descriptor,
anything beyond that
Colm MacCarthaigh wrote:
If httpd writes a complete line, to any kind of a file descriptor,
anything beyond that is out of our control and becomes a question of the
quality of the piped logger, filesystem or whatever else is on the other
side of that file-descriptor.
Maybe I'm just being
Colm MacCarthaigh wrote:
But in order to save overhead, this would require some intelligence, it
would not make much sense for the pluggable logger to re-parse this
string everytime, to figure out what it should be doing. And where does
it get its database, username and host information from?
I run a system with about 250 virtual hosts averaging around 100 hits per
second with little problem. Rather than having a piped log for each
virtual host though, we only use one piped log for every virtual host, and
add in the host name into the log. Your logging program then has ot have
the
Jeffrey Burgoyne wrote:
I run a system with about 250 virtual hosts averaging around 100 hits per
second with little problem.
100 hits per second is not that much relative to what we do. We may be
more of an extreme case, though.
Rather than having a piped log for each
virtual host
On Tue, Oct 04, 2005 at 08:37:08AM -0400, Brian Akins wrote:
In this case, from my patches:
LogFormat INSERT INTO foo VALUES ('%h', '%l'); foo-sql
CustomLog mysql://user:[EMAIL PROTECTED]/database foo-sql
and the mysql module would get the arrays of strings and lengths. at
init time,
On Mon, 2005-10-03 at 23:28 +0100, Colm MacCarthaigh wrote:
And after all of this, what if any, are the compelling reasons to
implement this in httpd at all? Why can't all of this be moved into
piped loggers? Why can't they just parse the logs as they get them and
do whatever whoever wants
On Tue, Oct 04, 2005 at 02:59:40PM +0200, Ondrej Sury wrote:
No, it cannot be implemented with two netcat commands (just tried it).
Sure, it can, use my Multicast Netcat;
http://people.heanet.ie/~colmmacc/mnc/
(version 1.3 is experimental and not working right now, avoid that, and
I'm
On Tue, 2005-10-04 at 13:55 +0100, Colm MacCarthaigh wrote:
On Tue, Oct 04, 2005 at 08:37:08AM -0400, Brian Akins wrote:
In this case, from my patches:
LogFormat INSERT INTO foo VALUES ('%h', '%l'); foo-sql
CustomLog mysql://user:[EMAIL PROTECTED]/database foo-sql
and the mysql
On Tue, 2005-10-04 at 14:06 +0100, Colm MacCarthaigh wrote:
On Tue, Oct 04, 2005 at 02:59:40PM +0200, Ondrej Sury wrote:
No, it cannot be implemented with two netcat commands (just tried it).
Sure, it can, use my Multicast Netcat;
Didn't know that anything like that exists. My comment
Colm MacCarthaigh wrote:
On Tue, Oct 04, 2005 at 08:37:08AM -0400, Brian Akins wrote:
No, I don't think it's so simple at all. Although it would have to parse
the SQL at init time (how does LogFormat know to ask the MySQL provider
to do this?)
LogFormat doesn't, but the init of the provider
On Tue, Oct 04, 2005 at 09:15:32AM -0400, Brian Akins wrote:
Why not just replace mod_log_config rather than plug into it.
Because mod_log_config handles a bunch of stuff for us. May be a
better solution would be to use the standard log_config way of
replacing init and writer and replace
Akins, Brian wrote:
CustomLog mysql://something common env=images
CustomLog file:///logs/my.log combined
CustomLog spread://somegroup refere
CustomLog buffer:///logs/other.log common
This patch implements the above. Within mod_log_config two providers
are provided: file and buffer. If no
On Mon, Oct 03, 2005 at 11:40:27AM -0400, Brian Akins wrote:
CustomLog mysql://something common env=images
CustomLog file:///logs/my.log combined
CustomLog spread://somegroup refere
CustomLog buffer:///logs/other.log common
This patch implements the above. Within mod_log_config two
Colm MacCarthaigh wrote:
Looks useful, but file://|/bin/foo would be very non-intuitive for piped
loggers, balancing the backwards compatibility might need a bit more, I
guess pipe:// or cmd:// schemes might make sense also.
True. I just used uri's as they seemed to just fit. I'm not
Jim Jagielski wrote:
'pipe' is clearer I think...
sure. Of course I just discovered that pip logs don't work. I think
it's the spaces in the command, ie |bin/rotatelogs /var/logs/logfile
5M. Hmmm.. looking into it...
--
Brian Akins
Lead Systems Engineer
CNN Internet Technologies
On 10/03/2005 08:49 PM, Colm MacCarthaigh wrote:
On Mon, Oct 03, 2005 at 11:40:27AM -0400, Brian Akins wrote:
[..cut..]
Looks useful, but file://|/bin/foo would be very non-intuitive for piped
I guess you still can use |/bin/foo because file is the default provider.
loggers, balancing
Rüdiger Plüm wrote:
Makes also sense to me since it seems to me that piped logging does not really
play
well with other things like spread or mysql. On the other side, what about the
buffered logging? Would it make sense to make it possible to turn this on
and off with each provider? If yes,
Jim Jagielski wrote:
True. I just used uri's as they seemed to just fit. I'm not
particularly tied to the configuration interface. Any interest in me
changing it to use cmd://?
'pipe' is clearer I think...
This patch just adds pipe:// as a log uri
--
Brian Akins
Lead Systems
On Mon, 2005-10-03 at 11:40 -0400, Brian Akins wrote:
Akins, Brian wrote:
CustomLog mysql://something common env=images
CustomLog file:///logs/my.log combined
CustomLog spread://somegroup refere
CustomLog buffer:///logs/other.log common
This patch implements the above. Within
Ondrej Sury wrote:
Just quick thought, maybe we should add:
ap_log_writer_close *close;
to struct log_provider_t. It's not absolutely necessary, because you
can use apr_pool_cleanup_register(...), but it will make writing addon
modules much cleaner.
That's debatable. I already use
On 10/03/2005 09:11 PM, Brian Akins wrote:
Rüdiger Plüm wrote:
Makes also sense to me since it seems to me that piped logging does
not really play
well with other things like spread or mysql. On the other side, what
about the
buffered logging? Would it make sense to make it possible to
Rüdiger Plüm wrote:
Or it likes to read many lines in one block as it can commit such things
as a batch rather than as single commits per line. Of course it is not required
to use DB transactions on mysql. But for Oracle this might improve performance.
but that would not be buffering in the
On 10/3/05, Brian Akins [EMAIL PROTECTED] wrote:
Rüdiger Plüm wrote: Or it likes to read many lines in one block as it can commit such things as a batch rather than as single commits per line. Of course it is not required to use DB transactions on mysql. But for Oracle this might improve
On 10/03/2005 09:53 PM, Brian Akins wrote:
Rüdiger Plüm wrote:
[..cut..]
but that would not be buffering in the sense that mod_log_config does
it. mod_log_config just keeps appending data to a buffer. in an sql
logger, to buffer you would keep a list/array/ring of log lines and
wrap
On Mon, Oct 03, 2005 at 11:40:27AM -0400, Brian Akins wrote:
Akins, Brian wrote:
CustomLog mysql://something common env=images
CustomLog file:///logs/my.log combined
CustomLog spread://somegroup refere
CustomLog buffer:///logs/other.log common
I've been looking more at this, and I'm kind
28 matches
Mail list logo
