Robin Bowes wrote:
John Peacock said the following on 26/11/2005 21:57:
Matt Sergeant wrote:
If we want logging to be parse-able by anything, the perhaps we should
consider the option of outputting key/value pairs, probably separated by
equals signs.
That's kind of horrible for anyone adding in ad-hoc debugging though. :-/
Actually, I kind of see where having a parseable log line might be acceptable
for certain log lines. Imagine it this way:
1) any plugin calling $self->log() gets printed the way we currently display log
lines, i.e. just join(" ", @log).
2) The log lines emitted by the logging plugin itself (i.e. in hook_deny or
hook_reset_transaction) could have a completely different format (one that is
parseable).
The reason this would be OK is that all /ad hoc/ logging shows up as #1, so no
change. The lines emitted in #2 are special, in that they contain additional
context of being at a known terminal state of the transaction, and are more
useful to keep longer term logs.
Right. I'd like to end up with three files:
1. normal log file
2. accepted messages, parseable format
3. rejected messages, parseable format
Ideally, these would be written to three separate files without having
to use multilog filtering to separate them. But I'm not sure how that
would work. I guess the normal log file could be written as it is now
and some other plugin could write directly to two files. Though I'd like
to keep multilog in the loop as it timestamps and rotates nicely.
Is there any way of writing to different file descriptors and having
additional multilog processes read them?
I can see where Robin is going and I think it is one possible way to deal with
it. However, I don't think this particular scheme should be implemented as part
of the core logging/adaptive plugin, though I don't see any reason why it
couldn't be included in the core as a different logging plugin, say
logging/parseable. I would suggest implementing it as an isa_plugin() instead
of duplicating all of the existing logic.
I'm happy to implement this in whatever way is best.
I'm not sure what you mean by isa_plugin() though. Can you elaborate?
R.
isa_plugin just makes some sub into a plugin instance, a
handy thing. "Is a plugin".
Instead of a cron script, use isa_plugin to spin off a log
parser, but I might prefer the log parser to be completely
independent of qpsmtpd. I don't see why it would be a
time critical operation, so a cron poll would work here.
You could make the logging output contain the info,
the parseability targets or identifiers, then independently
parse/move "current" into "qpsmtpd.log.1" in a cron script
if "current" has reached a certain size threshold. That's what
I'm doing with my web logs which are multilog format(with
the pid's mixed but in chrono order per pid). The cron also
handles rotating *.1 to *.2 etc at a size threshold. Of course
that could be something like *.1 and *.deny.1 and *.accept.1
in your terms.
If you knew what your column scheme was you would not
need key/value pairs, just the value, and let your parser
fill in the key later based on column number for human
readable version, perhaps not storing the keys anywhere
but just dynamically inserting them and then pipe to less
or have a server-side dynamic html method to browse logs.
-Bob
