John Peacock wrote:
Matt Sergeant wrote:
I still need to do some work on the logging - it currently only logs
the fd# with the lines to distinguish them. So you have to find the
connecting IP, and find the fd# and grep out lines for that fd#. Kind
of a pain to say the least.
But if you had the hook_reject log lines like Robin is suggesting, it
would be a simple matter to grep out the summary line with all of the
information in one place.
I'm liking his scheme more and more...
John
I've been able to duplicate Robin's data except for the return
detail--I'm only logging the plugin name for denies--the end
result is about the same grepping *.s files output at loglevel 7,
plus cheating by loading up relayclients into an array.
With Robin's method we could use loglevel 4 and emit the
summary line at LOGWARN. I like that, versus loglevel 7,
because at loglevel 7 I would need to store 20G of log files
to cover three days time if I was Matt Sergeant, or
100 200MB *.s files. It would take continuous
processing to grep the logs down to one csv line
per message, and at times log processing would
lag concurrent messages by 8:1. And I haven't factored
in a merge or join with dspam log, using message id's.
At loglevel 7, I'm compressing/grepping 164 lines into one
comma-separated value line, at a rate of 70,000 lines per
second. At first I had sed and awk going faster than perl,
but got my regexes and conditionals optimized and then
perl was more than twice as fast, plus awk won't handle
spaces in localpart.
-Bob
