I've written a 'qmail statistics' script that does the dirty work of
generating qmailanalog-acceptable log files:
- making sure you have an up-to-date pending file
- converting timestamps to fractional time using tai64nfrac
- running current and pending logs through matchup
It then calls qmailanalog's z-commands to analyze the results. It's intended
for creating weekly email messages to root, but it wouldn't be hard to tweak
to generate web pages or some such.
In its current state, it makes a few assumptions:
- your logs are rotated (archived), but are all available in the log
directory
- log stamps are in TAI64N format--at least, they're run through the
tai64nfrac
translator before sending them to matchup, so if they're not in TAI64N
they'd
better have the fractional-time stamps that matchup expects
Everything else that can change from installation to installation should be
defined in the first section ("Files and Paths"); change it as appropriate.
I'm sending it to the list in the hope that it will be useful to others, and
for comments and corrections--I'm sure there's a lot that could be improved.
- Bradey
#!/usr/bin/python
# qmstats.py v0.1
# qmail statistics generator (using qmailanalog)
# Bradey Honsinger <[EMAIL PROTECTED]>
#
# This script does the dirty work of generating an up-to-date pending
# file from archived (rotated) logs. It runs the logs through
# tai64nfrac to convert TAI64N timestamps to the fractional timestamps
# required by qmailanalog's matchup. After an producing a current
# matchup file from the current log file and an up-to-date pending file,
# it runs a list of qmailanalog commands on the current matchup file and
# writes the results to standard output.
#
# Note that the output is only the result of the current log file; since
# my log files are rotated once a week, that means that the statistics
# output are only for messages send and received so far this week.
#
# This script tries to be reasonably intelligent about generating the
# up-to-date pending file: If the pending file is newer than the archived
# log files, it isn't regenerated. If the pending file doesn't exist at all,
# it's regenerated completely, starting with the oldest log file (see the
# matchup man page). Since my log files are rotated weekly, the pending file
# should only need to be updated once a week--nice, since the old log files
# are about 2.5MB each.
#
# CUSTOMIZATION: Any parameters that need to be changed for your qmail
# installation should be in the "Files and Paths" section. If something
# doesn't work, it's most likely because your files are in different
# locations--double-check the paths carefully.
# Path, file, and process manipulation
import os
# String manipulation
import string
###################
# Files & paths
##################
# Logfile path (current and archived logs)
LOGFILE_PATH = "/var/log/qmail/qmail-send/"
# Current logfile
CURRENT_LOGFILE = LOGFILE_PATH + "current"
# Command to retrieve archived logfile names (which start with "@")
ARCHIVED_CMD = "ls -1 " + LOGFILE_PATH + "@*"
# Matchup command location
MATCHUP_CMD = "/usr/local/bin/tai64nfrac |
/usr/local/qmailanalog/bin/matchup"
# Workfile path - any file starting with "qmstat" goes here
WORKFILE_PATH = "/tmp/"
# List of pending messages in archived logs
ARCHIVED_PENDING = WORKFILE_PATH + "qmstat.archived.pending"
# List of current pending messages
CURRENT_PENDING = WORKFILE_PATH + "qmstat.current.pending"
# Matchup from current log + archived pending
CURRENT_MATCHUP = WORKFILE_PATH + "qmstat.current.matchup"
# List of commands (w/headings) to run on current log
CMDS = [
[ "Overall Statistics", "/usr/local/qmailanalog/bin/zoverall" ],
[ "Failures", "/usr/local/qmailanalog/bin/zfailures" ],
[ "Deferrals", "/usr/local/qmailanalog/bin/zdeferrals" ],
# sort whines about a broken pipe, so we throw away its errors
[ "Top Ten Senders", "/usr/local/qmailanalog/bin/zsenders | sort -rk 1
2>/dev/null | head -20" ],
[ "Top Ten Recipients", "/usr/local/qmailanalog/bin/zrecipients | sort -rk
2 2>/dev/null | head -16" ]
]
###################
# Function Definitions
##################
# Filter function: returns true if given file is newer than archived
# pending messages file
def NewerThanArchivedPending( file ):
if ( os.path.getmtime( ARCHIVED_PENDING ) < os.path.getmtime( file ) ):
return 1
else:
return None
# Update archived pending file with archived log
def UpdateArchivedPending( file ):
os.system( "cat %s %s | %s > /dev/null 5> %s" %
( file, ARCHIVED_PENDING, MATCHUP_CMD, ARCHIVED_PENDING ) )
###################
# Program Body
##################
# Get list of archived log files
os.chdir( LOGFILE_PATH )
fileCmd = os.popen( ARCHIVED_CMD )
archivedFiles = fileCmd.readlines( )
archivedFiles = map( string.strip, archivedFiles ) # Remove whitespace
archivedFiles.sort( ) # Most recent archived logfile now last element
fileCmd.close( )
# Update/create pending file with archived log files
if os.path.exists( ARCHIVED_PENDING ):
# Make sure pending contains all archived logs
# Note that archived logs are in order, since we sorted the list above
map( UpdateArchivedPending,
filter( NewerThanArchivedPending, archivedFiles ) )
else:
# Regenerate pending file from archived logs
# (assumes archived logs are complete)
map( UpdateArchivedPending, archivedFiles )
# Matchup current logfile and old pending messages
# (ignore current pending messages)
os.system( "cat %s %s | %s > %s 5> %s" %
( ARCHIVED_PENDING, CURRENT_LOGFILE, MATCHUP_CMD,
CURRENT_MATCHUP, CURRENT_PENDING ) )
# Run listed commands on current matchup file
for cmd in CMDS:
# Print heading
print "***************************************************************"
print cmd[0]
print "----------------"
# Print command output
cmdFile = os.popen( "cat %s | %s" % ( CURRENT_MATCHUP, cmd[1] ) )
cmdOutput = cmdFile.readlines( )
cmdFile.close( )
print string.join( cmdOutput )
# qmail statistics generation script
# Copyright (C) 2000 Construx Software
# Written by Bradey Honsinger <[EMAIL PROTECTED]>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# Refer to <http://www.fsf.org/copyleft/gpl.txt> for a copy of the GNU
# General Public License.
