I posted a question a few weeks back about trying to
log bandwidth used by virtual sites.
i was going about it in the completely wrong way, trying to sniff
at the network level. Plenty solutions for doing it that way by IP
address, but i was wanting it done via uri.
Anyways, i was dumb. Apache has another logging statement
TransferLog
which provides IP, date, file and bytes transferred.
Just use the statement in your <VirtualHost> block
e.g.
<VirtualHost 192.168.1.74>
ServerName www.8bitrecs.com
<snip...>
ErrorLog /usr/local/apache/8bitrecs/logs/8bitrecs_error.log
CustomLog /usr/local/apache/8bitrecs/logs/8bitrecs_access.log combined
TransferLog /usr/local/apache/8bitrecs/logs/8bit_t_log
</VirtualHost>
and you get lines in the form:
193.82.57.45 - - [15/Mar/2002:05:01:22 -0800] "GET / HTTP/1.1" 200 36
193.82.57.45 - - [14/Mar/2002:05:39:45 -0800] "GET /password.mp3 HTTP/1.1"
200 3044542
with the last entry per line being bytes transferred.
Anyways, quick hack, i wrote a perl script which reads a text file first,
which corresponds sites to logfiles e.g.
www.interphaze.org:/usr/local/apache/interphaze.org/logs/transfer_log
www.8bitrecs.com:/usr/local/apache/8bitrecs.com/logs/transfer_log
and then goes through each sites TransferLog file to summarize bytes
transferred daily.
The output it gives looks like:
###########################
Summary for www.beggars.com:
Total for 15 Mar 2002 is 42322522 bytes transferred
Summary for www.skam.com:
Total for 14 Mar 2002 is 2237 bytes transferred
Total for 15 Mar 2002 is 74564633 bytes transferred
Summary for www.cissme.com:
Total for 15 Mar 2002 is 4801 bytes transferred
##################
...and on... and on...
Anyone who is hosting multiple sites and wants to bill accordingly or is
just curious as to which sites are eating the most bandwidth may find it
interesting.
if you can shave any code from the script, that would be cool as well.
i'm not the most frugal with my code, especially the regex i'm sure can be
mightily improved.
cheers,
thor
--
[[EMAIL PROTECTED] ]#
#!/usr/bin/perl -w
#
# For parsing TransferLog format files from Apache
# Each site listed in $logfiles has its logfile parsed and
# summarized per day.
# thorsten sideb0ard march 2002 london town.
use strict;
my %sites;
my $logfiles = "./TransferLogFiles.txt";
open_log_lists();
scan_logs();
sub open_log_lists {
open (LOGLIST,$logfiles) || die "Can't get a handle on log lists: $!\n";
while (<LOGLIST>) {
next if /^$/;
next if /^#/;
(my $site,my $logfile) = split(':',$_);
$sites{$site} = $logfile;
}
}
sub scan_logs {
foreach my $s (keys %sites) {
my %timeline;
open (LOGFILE, $sites{$s}) || die "Cannot open log file: $!\n";
my @transfers=(<LOGFILE>);
foreach my $t (@transfers) {
my $day;
my $bytes;
if ($t =~ /\[([0-2][0-9]\/[a-zA-Z]{3}\/[0-9]{4}):.*\s([0-9]+)$/) {
$day = $1;
$bytes = $2;
$day =~ (s/\//" "/eg);
next if ! $day and $bytes;
if (exists($timeline{$day})) {
$timeline{$day} += $bytes;
} else {
$timeline{$day} = $bytes;
}
}
}
print "Summary for $s:\n";
foreach my $d (keys %timeline) {
print "Total for $d is $timeline{$d} bytes transferred\n";
}
print "\n";
}
}
