Hello,
After upgrading from apache 1.3 to apache 2.0 I run into problems
with the DBILogger.pm
This packages is used to insert logging informations into a mysql
database.
To work together with the new mod_perl2 api I did lightly changes to
the DBILogger.pm (attached below).
I installed Debian 3.1 Sarge (with almost standard packages) and use
Apache2 (2.0.54-5) with mod_perl2 (1.999.21).
Everything works okay so far. But sometimes an apache child process
exits randomly with
[Wed Feb 01 15:59:44 2006] [notice] child pid 24437 exit signal
Segmentation fault (11)
This doesn't happen with each page request. Approximately every 8 or
10th request causes the segfault in the error.log
Same manner with the most recent mod_perl2 (2.0.2)
Any ideas about what is the problem?
it follows my config:
Used Components and their Configuration:
*** mod_perl version 1.999021
*** using /usr/lib/perl5/Apache2/Apache/BuildConfig.pm
*** Makefile.PL options:
MP_APR_LIB => aprext
MP_APXS => /usr/bin/apxs2
MP_CCOPTS => -g -Wall
MP_COMPAT_1X => 1
MP_DEBUG => 1
MP_GENERATE_XS => 1
MP_INCLUDE_DIR => /usr/include/apache2 /usr/include/apr-0
MP_INST_APACHE2 => 1
MP_LIBNAME => mod_perl
MP_TRACE => 1
MP_USE_DSO => 1
MP_USE_GTOP => 1
MP_USE_STATIC =>
*** /usr/bin/perl -V
Summary of my perl5 (revision 5 version 8 subversion 4) configuration:
Platform:
osname=linux, osvers=2.6.15, archname=i386-linux-thread-multi
uname='linux ninsei 2.6.15 #1 smp preempt sat jan 7 12:47:52 pst
2006 i686 gnulinux '
config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -
Dcccdlflags=-fPIC -Darchname=i386-linux -Dprefix=/usr -Dprivlib=/usr/
share/perl/5.8 -Darchlib=/usr/lib/perl/5.8 -Dvendorprefix=/usr -
Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/
usr/local -Dsitelib=/usr/local/share/perl/5.8.4 -Dsitearch=/usr/local/
lib/perl/5.8.4 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/
man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/
man3 -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -
Uafs -Ud_csh -Uusesfio -Uusenm -Duseshrplib -Dlibperl=libperl.so.
5.8.4 -Dd_dosuid -des'
hint=recommended, useposix=true, d_sigaction=define
usethreads=define use5005threads=undef useithreads=define
usemultiplicity=define
useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
use64bitint=undef use64bitall=undef uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -
DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
optimize='-O2',
cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -
DDEBIAN -fno-strict-aliasing -I/usr/local/include'
ccversion='', gccversion='3.3.5 (Debian 1:3.3.5-13)',
gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
ivtype='long', ivsize=4, nvtype='double', nvsize=8,
Off_t='off_t', lseeksize=8
alignbytes=4, prototype=define
Linker and Libraries:
ld='cc', ldflags =' -L/usr/local/lib'
libpth=/usr/local/lib /lib /usr/lib
libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
perllibs=-ldl -lm -lpthread -lc -lcrypt
libc=/lib/libc-2.3.2.so, so=so, useshrplib=true,
libperl=libperl.so.5.8.4
gnulibc_version='2.3.2'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib'
Characteristics of this binary (from libperl):
Compile-time options: MULTIPLICITY USE_ITHREADS USE_LARGE_FILES
PERL_IMPLICIT_CONTEXT
Built under linux
Compiled at Jan 12 2006 14:54:46
%ENV:
PERL_LWP_USE_HTTP_10="1"
@INC:
/etc/perl
/usr/local/lib/perl/5.8.4
/usr/local/share/perl/5.8.4
/usr/lib/perl5
/usr/share/perl5
/usr/lib/perl/5.8
/usr/share/perl/5.8
/usr/local/lib/site_perl
.
*** Packages of interest status:
Apache::Request: -
CGI : 3.04
LWP : 5.803
mod_perl : 1.999021
[Wed Feb 01 15:59:44 2006] [notice] child pid 24437 exit signal
Segmentation fault (11), possible coredump in /var/lib/apache2/tmp
(gdb) bt
#0 0x405c8cee in modperl_dir_config ()
from /usr/lib/apache2/modules/mod_perl.so
#1 0x40bb93f9 in XS_Apache__RequestRec_dir_config ()
from /usr/lib/perl5/Apache2/auto/Apache/RequestUtil/RequestUtil.so
#2 0x4067bb09 in Perl_pp_entersub () from /usr/lib/libperl.so.5.8
#3 0x40673ce9 in Perl_runops_standard () from /usr/lib/libperl.so.5.8
#4 0x40616459 in Perl_call_sv () from /usr/lib/libperl.so.5.8
#5 0x4061623d in Perl_call_sv () from /usr/lib/libperl.so.5.8
#6 0x40c85b0d in ?? () from /usr/lib/perl5/Apache2/auto/APR/Pool/
Pool.so
#7 0x087ce470 in ?? ()
#8 0x086bf2e4 in ?? ()
#9 0x00000004 in ?? ()
#10 0x4030f5d8 in ?? () from /usr/lib/libapr-0.so.0
#11 0x089bb378 in ?? ()
#12 0x08a2a778 in ?? ()
#13 0x087687f8 in ?? ()
#14 0x4030f5d8 in ?? () from /usr/lib/libapr-0.so.0
#15 0x089bd418 in ?? ()
#16 0x089bd408 in ?? ()
#17 0xbffff198 in ?? ()
#18 0x4030b76d in apr_pool_cleanup_run () from /usr/lib/libapr-0.so.0
#19 0x4030b76d in apr_pool_cleanup_run () from /usr/lib/libapr-0.so.0
#20 0x4030ae09 in apr_pool_clear () from /usr/lib/libapr-0.so.0
---Type <return> to continue, or q <return> to quit---
#21 0x080767c5 in ap_graceful_stop_signalled ()
#22 0x08076ac7 in ap_graceful_stop_signalled ()
#23 0x08076be8 in ap_graceful_stop_signalled ()
#24 0x0807745a in ap_mpm_run ()
#25 0x0807da8d in main ()
----------------------------------------------------------
# file: /etc/apache2/mods-enabled/perl.conf
PerlModule Apache2
PerlRequire /etc/apache2/mod_perl2_startup.pl
PerlLogHandler Apache::DBILogger2
PerlSetVar DBILogger_data_source DBI:mysql:host=localhost;database=log
PerlSetVar DBILogger_table apache2
PerlSetVar DBILogger_username apachelog_user
PerlSetVar DBILogger_password apachelog_pass
-----------------------------------------------------
# file: /etc/apache2/mod_perl2_startup.pl
use ModPerl::MethodLookup;
ModPerl::MethodLookup::preload_all_modules();
use ModPerl::Util (); #for CORE::GLOBAL::exit
use Apache2::RequestRec ();
use Apache2::RequestIO ();
use Apache2::RequestUtil ();
use Apache2::DBILogger2;
use Date::Format;
use APR::Table ();
use ModPerl::Registry ();
use Apache2::Const -compile => ':common';
use APR::Const -compile => ':common';
1;
------------------------------------------
# file: /usr/local/lib/perl/5.8.4/Apache2/DBILogger2.pm
package Apache::DBILogger2;
require 5.004;
#use strict;
use warnings;
use Apache2::Const -compile => qw(OK DECLINED);
use Apache2::RequestRec ();
use Apache2::Connection ();
use DBI;
use Date::Format;
$Apache::DBILogger2::revision = sprintf("%d.%02d", q$Revision: 1.20 $
=~ /(\d+)\.(\d+)/o);
$Apache::DBILogger2::VERSION = "0.93";
sub reconnect($$) {
my ($dbhref, $r) = @_;
$$dbhref->disconnect;
$r->log_error("Reconnecting to DBI server");
$$dbhref = DBI->connect($r->dir_config("DBILogger_data_source"), $r-
>dir_config("DBILogger_username"), $r->dir_config
("DBILogger_password"));
unless ($$dbhref) {
$r->log_error("Apache::DBILogger2 could not connect to ".$r-
>dir_config("DBILogger_data_source")." - ".$DBI::errstr);
return Apache::DECLINED;
}
}
sub logger {
# my @args = (@_ && ref $_[0] eq ARRAY) ? @{ +shift } : shift
# my $r = shift;
my $s = $r->server;
my $c = $r->connection;
my %data = (
'server' => $s->server_hostname,
'bytes' => $r->bytes_sent,
'filename' => $r->filename || '',
'remotehost'=> $c->remote_host || '',
'remoteip' => $c->remote_ip || '',
'status' => $r->status || '',
'urlpath' => $r->uri || '',
'referer' => $r->headers_in->{"Referer"} || '',
'useragent' => $r->headers_in->{'User-Agent'} || '',
'timeserved'=> time2str("%Y-%m-%d %X", time),
'contenttype' => $r->content_type || ''
);
if (my $user = $r->user()) {
$data{user} = $user;
}
$data{usertrack} = $r->pnotes('cookie') || '';
my $dbh = DBI->connect($r->dir_config("DBILogger_data_source"), $r-
>dir_config("DBILogger_username"), $r->dir_config
("DBILogger_password"));
unless ($dbh) {
$r->log_error("Apache::DBILogger could not connect to ".$r-
>dir_config("DBILogger_data_source")." - ".$DBI::errstr);
return Apache::DECLINED;
}
my @valueslist;
foreach (keys %data) {
$data{$_} = $dbh->quote($data{$_});
push @valueslist, $data{$_};
}
my $table = $r->dir_config("DBILogger_table") || 'requests';
my $statement = "insert into $table (". join(',', keys %data) .")
VALUES (". join(',', @valueslist) .")";
my $tries = 0;
TRYAGAIN: my $sth = $dbh->prepare($statement);
unless ($sth) {
$r->log_error("Apache::DBILogger2 could not prepare sql query
($statement): $DBI::errstr");
return Apache::DECLINED;
}
my $rv = $sth->execute;
unless ($rv) {
$r->log_error("Apache::DBILogger2 had problems executing query
($statement): $DBI::errstr");
# unless ($tries++ > 1) {
# &reconnect(\$dbh, $r);
# goto TRYAGAIN;
# }
}
$sth->finish;
$dbh->disconnect;
Apache::OK;
}
# #perl pun: <q[merlyn]> windows is for users who can't handle the
power of the mac.
sub handler {
$r=shift;
# $r->log_error("Reconnecting to DBI server");
$r->connection->pool->cleanup_register( "Apache::DBILogger2::logger" );
return Apache::OK;
}
1;
__END__
=head1 NAME
Apache::DBILogger - Tracks what's being transferred in a DBI database
=head1 SYNOPSIS
# Place this in your Apache's httpd.conf file
PerlLogHandler Apache::DBILogger
PerlSetVar DBILogger_data_source DBI:mysql:httpdlog
PerlSetVar DBILogger_username httpduser
PerlSetVar DBILogger_password secret
PerlSetvar DBILogger_table requests
Create a database with a table named B<requests> like this:
CREATE TABLE requests (
server varchar(127) DEFAULT '' NOT NULL,
bytes mediumint(9) DEFAULT '0' NOT NULL,
user varchar(15) DEFAULT '' NOT NULL,
filename varchar(200) DEFAULT '' NOT NULL,
remotehost varchar(150) DEFAULT '' NOT NULL,
remoteip varchar(15) DEFAULT '' NOT NULL,
status smallint(6) DEFAULT '0' NOT NULL,
timeserved datetime DEFAULT '0000-00-00 00:00:00' NOT NULL,
contenttype varchar(50) DEFAULT '' NOT NULL,
urlpath varchar(200) DEFAULT '' NOT NULL,
referer varchar(250) DEFAULT '' NOT NULL,
useragent varchar(250) DEFAULT '' NOT NULL,
usertrack varchar(100) DEFAULT '' NOT NULL,
KEY server_idx (server),
KEY timeserved_idx (timeserved)
);
Its recommended that you include
use Apache::DBI;
use DBI;
use Apache::DBILogger;
in your startup.pl script. Please read the Apache::DBI documentation for
further information.
=head1 DESCRIPTION
This module tracks what's being transfered by the Apache web server in a
SQL database (everything with a DBI/DBD driver). This allows to get
statistics (of almost everything) without having to parse the log
files (like the Apache::Traffic module, just in a "real" database,
and with
a lot more logged information).
Apache::DBILogger will track the cookie from 'mod_usertrack' if it's
there.
After installation, follow the instructions in the synopsis and restart
the server.
The statistics are then available in the database. See the section
VIEWING
STATISTICS for more details.
=head1 PREREQUISITES
You need to have compiled mod_perl with the LogHandler hook in order
to use this module. Additionally, the following modules are required:
o DBI
o Date::Format
=head1 INSTALLATION
To install this module, move into the directory where this file is
located and type the following:
perl Makefile.PL
make
make test
make install
This will install the module into the Perl library directory.
Once installed, you will need to modify your web server's configuration
file so it knows to use Apache::DBILogger during the logging phase.
=head1 VIEWING STATISTICS
Please see the bin/ directory in the distribution for a
statistics script.
Some funny examples on what you can do might include:
=over 4
=item hit count and total bytes transfered from the virtual server
www.company.com
select count(id),sum(bytes) from requests
where server="www.company.com"
=item hit count and total bytes from all servers, ordered by number
of hits
select server,count(id) as hits,sum(bytes) from requests
group by server order by hits desc
=item count of hits from macintosh users
select count(id) from requests where useragent like "%Mac%"
=item hits and total bytes in the last 30 days
select count(id),sum(bytes) from requests where
server="www.company.com" and TO_DAYS(NOW()) -
TO_DAYS(timeserved) <= 30
This is pretty unoptimal. It would be faster to calculate the dates
in perl and write them in the sql query using f.x. Date::Format.
=item hits and total bytes from www.company.com on mondays.
select count(id),sum(bytes) from requests where
server="www.company.com" and dayofweek(timeserved) = 2
=back
It's often pretty interesting to view the referer info too.
See your sql server documentation of more examples. I'm a happy mySQL
user, so I would continue on
http://www.tcx.se/Manual_chapter/manual_toc.html
=head1 LOCKING ISSUES
MySQL 'read locks' the table when you do a select. On a big table
(like a large httpdlog) this might take a while, where your httpds
can't insert new logentries, which will make them 'hang' until the
select is done.
One way to work around this is to create another table
(f.x. requests_insert) and get the httpd's to insert to this table.
Then run a script from crontab once in a while which does something
like this:
LOCK TABLES requests WRITE, requests_insert WRITE
insert into requests select * from requests_insert
delete from requests_insert
UNLOCK TABLES
You can use the moverows.pl script from the bin/ directory.
Please note that this won't work if you have any unique id field!
You'll get duplicates and your new rows won't be inserted, just
deleted. Be careful.
=head1 TRAPS
I've experienced problems with 'Packets too large' when using
Apache::DBI, mysql and DBD::mysql 2.00 (from the Msql-mysql 1.18x
packages). The DBD::mysql module from Msql-mysql 1.19_17 seems to
work fine with Apache::DBI.
You might get problems with Apache 1.2.x. (Not supporting
post_connection?)
=head1 SUPPORT
This module is supported via the mod_perl mailinglist
(modperl@apache.org, subscribe by sending a mail to
[EMAIL PROTECTED]).
I would like to know which databases this module have been tested on,
so please mail me if you try it.
The latest version can be found on your local CPAN mirror or at
C<ftp://ftp.netcetera.dk/pub/perl/>
=head1 AUTHOR
Copyright (C) 1998, Ask Bjoern Hansen <[EMAIL PROTECTED]>. All rights
reserved. This module is free software; you may redistribute it and/or
modify it under the same terms as Perl itself.
=head1 SEE ALSO
perl(1), mod_perl(3)
=cut
------------------------------------------
