on 3/28/03 7:30 AM, Ronan Chilvers at [EMAIL PROTECTED] appended the
following bits to my mbox:

> Is this reasonably efficient on your production server ?  Do you have any
> problems with it ?

It's pretty efficient on my production server.  Since I don't index the
initial table, the inserts go quickly.

One possible problem is that there is a MySQL process open for every apache
child.  We run a quad-xeon server with 1 GB of RAM so it's not an issue for

> Also, how are you dealing with growing table sizes ?  Do you summarise out
> data to sub tables ?  Or do you limit the date range and periodically dump
> stuff that is out of range ?

Right now, I'm copying the data nightly from the "live" table to the
"archive" one.  MySQL tends to corrupt index files when you perform long
queries (joins, grouping with conditions, etc) and other clients are trying
to insert data at the same time.  Without indices, the querying is pretty

We only keep the last month of logs in the database. The tables get huge
with hundreds of thousands of records in them.  We still do long term
processing of standard log files with Webalizer.

> Part of the reason for using Perl / PHP, etc was to allow the log data to be
> filtered and sorted prior to inserting to the db.  This would then allow
> automatic summarising so you could maintian host tables, request tables, etc
> and limit the size of the main transfer log tables.

Those are good benefits.  I'm sure it would be faster than working with the
whole dataset.  

Regarding your previous post, I'd probably stick to Perl (apologies to some
on this list).  I know the recent CLI improvements are supposed to help a
lot, but I still feel Perl plays better with the shell, especially with file

> My approach to the queries thus far has been to keep different 'reports'
> (which essentially narrow down to individual SQL queries) in a db.  A user
> picks a report, the SQL is selected and run and the data is then displayed
> (also playing with JPGraph to produce some nice charts).

Cool.  We'd love to see some of your results when they are usable.


Paul Burney


// the statement formerly known as prince
if ($the_elevator == 'tries to bring you down') {
    go_crazy('punch a higher floor');


PHP Database Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Reply via email to