Tommy Pham wrote:

    Many of my customers have coming up on 20 years of data available. There has
    been a debate on transferring historic data to a separate database, but
    having it available is not causing a problem, except for some counts and
    larger search actions, and being able to see how long a client has been
    visiting is often useful. Statistical analysis is always done on a separate
    machine, with a replicated copy of the data, so as not to affect the active
    users ...

What kind of counts/filters?  What kind of RAID subsystem is the storage?
What's the total size of the DB?  Up to 20 years of data should be in the peta
range.  In that peta range, if you're not having performance issue and not using
either RAID 0, 0+1, 10, 50, or 60, I'd love to hear about the application and
database design in details. :)

We are still only in hundreds on Mb and historic data is has less detail than the current 'transactions'. The current postcode table is 500Mb, and while the LLPG data would increase that by the order of 100, it's currently only restricted to a councils immediate working area, so we keep the problem contained. Dropping back to postcode for out of area enquiries. Users complain if an enquiry takes more than a few seconds, and Firebird is giving me more than adequate performance, and allows shadow data to be created via triggers to reduce the need for 'counting'.

I have a new 'application' which is using the same search criteria but the data volume is growing a lot faster, 10Gb on the test system here, but I am still seeing the same search speeds once the correct indexes have been generated. But it will take a few more years before that starts reaching the 100Gb level :)

Lester Caine - G8HFL
Contact -
L.S.Caine Electronic Services -
EnquirySolve -
Model Engineers Digital Workshop -
Firebird -

PHP General Mailing List (
To unsubscribe, visit:

Reply via email to