Hi,
Firstly, thanks for bringing these results back to the mailing list...
having seen this sort of problem previously, but without (IIRC) having
done side-by-side comparisons between these various techniques, I'm keen
to see what you find.
"Differentiated Software Solutions Pvt. Ltd" wrote:
> 2. Building the entire DB into a hash variable inside the mod_perl program
> is the fastest.... we found it to be 25 times faster than querying a
> postgres database !!
> 3. We have a problem rebuilding this database in the ram.... even say every
> 1000 requests. We tried using dbm and found it a good compromise solution.
> We found that it is about 8 times faster than postgres querying.
I assume from this that your data changes, but slowly, and you're
getting better performance by accepting that your data be slightly out
of date.
> 4. Another surprising finding was.... we built a denormalised db on the
> Linux file system itself, by using the directory and file name as the key on
> which we wanted to search. We found that dbm was faster than this.
I'm curious about how you're dealing with the concurrency aspect with
solutions 2-3. My guess is that, for 2, you're simply storing a hash in
the memory, which means that each Apache child has its own copy. There
will, every 1000 requests in that child, be the overhead of querying the
DB & rebuilding the hash.
3 presumably means having only _one_ DBMfile. Do the CGI/mod-Perl
processes rebuild this periodically, or is this done offline by another
process? Do the CGI/mod-Perl processes have to wait while writes are
going on?
Cheers
--
Tim Sweetman
A L Digital
---- moving sideways --->
