Jim Gallacher wrote:
It just occured to me that the performance problem may be related to
opening and closing the dbm file for every record insertion. Adjusting
the test so that the file is only opened once, I get O(1), and a great
speed boost: 0.2 seconds / per 1000 records all the way up to 50,000
records. At that point I start to see period performance hits due to
disk syncing, but that is to be expected.
I have no idea what to do with this knowledge unless we can figure out a
way to keep the dbm file open across multiple requests. Ideas??
Keeping the dbm file open over multiple requests is really no different than
the connection pool problem. I did run into problems with corrupt databases
when I kept the file open a very long time, but that was a couple years ago.
This may not be a problem anymore. I'll have to test it and see for myself.
As for the MySQL implementation, well, I've been promising it many
times, but I can provide a connection pool implementation that could
speed up applicative code as well as your session management code.
What I would need to do is to make it mod_python friendly, i.e. make
it configurable through PythonOption directives. Do you think it would
be a good idea to integrate it into mod_python ?
Connection pooling seems like a common request on the mailing list, so
I'd say yes.
There seem to be a lot of things that would be good to include in
mod_python, as they are often asked questions on the mailing list, but I
suggest a "contrib" or "example" dir to hold these things, since they're not
likely to work for everybody in every situation without some tweaks. That
way people can edit and use these things in their projects without having to
worry about editing installed modules in site-packages/mod_python.
Nick
