Hello,
In an attempt to offload SELECT queries from our database and decrease the time spent gathering data, we have begun to aggressively cache data in users' sessions (StateDB=MLDBM::Sync::SDBM_File, StateSerializer=Data::Dumper). Unfortunately, when the data set is too large the process of reading/thawing the data becomes painfully slow. Switching to DB_File didn't seem to help. We haven't tried Storable but my inclination is that it wouldn't make that much of a difference. The session files are on the order of a 100k.
Storable might help. It might be twice as fast as Dumper for large data sets like this.
1) Is the thawing of the data to blame?
Yes. This really is too much data I would say. You should really just try to store small bits in $Session. Any global data might be better to cache in a global data structure, or even use the CacheDB and includes caching.
2) How might we speed this process up or are we heading down a blind
alley?
Try SessionSerialize setting, similar to calling $Session->Lock in the Script_OnStart. Try not accessing keys too often. You might try to make a read only copy of data like this:
use vars qw($SessionReadOnly);
sub Script_OnStart {
$SessionReadOnly = { %$Session };
}so if you are accessing same data multiple times, this might be faster.
3) Lastly, I have heard good things about IPC:MM (fastest gun in the west when it comes to using Shared memory for a cache). Could this be used instead of MLDBM::Sync::SDBM_File?
No, but you could have your own cache layer reference per user by $Session->SessionID. You might also try using Tie::TextDir either for StateDB or for CacheDB ( if you start includes caching ). Note that if you have a really fast local database like MySQL, you might be better off not caching at all & lettings most queries hit the database.
Any hints or help would be appreciated!
My hints were very generic. A lot of it has to do with exactly how you are using $Session programmatically.
Regards,
Josh
________________________________________________________________ Josh Chamas, Founder phone:925-552-0128 Chamas Enterprises Inc. http://www.chamas.com NodeWorks Link Checking http://www.nodeworks.com
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
