Re: Optimising cache performance

[Clinton Gormley]

Thanks for your feedback - a couple more questions

First, I'm assuming this is for a distributed system running on multiple 
servers.  If not, you should just download one of the cache modules from 
CPAN.  They're good.

For now it's not a distributed system, and I have been using Cache::FileCache.  But that still means freezing and thawing objects - which I'm trying to minimise.

I suggest you use either Cache::Mmap or IPC::MM for your local cache. 
They are both very fast and will save you memory.  Also, Cache::Mmap is 
only limited by the size of your disk, so you don't have to do any purging.

When you say that Cache::Mmap is only limited by the size of your disk, is that because the file in memory gets written to disk as part of VM? ( I don't see any other mention of files in the docs.) Which presumably means resizing your VM to make space for the cache?

You seem to be taking a lot of care to ensure that everything always has 
the latest version of the data.  If you can handle slightly out-of-date 

Call me anal ;) Most of the time it wouldn't really matter, but sometimes it could be extremely off-putting

If everything really does have to be 100% up-to-date, then what you're 
doing is reasonable.  It would be nice to not do the step that checks 
for outdated objects before processing the request, but instead do it in 
a cleanup handler, although that could lead to stale data being used now 
and then.

Yes - had considered that.

If you were using a shared cache like Cache::Mmap, you could have a cron 
job or a separate Perl daemon that simply purges outdated objects every 
minute or so, and leave that out of your mod_perl code completely.

I see the author of IPC::MM has an e-toys address - was this something you used at e-toys?  I know very little about shared memory segments, but is MM used to share small data objects, rather than to keep large caches in shared memory?

Ralph Engelschall writes in the MM documentation :
"The maximum size of a continuous shared memory segment one can allocate depends on the underlaying platform. This cannot be changed, of course. But currently the high-level malloc(3)-style API just uses a single shared memory segment as the underlaying data structure for an MM object which means that the maximum amount of memory an MM object represents also depends on the platform."

What implications does this have on the size of the cache that can be created with IPC::MM


thanks

Clinton Gormley