I have a faint memory of the memcache only being able to store cachesize/chunksize number of files even if the files are much smaller than chunksize, ie that the memcache is an array of chunksize blocks. If this is correct, then you should probably reduce the chunksize when doing small-sized-memcache to be able to fit enough files.
You bring up an important point which I failed to mention. The proposals I made regarding chunkSize, cacheFiles, and dCacheSize are all relevant only when using disk cache. The memory cache architecture works somewhat differently; memcache chunks are fixed-size and take up the same amount of storage regardless of how "full" they are. So, the cache can store only cachesize/chunksize chunks.
As you suggested, this argues for much smaller chunks than would be used for a disk cache, and in fact the default chunkSize for memcache is only 13 (8K). I'm not suggesting changing this default, though of course folks who think they can spare the memory could choose to raise it (and cacheBlocks) an order of magnitude or two.
Furthermore, separate configuration of cacheFiles and dCacheSize is meaningless. The value of cacheFiles is ignored entirely; the number of chunks is exactly the number that will fit. The value of dCacheSize must be the same as the number of chunks, because memcache has no CacheItems file, and the dcache is the authoritative index rather than merely a cache. In practice what this means is that you cannot simultaneously specify both cacheBlocks and dCacheSize; whichever is left out will be computed from the other.
Regarding the memcache, is there a known maxsize? Again, my faint memory reminds me of someone mumbling that noone has tested memcache at sizes a modern machine can handle (ie gigs of memcache).
I do not believe there is a known maximum; I've certainly not used a memcache on that order. We used to use memcache many years ago during Linux installs, because it was easier to configure in the limited environment available in the installer at that time. These days we use a disk cache on an in-memory ext2 filesystem.
A clear step up from the current situation. I have to thank Jeffrey for his thorough explanation of the different tuning options and I sincerely hope that it finds it way into documentation and/or the afsd sourcecode. It's clearly better than what's in there, especially the comments like "another guess, perhaps Honeyman will have a grad student write a paper"
Well, you know, if Peter had ever found someone to write that paper....
-- Jeffrey T. Hutzelman (N3NHS) <[EMAIL PROTECTED]> Sr. Research Systems Programmer School of Computer Science - Research Computing Facility Carnegie Mellon University - Pittsburgh, PA
_______________________________________________ OpenAFS-devel mailing list [email protected] https://lists.openafs.org/mailman/listinfo/openafs-devel
