On Thu, 12 May 2005, Jeffrey Hutzelman wrote:
<snip>
All disk-cache arguments makes sense to me at least.
3) OK, so much for the disk cache; now on to the in-memory structures. The CacheItems file is used to store an index of all the cache files, containing the FID, DV, offset, and size of each file in the cache, along with some additional information. It is structured as an array, with one entry (about 40 bytes) for each cache file. Rather than keep all of this data in memory or keep searching it on disk, the cache manager keeps a subset of this data in memory, in dcache entries. The dCacheSize is the number of these entries that are kept in memory.
The default dCacheSize is currently half the number of cache files, but not less than 300 and not more than 2000. I agree this range is probably too small. Something in the range of 2000-10000 would seem reasonable; however, it should _never_ be larger than cacheFiles.
The dCacheSize setting should approximate the size of the workstation's working set of chunks. If the chunk size is large, this is close to the number of files whose contents (not metadata) are in the working set. If the chunk size is very small, then it's probably some multiple of that number, though it likely gets complex.
Unfortunately, I don't know a good way to guess what the size of a random machine's working set is going to be. So we're probably back to using some property of the cache (cacheBlocks or cacheFiles) as an approximation. The existing code uses cacheFiles/2, which might be a little over-agressive, but I suspect that cacheFiles/10 is on the low side. Let's keep it at cacheFiles/2 for now.
I have a faint memory of the memcache only being able to store cachesize/chunksize number of files even if the files are much smaller than chunksize, ie that the memcache is an array of chunksize blocks. If this is correct, then you should probably reduce the chunksize when doing small-sized-memcache to be able to fit enough files.
Regarding the memcache, is there a known maxsize? Again, my faint memory reminds me of someone mumbling that noone has tested memcache at sizes a modern machine can handle (ie gigs of memcache).
4) The vcache stores metadata about files in AFS. Any time we need to<snip>
The default vcache size is set to 300, which is probably way too small. Unfortunately, I still don't know a good way to approximate the size of a workstation's working set. However, the problem is similar to the problem of sizing the dcache, so I'll propose making them dependent on each other, based on the chunk size:
- chunkSize < 13: cacheStatEntries = dCacheSize / 4 - chunkSize < 16: cacheStatEntries = dCacheSize - chunkSize > 16: cacheStatEntries = dCacheSize * 1.5
Further, if cacheStatEntries is configured and dCacheSize is not, then perhaps we should set dCacheSize based on these formulas rather than on cacheFiles, since the configured setting is more likely to reflect the user's impression of the working set size and the amount of memory available to devode to AFS.
Sounds reasonable.
5) The volume cache stores cached information about volumes, including name-to-ID mappings, which volumes have RO clones, and where they are located. The size of the volume cache should approximate the size of the workstation's working set of volumes. Entries in this cache are updated every 2 hours whether they need it or not, so unless you have a busy machine accessing lots of volumes at once, a pretty small number will probably be fine.
Even though it was set for the small memory sizes of the 1980's, the default value of 50 is probably sufficient for single-user systems. For a larger multi-user system, a larger value might be appropriate. I'm going to go out on a limb here and guess that such a system ought to have something like 3-5 volcache entries per active user, bearing in mind that some of these will be used to cover shared volumes and things used by the system.
It seems appropriate to make sure the default is sufficient for both single-user workstations and multi-user systems with a small number of active users. To that end, I'll propose bumping the default number of volume cache entries to, say, 200.
I have a gut feeling that 200 is too small for servers in many environments. Are there further implications of having this larger other than a few bytes of memory spent?
Is there any way to check the current volcache status? Facts are always better than theories ;)
A quick summary of what I've proposed:
-blocks (configured; maybe computed by script) -chunksize 18-20, <= (cacheBlocks/1000 or cacheBlocks/cacheFiles) -files max(1000, cacheBlocks/32, 1.5 * (cacheBlocks/i)) -dcache cacheFiles/2, min 2000, max 10000 -stat chunksize <= 13: dCacheSize / 4 13 < chunksize < 16: dCacheSize chunksize => 16: dCacheSize * 1.5 -volumes 200
A clear step up from the current situation. I have to thank Jeffrey for his thorough explanation of the different tuning options and I sincerely hope that it finds it way into documentation and/or the afsd sourcecode. It's clearly better than what's in there, especially the comments like "another guess, perhaps Honeyman will have a grad student write a paper" ;)
/Nikke - feeling educated :) -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Niklas Edmundsson, Admin @ {acc,hpc2n}.umu.se | [EMAIL PROTECTED] --------------------------------------------------------------------------- "I've got a very bad feeling about this." - Han Solo =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= _______________________________________________ OpenAFS-devel mailing list [email protected] https://lists.openafs.org/mailman/listinfo/openafs-devel
