On 5/16/2010 7:11 PM, Jason R. Banfelder wrote: > Hello, > > I am evaluating AFS (currently in the reading stage) and have a couple > of questions regarding the applicability of AFS to environments with > large-ish files (some files in our workflows are ca. 250 GB) that I am > hoping the community may be able to help with... > > First, does AFS impose any limits on the size of the client disk cache? > I am most concerned about x86_64 Linux clients, where I don't see any > mention of limits in the man pages. I did find mention of limits on some > (older?) references the Windows client on the web, but it was not clear > if they were AFS imposed limits or just local policy. I'd be looking at > a 1 to 2 TB cache on Linux clients -- is that supported/consistent with > best practices?
The existing cache size limit on 32-bit Windows is the result of the architecture decision to access the cache file as a system paging file. The UNIX disk cache architecture is very different. There is no hard limit on the size of the cache but I suspect that although you can create a 2TB cache the performance may have some inefficiencies due to the fact that the implementation was never designed to handle caches that large. In other words, you should try it but be aware that you might want to apply resources to profiling the source code in this use case. We would be very interested in seeing improvements in this area. > Second, is there any description of chunking behavior that I can read up > on? The asfd man page states that "AFS uses partial file transfer", but > it is not clear in which direction(s?), and if this only the RPC level. > If I run the UNIX head command on a 250GB file, will the whole file be > transferred to the client's cache (this would take multiple minutes in > our environment) or just the first chunk? If the whole file is > transferred, would the head command be able to return the first ten > lines after the first chunk arrived on the client's cache, or will the > read block until the whole file is transferred? The OpenAFS cache managers do not require whole file transfers in order to provide access. If you have a 2TB file and only read 1 byte from it, one chunk of data will be read and stored in the cache. > Thanks in advance for your help and consideration. Jeffrey Altman
smime.p7s
Description: S/MIME Cryptographic Signature
