On 9/4/2014 4:33 AM, Lionel Cons wrote: > On 4 September 2014 03:37, Jeffrey Altman <[email protected]> > wrote: >> On 9/3/2014 1:29 PM, Lionel Cons wrote: >>> Will AFS3 include support for sparse files, e.g. files which have one >>> or multiple holes (see POSIX lseek() documentation about SEEK_HOLE and >>> SEEK_DATA) where no data reside? >>> >>> CERN has a lot of projects which rely on sparse files and it would be >>> a mandatory feature for a migration to AFS3. >>> >>> Lionel >> >> Hi Lionel, >> >> You are writing to the AFS3 Standardization list which is the >> appropriate location for discussing AFS3 RPC protocol extensions. In >> case you aren't aware, in the AFS model file pointer operations are >> local to the client and are satisfied by a cache manager that obtains >> chunks of files at a time from the file servers. Cache managers do not >> in general cache whole files. Instead the cache managers fetch only >> those chunks of the file that are requested by the application. While >> the files themselves might not be sparse the caching of file data by the >> clients will be if that is the access pattern issued by the application. >> >> In the existing AFS3 protocol the FetchData and StoreData operations do >> not support compression and do not support any notion of sparseness (a >> range of zero bytes.) Nor do the file servers recognize that a chunk >> being written is entirely zero bytes and in turn store the file as a >> sparse file in its backend. This is one of the reasons that using ZFS >> with disk compression as the backing store can be a big win. > > There is one very large, and very bad misconception: > Holes in a sparse file return '\0' (zero) bytes on read(), but it does > not mean that all data which contain '\0' bytes are holes. This > misconception can lead to serious data loss situations because > applications need to be able to differ between holes in a file - > representing 'no data here' and large ranges containing '\0' bytes - > which represent valid data containing '\0' bytes.
If you are building your application around such properties you need to be extremely careful to understand the sparse file implementation of the file system that is storing the data. File systems differ widely in their implementations. While I have never seen a file system convert a stream of zero bytes into a hole, doing so is permitted. What is more common is that due to resource allocation strategies holes are only permitted at certain alignments and and non-hole series of zero bytes can be introduced silently. >> In summary, at the present time there is no protocol support for a cache >> manager to communicate the Fetching or Storing of a sparse range nor is >> there any support for a cache manager to query a file server to request >> allocated ranges. As a result is not currently possible for an AFS >> cache manager to implement VFS layer support for lseek(2) SEEK_HOLE and >> SEEK_DATA when it is present. (Nor is there the ability to implement >> Microsoft Windows' FSCTL_SET_SPARSE, FSCTL_SET_ZERO_DATA, and >> FSCTL_QUERY_ALLOCATED_RANGES sparse file operations.) >> >> From an AFS client implementation perspective, for example OpenAFS on >> Linux (3.1 or later) or Solaris it would be possible for the cache >> manager to support SEEK_DATA and SEEK_HOLE as recommended by the Linux >> man page. SEEK_HOLE always returns the offset of the end of file and >> SEEK_DATA always return the offset specified in the lseek(). Discussion >> of adding that level of support should take place on the >> [email protected] mailing list. The implementation I described above is compliant with the specification. Yet it will not meet your requirements. >> >> What are the requirements of the applications in question? > > 1. Databases For databases a bigger problem is the lack of byte range locking in the AFS3 protocol. > 2. Simulation software which, basically described, creates one large > sparse file (typically something like 2PB or larger) and then calls > mmap() on it. Filesystem will allocate space on an as needed basis. > Holes in these files represent areas of 'no data' or 'no data > calculated here yet' while real data - even huge ranges of '\0' bytes > - represent calculated areas. This would be a poor choice for AFS at the moment and most other network file systems. Jeffrey Altman
smime.p7s
Description: S/MIME Cryptographic Signature
