Re: Metadata writtenback notification? -- was Re: fscache: Redesigning the on-disk cache

2021-03-09 Thread David Howells
Dave Chinner wrote: > > > There was a discussion about fsyncing a range of files on LSFMM [1]. > > > In the last comment on the article dchinner argues why we already have > > > that > > > API (and now also with io_uring(), but AFAIK, we do not have a useful > > > wait_for_sync() API. And it

Re: fscache: Redesigning the on-disk cache

2021-03-09 Thread David Howells
Dave Chinner wrote: > > > With ->fiemap() you can at least make the distinction between a non > > > existing and an UNWRITTEN extent. > > > > I can't use that for XFS, Ext4 or btrfs, I suspect. Christoph and Dave's > > assertion is that the cache can't rely on the backing filesystem's metadata

Re: Metadata writtenback notification? -- was Re: fscache: Redesigning the on-disk cache

2021-03-08 Thread Matthew Wilcox
On Tue, Mar 09, 2021 at 09:32:47AM +1100, Dave Chinner wrote: > On Mon, Mar 08, 2021 at 11:28:41AM +, David Howells wrote: > > Possibly it's sufficient to just clear the excess page space before > > writing, but that doesn't necessarily stop a writable mmap from > > scribbling

Re: Metadata writtenback notification? -- was Re: fscache: Redesigning the on-disk cache

2021-03-08 Thread Dave Chinner
On Mon, Mar 08, 2021 at 11:28:41AM +, David Howells wrote: > Amir Goldstein wrote: > > > > But after I've written and sync'd the data, I set the xattr to mark the > > > file not open. At the moment I'm doing this too lazily, only doing it > > > when a netfs file gets evicted or when the

Re: fscache: Redesigning the on-disk cache

2021-03-08 Thread Dave Chinner
On Mon, Mar 08, 2021 at 09:13:55AM +, David Howells wrote: > Amir Goldstein wrote: > > > > (0a) As (0) but using SEEK_DATA/SEEK_HOLE instead of bmap and opening the > > > file for every whole operation (which may combine reads and writes). > > > > I read that NFSv4 supports hole

Re: fscache: Redesigning the on-disk cache

2021-03-08 Thread David Howells
J. Bruce Fields wrote: > On Mon, Mar 08, 2021 at 09:13:55AM +, David Howells wrote: > > Amir Goldstein wrote: > > > With ->fiemap() you can at least make the distinction between a non > > > existing > > > and an UNWRITTEN extent. > > > > I can't use that for XFS, Ext4 or btrfs, I suspect.

Re: fscache: Redesigning the on-disk cache

2021-03-08 Thread J. Bruce Fields
On Mon, Mar 08, 2021 at 09:13:55AM +, David Howells wrote: > Amir Goldstein wrote: > > With ->fiemap() you can at least make the distinction between a non existing > > and an UNWRITTEN extent. > > I can't use that for XFS, Ext4 or btrfs, I suspect. Christoph and Dave's > assertion is that

Metadata writtenback notification? -- was Re: fscache: Redesigning the on-disk cache

2021-03-08 Thread David Howells
Amir Goldstein wrote: > > But after I've written and sync'd the data, I set the xattr to mark the > > file not open. At the moment I'm doing this too lazily, only doing it > > when a netfs file gets evicted or when the cache gets withdrawn, but I > > really need to add a queue of objects to be

Re: fscache: Redesigning the on-disk cache

2021-03-08 Thread Amir Goldstein
On Mon, Mar 8, 2021 at 11:14 AM David Howells wrote: > > Amir Goldstein wrote: > > > > (0a) As (0) but using SEEK_DATA/SEEK_HOLE instead of bmap and opening the > > > file for every whole operation (which may combine reads and writes). > > > > I read that NFSv4 supports hole punching, so

Re: fscache: Redesigning the on-disk cache

2021-03-08 Thread David Howells
Amir Goldstein wrote: > > (0a) As (0) but using SEEK_DATA/SEEK_HOLE instead of bmap and opening the > > file for every whole operation (which may combine reads and writes). > > I read that NFSv4 supports hole punching, so when using ->bmap() or SEEK_DATA > to keep track of present data,

Re: fscache: Redesigning the on-disk cache

2021-03-05 Thread Amir Goldstein
On Thu, Mar 4, 2021 at 4:10 PM David Howells wrote: > > I'm looking at redesigning the on-disk cache format used by fscache's > cachefiles driver to try and eliminate the number of synchronous metadata > operations done by the driver, to improve culling performance and to reduce > the amount of

fscache: Redesigning the on-disk cache - LRU handling

2021-03-04 Thread David Howells
David Howells wrote: > > (3) OpenAFS-style format. One index file to look up {file_key,block#} and an > array of data files, each holding one block (e.g. a 256KiB-aligned chunk > of a file). Each index entry has valid start/end offsets for easy > truncation. > > The

fscache: Redesigning the on-disk cache

2021-03-03 Thread David Howells
I'm looking at redesigning the on-disk cache format used by fscache's cachefiles driver to try and eliminate the number of synchronous metadata operations done by the driver, to improve culling performance and to reduce the amount of opens/files open. I also need to stop relying on the backing