Re: [Linux-cachefs] [RFC PATCH 02/53] netfs: Track the fpos above which the server has no data

2023-10-16 Thread David Howells
Jeff Layton wrote: > > (7) If stored data is culled from the local cache, we must set zero_point > > above that if the data also got written to the server. > > When you say culled here, it sounds like you're just throwing out the > dirty cache without writing the data back. That shouldn't

Re: [Linux-cachefs] [RFC PATCH 08/53] netfs: Add rsize to netfs_io_request

2023-10-16 Thread David Howells
Jeff Layton wrote: > > + rreq->rsize = 4 * 1024 * 1024; > > return 0; > ... > > + rreq->rsize = 1024 * 1024; > > + > > Holy magic numbers, batman! I think this deserves a comment that > explains how you came up with these values. Actually, that should be set to something like the object

Re: [Linux-cachefs] [RFC PATCH 09/53] netfs: Implement unbuffered/DIO vs buffered I/O locking

2023-10-16 Thread David Howells
Jeff Layton wrote: > It's nice to see this go into common code, but why not go ahead and > convert ceph (and possibly NFS) to use this? Is there any reason not to? I'm converting ceph on a follow-on branch and for ceph this will be dealt with there. I could do NFS round about here, I suppose.

[Linux-cachefs] [RFC PATCH 41/53] netfs: Rearrange netfs_io_subrequest to put request pointer first

2023-10-13 Thread David Howells
Rearrange the netfs_io_subrequest struct to put the netfs_io_request pointer (rreq) first. This then allows netfs_io_subrequest to be put in a union with a pointer to a wrapper around netfs_io_request for cifs. Signed-off-by: David Howells cc: Steve French cc: Shyam Prasad N cc: Rohith

[Linux-cachefs] [RFC PATCH 51/53] cifs: Remove some code that's no longer used, part 1

2023-10-13 Thread David Howells
Remove some code that was #if'd out with the netfslib conversion. This is split into parts for file.c as the diff generator otherwise produces a hard to read diff for part of it where a big chunk is cut out. Signed-off-by: David Howells cc: Steve French cc: Shyam Prasad N cc: R

[Linux-cachefs] [RFC PATCH 53/53] cifs: Remove some code that's no longer used, part 3

2023-10-13 Thread David Howells
Remove some code that was #if'd out with the netfslib conversion. This is split into parts for file.c as the diff generator otherwise produces a hard to read diff for part of it where a big chunk is cut out. Signed-off-by: David Howells cc: Steve French cc: Shyam Prasad N cc: R

[Linux-cachefs] [RFC PATCH 50/53] cifs: Cut over to using netfslib

2023-10-13 Thread David Howells
appen automatically? Does netfs need to do that? Signed-off-by: David Howells cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: Jeff Layton cc: linux-c...@vger.kernel.org cc: linux-cachefs@redhat.com cc: linux-fsde...@vger.kernel.org cc: linux...@kvack.org --- fs/netfs

[Linux-cachefs] [RFC PATCH 52/53] cifs: Remove some code that's no longer used, part 2

2023-10-13 Thread David Howells
Remove some code that was #if'd out with the netfslib conversion. This is split into parts for file.c as the diff generator otherwise produces a hard to read diff for part of it where a big chunk is cut out. Signed-off-by: David Howells cc: Steve French cc: Shyam Prasad N cc: R

[Linux-cachefs] [RFC PATCH 49/53] cifs: Move cifs_loose_read_iter() and cifs_file_write_iter() to file.c

2023-10-13 Thread David Howells
Move cifs_loose_read_iter() and cifs_file_write_iter() to file.c so that they are colocated with similar functions rather than being split with cifsfs.c. Signed-off-by: David Howells cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: Jeff Layton cc: linux-c...@vger.kernel.org cc

[Linux-cachefs] [RFC PATCH 45/53] cifs: Replace cifs_writedata with a wrapper around netfs_io_subrequest

2023-10-13 Thread David Howells
Replace the cifs_writedata struct with the same wrapper around netfs_io_subrequest that was used to replace cifs_readdata. Signed-off-by: David Howells cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: Jeff Layton cc: linux-c...@vger.kernel.org cc: linux-cachefs@redhat.com cc

[Linux-cachefs] [RFC PATCH 48/53] cifs: Implement netfslib hooks

2023-10-13 Thread David Howells
cifs's wsize and available credits. As each subop is created, it can be dispatched or queued for dispatch. At this point, cifs is not wired up to actually *use* netfslib; that will be done in a subsequent patch. Signed-off-by: David Howells cc: Steve French cc: Shyam Prasad N cc: R

[Linux-cachefs] [RFC PATCH 47/53] cifs: Make wait_mtu_credits take size_t args

2023-10-13 Thread David Howells
Make the wait_mtu_credits functions use size_t for the size and num arguments rather than unsigned int as netfslib uses size_t/ssize_t for arguments and return values to allow for extra capacity. Signed-off-by: David Howells cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: Jeff

[Linux-cachefs] [RFC PATCH 44/53] cifs: Share server EOF pos with netfslib

2023-10-13 Thread David Howells
Use cifsi->netfs_ctx.remote_i_size instead of cifsi->server_eof so that netfslib can refer to it to. Signed-off-by: David Howells cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: Jeff Layton cc: linux-c...@vger.kernel.org cc: linux-cachefs@redhat.com cc: linu

[Linux-cachefs] [RFC PATCH 46/53] cifs: Use more fields from netfs_io_subrequest

2023-10-13 Thread David Howells
Use more fields from netfs_io_subrequest instead of those incorporated into cifs_io_subrequest from cifs_readdata and cifs_writedata. Signed-off-by: David Howells cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: Jeff Layton cc: linux-c...@vger.kernel.org cc: linux-cachefs

[Linux-cachefs] [RFC PATCH 42/53] afs: Use the netfs write helpers

2023-10-13 Thread David Howells
Make afs use the netfs write helpers. Signed-off-by: David Howells cc: Marc Dionne cc: Jeff Layton cc: linux-...@lists.infradead.org cc: linux-cachefs@redhat.com cc: linux-fsde...@vger.kernel.org cc: linux...@kvack.org --- fs/afs/file.c | 65 +++- fs/afs/internal.h

[Linux-cachefs] [RFC PATCH 43/53] cifs: Replace cifs_readdata with a wrapper around netfs_io_subrequest

2023-10-13 Thread David Howells
: David Howells cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: Jeff Layton cc: linux-c...@vger.kernel.org cc: linux-cachefs@redhat.com cc: linux-fsde...@vger.kernel.org cc: linux...@kvack.org --- fs/smb/client/cifsglob.h | 22 ++ fs/smb/client/cifsproto.h | 9

[Linux-cachefs] [RFC PATCH 39/53] netfs: Provide a launder_folio implementation

2023-10-13 Thread David Howells
Provide a launder_folio implementation for netfslib. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: linux-fsde...@vger.kernel.org cc: linux...@kvack.org --- fs/netfs/buffered_write.c| 71 fs/netfs/main.c | 1

[Linux-cachefs] [RFC PATCH 37/53] netfs: Support decryption on ubuffered/DIO read

2023-10-13 Thread David Howells
Support unbuffered and direct I/O reads from an encrypted file. This may require making a larger read than is required into a bounce buffer and copying out the required bits. We don't decrypt in-place in the user buffer lest userspace interfere and muck up the decryption. Signed-off-by:

[Linux-cachefs] [RFC PATCH 40/53] netfs: Implement a write-through caching option

2023-10-13 Thread David Howells
re written back in the normal way; similarly failed writethrough writes are marked dirty and left to writeback to retry. Another option would be to simply invalidate them, but the contents can be simultaneously accessed by read() and through mmap. Signed-off-by: David Howells cc: Jeff Layton

[Linux-cachefs] [RFC PATCH 38/53] netfs: Support encryption on Unbuffered/DIO write

2023-10-13 Thread David Howells
Support unbuffered and direct I/O writes to an encrypted file. This may require making an RMW cycle if the write is not appropriately aligned with respect to the crypto blocks. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: linux-fsde...@vger.kernel.org cc: linux

[Linux-cachefs] [RFC PATCH 34/53] netfs: Make netfs_skip_folio_read() take account of blocksize

2023-10-13 Thread David Howells
Make netfs_skip_folio_read() take account of blocksize such as crypto blocksize. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: linux-fsde...@vger.kernel.org cc: linux...@kvack.org --- fs/netfs/buffered_read.c | 32 +--- 1 file changed

[Linux-cachefs] [RFC PATCH 33/53] netfs: Provide minimum blocksize parameter

2023-10-13 Thread David Howells
Add a parameter for minimum blocksize in the netfs_i_context struct. This can be used, for instance, to force I/O alignment for content encryption. It also requires the use of an RMW cycle if a write we want to do doesn't meet the block alignment requirements. Signed-off-by: David Howell

[Linux-cachefs] [RFC PATCH 36/53] netfs: Decrypt encrypted content

2023-10-13 Thread David Howells
block size is equal to or less than the size of a page (in which case we might be better doing it as pages become read), but it will handle crypto blocks larger than the size of a page. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: linux-fsde...@vger.kernel.org

[Linux-cachefs] [RFC PATCH 35/53] netfs: Perform content encryption

2023-10-13 Thread David Howells
also by this same mechanism. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: linux-fsde...@vger.kernel.org cc: linux...@kvack.org --- fs/netfs/Makefile| 1 + fs/netfs/buffered_write.c| 3 +- fs/netfs/crypto.c| 89

[Linux-cachefs] [RFC PATCH 32/53] netfs: Provide a writepages implementation

2023-10-13 Thread David Howells
Provide an implementation of writepages for network filesystems to delegate to. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: linux-fsde...@vger.kernel.org cc: linux...@kvack.org --- fs/netfs/buffered_write.c | 627 ++ include

[Linux-cachefs] [RFC PATCH 31/53] netfs: Provide netfs_file_read_iter()

2023-10-13 Thread David Howells
Provide a top-level-ish function that can be pointed to directly by ->read_iter file op. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: linux-fsde...@vger.kernel.org cc: linux...@kvack.org --- fs/netfs/buffered_read.c |

[Linux-cachefs] [RFC PATCH 30/53] netfs: Allow buffered shared-writeable mmap through netfs_page_mkwrite()

2023-10-13 Thread David Howells
Provide an entry point to delegate a filesystem's ->page_mkwrite() to. This checks for conflicting writes, then attached any netfs-specific group marking (e.g. ceph snap) to the page to be considered dirty. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: li

[Linux-cachefs] [RFC PATCH 29/53] netfs: Implement buffered write API

2023-10-13 Thread David Howells
Institute a netfs write helper, netfs_file_write_iter(), to be pointed at by the network filesystem ->write_iter() call. Make it handled buffered writes by calling the previously defined netfs_perform_write() to copy the source data into the pagecache. Signed-off-by: David Howells cc: J

[Linux-cachefs] [RFC PATCH 28/53] netfs: Implement unbuffered/DIO write support

2023-10-13 Thread David Howells
requirements. Also provide a field that can tell the code to add some extra space onto the bounce buffer for use by the filesystem in the case of a content-encrypted file. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: linux-fsde...@vger.kernel.org cc: linux...@kvack.org

[Linux-cachefs] [RFC PATCH 27/53] netfs: Implement support for unbuffered/DIO read

2023-10-13 Thread David Howells
further subrequests may be copied into the buffer and then abandoned. In the future, this will also take care of doing an unbuffered read from encrypted content, with the decryption being done by the library. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: li

[Linux-cachefs] [RFC PATCH 26/53] netfs: Allocate multipage folios in the writepath

2023-10-13 Thread David Howells
Allocate a multipage folio when copying data into the pagecache if possible if there's sufficient data to warrant it. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: linux-fsde...@vger.kernel.org cc: linux...@kvack.org --- fs/netfs/buffered_write.

[Linux-cachefs] [RFC PATCH 23/53] netfs: Dispatch write requests to process a writeback slice

2023-10-13 Thread David Howells
st() and netfs_put_write_request() are provided to refcount a request. These take constants from the netfs_wreq_trace enum for logging into ftrace. (6) The ->free_write_request is method is called to ask the filesystem to clean up a request. Signed-off-by: David Howells cc: Jeff

[Linux-cachefs] [RFC PATCH 25/53] netfs: Make netfs_read_folio() handle streaming-write pages

2023-10-13 Thread David Howells
over to folio->private. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: linux-fsde...@vger.kernel.org cc: linux...@kvack.org --- fs/netfs/buffered_read.c | 61 ++-- include/trace/events/netfs.h | 2 ++ 2 files changed,

[Linux-cachefs] [RFC PATCH 24/53] netfs: Provide func to copy data to pagecache for buffered write

2023-10-13 Thread David Howells
dirty, but not uptodate. It will also tag pages as belonging to fs-specific write groups if so directed by the filesystem. This is derived from generic_perform_write(), but doesn't use ->write_begin() and ->write_end(), having that logic rolled in instead. Signed-off-by: David Ho

[Linux-cachefs] [RFC PATCH 22/53] netfs: Prep to use folio->private for write grouping and streaming write

2023-10-13 Thread David Howells
in operation on a dirty folio. (2) If private is set, with bit 0 clear, then this points to a group. (3) If private is set, with bit 0 set, then this points to a netfs_folio struct (with bit 0 AND'ed out). Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc

[Linux-cachefs] [RFC PATCH 21/53] netfs: Make the refcounting of netfs_begin_read() easier to use

2023-10-13 Thread David Howells
Make the refcounting of netfs_begin_read() easier to use by not eating the caller's ref on the netfs_io_request it's given. This makes it easier to use when we need to look in the request struct after. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc:

[Linux-cachefs] [RFC PATCH 20/53] fscache: Add a function to begin an cache op from a netfslib request

2023-10-13 Thread David Howells
cause dependency cycles as fscache calls into netfslib directly. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: linux-fsde...@vger.kernel.org cc: linux...@kvack.org --- fs/9p/vfs_addr.c| 18 ++ fs/afs/file.c | 14 +- fs

[Linux-cachefs] [RFC PATCH 18/53] netfs: Add a hook to allow tell the netfs to update its i_size

2023-10-13 Thread David Howells
Add a hook for netfslib's write helpers to call to tell the network filesystem that it should update its i_size. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: linux-fsde...@vger.kernel.org cc: linux...@kvack.org --- include/linux/netfs.h | 4 1 file ch

[Linux-cachefs] [RFC PATCH 19/53] netfs: Make netfs_put_request() handle a NULL pointer

2023-10-13 Thread David Howells
Make netfs_put_request() just return if given a NULL request pointer. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: linux-fsde...@vger.kernel.org cc: linux...@kvack.org --- fs/netfs/objects.c | 23 +-- 1 file changed, 13 insertions(+), 10

[Linux-cachefs] [RFC PATCH 17/53] netfs: Extend the netfs_io_*request structs to handle writes

2023-10-13 Thread David Howells
, decrypted if necessary, the modifications made, then the modified data can be reencrypted/recompressed and sent back to the server. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: linux-fsde...@vger.kernel.org cc: linux...@kvack.org --- fs/netfs/internal.h | 6

[Linux-cachefs] [RFC PATCH 16/53] netfs: Export netfs_put_subrequest() and some tracepoints

2023-10-13 Thread David Howells
Export netfs_put_subrequest() and the netfs_rreq and netfs_sreq tracepoints. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: linux-fsde...@vger.kernel.org cc: linux...@kvack.org --- fs/netfs/main.c| 3 +++ fs/netfs/objects.c | 1 + 2 files changed, 4

[Linux-cachefs] [RFC PATCH 14/53] netfs: Add func to calculate pagecount/size-limited span of an iterator

2023-10-13 Thread David Howells
: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: linux-fsde...@vger.kernel.org cc: linux...@kvack.org --- fs/netfs/iterator.c | 97 +++ include/linux/netfs.h | 2 + 2 files changed, 99 insertions(+) diff --git a/fs/netfs/iterator.c b/fs

[Linux-cachefs] [RFC PATCH 15/53] netfs: Limit subrequest by size or number of segments

2023-10-13 Thread David Howells
Limit a subrequest to a maximum size and/or a maximum number of contiguous physical regions. This permits, for instance, an subreq's iterator to be limited to the number of DMA'able segments that a large RDMA request can handle. Signed-off-by: David Howells cc: Jeff Layton cc: lin

[Linux-cachefs] [RFC PATCH 13/53] netfs: Add bounce buffering support

2023-10-13 Thread David Howells
Add a second xarray struct to netfs_io_request for the purposes of holding a bounce buffer for when we have to deal with encrypted/compressed data or if we have to up/download data in blocks larger than we were asked for. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com

[Linux-cachefs] [RFC PATCH 12/53] netfs: Provide tools to create a buffer in an xarray

2023-10-13 Thread David Howells
Provide tools to create a buffer in an xarray, with a function to add new folios with a mark. This will be used to create bounce buffer and can be used more easily to create a list of folios the span of which would require more than a page's worth of bio_vec structs. Signed-off-by: David Ho

[Linux-cachefs] [RFC PATCH 11/53] netfs: Add support for DIO buffering

2023-10-13 Thread David Howells
it may be immediately deallocated. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: linux-fsde...@vger.kernel.org cc: linux...@kvack.org --- fs/netfs/objects.c| 10 ++ include/linux/netfs.h | 3 +++ 2 files changed, 13 insertions(+) diff --git a/fs

[Linux-cachefs] [RFC PATCH 10/53] netfs: Add iov_iters to (sub)requests to describe various buffers

2023-10-13 Thread David Howells
mpression. These iov_iters persist for the lifetime of the (sub)request, and so can be accessed multiple times without worrying about them being deallocated upon return to the caller. The network filesystem must appropriately advance the iterator before terminating the request. Signed-off-by: Davi

[Linux-cachefs] [RFC PATCH 07/53] netfs: Provide invalidate_folio and release_folio calls

2023-10-13 Thread David Howells
Provide default invalidate_folio and release_folio calls. These will need to interact with invalidation correctly at some point. They will be needed if netfslib is to make use of folio->private for its own purposes. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com

[Linux-cachefs] [RFC PATCH 09/53] netfs: Implement unbuffered/DIO vs buffered I/O locking

2023-10-13 Thread David Howells
Borrow NFS's direct-vs-buffered I/O locking into netfslib. Similar code is also used in ceph. Modify it to have the correct checker annotations for i_rwsem lock acquisition/release and to return -ERESTARTSYS if waits are interrupted. Signed-off-by: David Howells cc: Jeff Layton cc:

[Linux-cachefs] [RFC PATCH 08/53] netfs: Add rsize to netfs_io_request

2023-10-13 Thread David Howells
Add an rsize parameter to netfs_io_request to be filled in by the network filesystem when the request is initialised. This indicates the maximum size of a read request that the netfs will honour in that region. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: linux

[Linux-cachefs] [RFC PATCH 05/53] netfs: Add a ->free_subrequest() op

2023-10-13 Thread David Howells
Add a ->free_subrequest() op so that the netfs can clean up data attached to a subrequest. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: linux-fsde...@vger.kernel.org cc: linux...@kvack.org --- fs/netfs/objects.c| 2 ++ include/linux/netfs.h | 1 + 2 fi

[Linux-cachefs] [RFC PATCH 06/53] afs: Don't use folio->private to record partial modification

2023-10-13 Thread David Howells
dth. Remove this, at least for now, and accept the extra network load (which doesn't matter in the common case of writing a whole file at a time from beginning to end). This makes folio->private available for netfslib to use. Signed-off-by: David Howells cc: Marc Dionne cc: Jeff Layton

[Linux-cachefs] [RFC PATCH 03/53] netfs: Note nonblockingness in the netfs_io_request struct

2023-10-13 Thread David Howells
Allow O_NONBLOCK to be noted in the netfs_io_request struct. Also add a flag, NETFS_RREQ_BLOCKED to record if we did block. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: linux-fsde...@vger.kernel.org cc: linux...@kvack.org --- fs/netfs/objects.c| 2

[Linux-cachefs] [RFC PATCH 04/53] netfs: Allow the netfs to make the io (sub)request alloc larger

2023-10-13 Thread David Howells
Allow the network filesystem to specify extra space to be allocated on the end of the io (sub)request. This allows cifs, for example, to use this space rather than allocating its own cifs_readdata struct. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: linux-fsde

[Linux-cachefs] [RFC PATCH 02/53] netfs: Track the fpos above which the server has no data

2023-10-13 Thread David Howells
int above that. Assuming the above, any read from the server at or above the zero_point position will return all zeroes. The zero_point value can be stored in the cache, provided the above rules are applied to it by any code that culls part of the local cache. Signed-off-by: David Howells c

[Linux-cachefs] [RFC PATCH 01/53] netfs: Add a procfile to list in-progress requests

2023-10-13 Thread David Howells
Add a procfile, /proc/fs/netfs/requests, to list in-progress netfslib I/O requests. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: linux-fsde...@vger.kernel.org cc: linux...@kvack.org --- fs/netfs/internal.h | 22 +++ fs/netfs/main.c | 91

[Linux-cachefs] [RFC PATCH 05/53] netfs: Add a ->free_subrequest() op

2023-10-13 Thread David Howells
Add a ->free_subrequest() op so that the netfs can clean up data attached to a subrequest. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: linux-fsde...@vger.kernel.org cc: linux...@kvack.org --- fs/netfs/objects.c| 2 ++ include/linux/netfs.h | 1 + 2 fi

[Linux-cachefs] [RFC PATCH 04/53] netfs: Allow the netfs to make the io (sub)request alloc larger

2023-10-13 Thread David Howells
Allow the network filesystem to specify extra space to be allocated on the end of the io (sub)request. This allows cifs, for example, to use this space rather than allocating its own cifs_readdata struct. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: linux-fsde

[Linux-cachefs] [RFC PATCH 03/53] netfs: Note nonblockingness in the netfs_io_request struct

2023-10-13 Thread David Howells
Allow O_NONBLOCK to be noted in the netfs_io_request struct. Also add a flag, NETFS_RREQ_BLOCKED to record if we did block. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: linux-fsde...@vger.kernel.org cc: linux...@kvack.org --- fs/netfs/objects.c| 2

[Linux-cachefs] [RFC PATCH 02/53] netfs: Track the fpos above which the server has no data

2023-10-13 Thread David Howells
int above that. Assuming the above, any read from the server at or above the zero_point position will return all zeroes. The zero_point value can be stored in the cache, provided the above rules are applied to it by any code that culls part of the local cache. Signed-off-by: David Howells c

[Linux-cachefs] [RFC PATCH 01/53] netfs: Add a procfile to list in-progress requests

2023-10-13 Thread David Howells
Add a procfile, /proc/fs/netfs/requests, to list in-progress netfslib I/O requests. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: linux-fsde...@vger.kernel.org cc: linux...@kvack.org --- fs/netfs/internal.h | 22 +++ fs/netfs/main.c | 91

[Linux-cachefs] [PATCH v2] netfs: Only call folio_start_fscache() one time for each folio

2023-09-18 Thread David Howells
tfs helpers" Link: https://bugzilla.redhat.com/show_bug.cgi?id=2210612 Signed-off-by: Dave Wysochanski Reviewed-by: Jeff Layton Signed-off-by: David Howells Link: https://lore.kernel.org/r/20230608214137.856006-1-dwyso...@redhat.com/ # v1 Link: https://lore.kernel.org/r/20230915185704

Re: [Linux-cachefs] [PATCH] netfs: Only call folio_start_fscache() one time for each folio

2023-09-15 Thread David Howells
Okay, this looks reasonable. Should I apply Jeff's suggestion before I send it to Linus? David -- Linux-cachefs mailing list Linux-cachefs@redhat.com https://listman.redhat.com/mailman/listinfo/linux-cachefs

Re: [Linux-cachefs] [PATCH -next] fscache: Remove duplicated include

2023-08-14 Thread David Howells
GUO Zihua wrote: > Remove duplicated include for linux/uio.h. Resolves checkincludes > message. > > Signed-off-by: GUO Zihua Acked-by: David Howells -- Linux-cachefs mailing list Linux-cachefs@redhat.com https://listman.redhat.com/mailman/listinfo/linux-cachefs

Re: [Linux-cachefs] [PATCH -next] fscache: Remove duplicated include

2023-08-14 Thread David Howells
GUO Zihua wrote: > Remove duplicated include for linux/uio.h. Resolves checkincludes > message. > > Signed-off-by: GUO Zihua Acked-by: David Howells -- Linux-cachefs mailing list Linux-cachefs@redhat.com https://listman.redhat.com/mailman/listinfo/linux-cachefs

[Linux-cachefs] [PATCH v7 0/2] mm, netfs, fscache: Stop read optimisation when folio removed from pagecache

2023-06-28 Thread David Howells
31368.2334525.8569808925687731937.st...@warthog.procyon.org.uk/ # v5 Link: https://lore.kernel.org/r/20230216150701.3654894-1-dhowe...@redhat.com/ # v6 David Howells (2): mm: Merge folio_has_private()/filemap_release_folio() call pairs mm, netfs, fscache: Stop read optimisation when folio

[Linux-cachefs] [PATCH v7 1/2] mm: Merge folio_has_private()/filemap_release_folio() call pairs

2023-06-28 Thread David Howells
ck in fuse. Reported-by: Rohith Surabattula Suggested-by: Matthew Wilcox Signed-off-by: David Howells cc: Matthew Wilcox cc: Linus Torvalds cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: Dave Wysochanski cc: Dominique Martinet cc: Ilya Dryomov cc: "Theodore Ts'o"

[Linux-cachefs] [PATCH v7 2/2] mm, netfs, fscache: Stop read optimisation when folio removed from pagecache

2023-06-28 Thread David Howells
if a cachefiles object already contains data when we open it. Fixes: 1f67e6d0b188 ("fscache: Provide a function to note the release of a page") Fixes: 047487c947e8 ("cachefiles: Implement the I/O routines") Reported-by: Rohith Surabattula Suggested-by: Matthew Wilcox Signed-off-by: Dav

Re: [Linux-cachefs] [PATCH] cachefiles: allocate static minor for /dev/cachefiles

2023-06-22 Thread David Howells
Marcel Holtmann wrote: > The cachefiles misc character device uses MISC_DYNAMIC_MINOR and thus > doesn't support module auto-loading. Assign a static minor number for it > and provide appropriate module aliases for it. This is enough for kmod to > create the /dev/cachefiles device node on startup

[Linux-cachefs] [PATCH net-next v3 04/10] Move netfs_extract_iter_to_sg() to lib/scatterlist.c

2023-06-06 Thread David Howells
Move netfs_extract_iter_to_sg() to lib/scatterlist.c as it's going to be used by more than just network filesystems (AF_ALG, for example). Signed-off-by: David Howells cc: Jeff Layton cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: Jens Axboe cc: Herbert Xu cc: &qu

[Linux-cachefs] [PATCH net-next v3 02/10] Fix a couple of spelling mistakes

2023-06-06 Thread David Howells
Fix a couple of spelling mistakes in a comment. Suggested-by: Simon Horman Link: https://lore.kernel.org/r/zhh2msrqel4gs...@corigine.com/ Link: https://lore.kernel.org/r/zhh1nqzwogzxl...@corigine.com/ Signed-off-by: David Howells Reviewed-by: Simon Horman cc: Jeff Layton cc: Steve French cc

[Linux-cachefs] [PATCH net-next v3 03/10] Wrap lines at 80

2023-06-06 Thread David Howells
Wrap a line at 80 to stop checkpatch complaining. Signed-off-by: David Howells cc: Jeff Layton cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: Jens Axboe cc: Herbert Xu cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Matthew

[Linux-cachefs] [PATCH net-next v3 01/10] Drop the netfs_ prefix from netfs_extract_iter_to_sg()

2023-06-06 Thread David Howells
Rename netfs_extract_iter_to_sg() and its auxiliary functions to drop the netfs_ prefix. Signed-off-by: David Howells cc: Jeff Layton cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: Jens Axboe cc: Herbert Xu cc: "Matthew Wilcox (Oracle)" cc: "David S. Mi

[Linux-cachefs] [PATCH net-next v2 04/10] Move netfs_extract_iter_to_sg() to lib/scatterlist.c

2023-05-30 Thread David Howells
Move netfs_extract_iter_to_sg() to lib/scatterlist.c as it's going to be used by more than just network filesystems (AF_ALG, for example). Signed-off-by: David Howells cc: Jeff Layton cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: Jens Axboe cc: Herbert Xu cc: &qu

[Linux-cachefs] [PATCH net-next v2 03/10] Wrap lines at 80

2023-05-30 Thread David Howells
Wrap a line at 80 to stop checkpatch complaining. Signed-off-by: David Howells cc: Jeff Layton cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: Jens Axboe cc: Herbert Xu cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Matthew

[Linux-cachefs] [PATCH net-next v2 02/10] Fix a couple of spelling mistakes

2023-05-30 Thread David Howells
Fix a couple of spelling mistakes in a comment. Suggested-by: Simon Horman Link: https://lore.kernel.org/r/zhh2msrqel4gs...@corigine.com/ Link: https://lore.kernel.org/r/zhh1nqzwogzxl...@corigine.com/ Signed-off-by: David Howells cc: Jeff Layton cc: Steve French cc: Shyam Prasad N cc: Rohith

[Linux-cachefs] [PATCH net-next v2 01/10] Drop the netfs_ prefix from netfs_extract_iter_to_sg()

2023-05-30 Thread David Howells
Rename netfs_extract_iter_to_sg() and its auxiliary functions to drop the netfs_ prefix. Signed-off-by: David Howells cc: Jeff Layton cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: Jens Axboe cc: Herbert Xu cc: "Matthew Wilcox (Oracle)" cc: "David S. Mi

Re: [Linux-cachefs] [PATCH net-next 1/8] Move netfs_extract_iter_to_sg() to lib/scatterlist.c

2023-05-28 Thread David Howells
If it comes to a respin, I'll stick in an extra patch to fix the spellings - and if not, I'll submit the patch separately. It shouldn't be changed in with the movement of code to give git analysis a better chance of tracking the movement. David -- Linux-cachefs mailing list Linux-cachefs@redhat.c

[Linux-cachefs] [PATCH net-next 2/8] Drop the netfs_ prefix from netfs_extract_iter_to_sg()

2023-05-26 Thread David Howells
Rename netfs_extract_iter_to_sg() and its auxiliary functions to drop the netfs_ prefix. Signed-off-by: David Howells cc: Jeff Layton cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: Jens Axboe cc: Herbert Xu cc: "David S. Miller" cc: linux-cry...@vger.kernel.org

[Linux-cachefs] [PATCH net-next 1/8] Move netfs_extract_iter_to_sg() to lib/scatterlist.c

2023-05-26 Thread David Howells
Move netfs_extract_iter_to_sg() to lib/scatterlist.c as it's going to be used by more than just network filesystems (AF_ALG, for example). Signed-off-by: David Howells cc: Jeff Layton cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: Jens Axboe cc: Herbert Xu cc: &qu

[Linux-cachefs] [PATCH] cachefilesd: Allow the daemon to run as a non-root user

2023-05-19 Thread David Howells
. This also requires the SELinux policy to be changed so that cachefilesd can access /etc/passwd, otherwise only numeric uids and gids can be set. Signed-off-by: David Howells --- cachefilesd.c | 59 +++-- cachefilesd.conf.5 |7 ++ 2

[Linux-cachefs] [PATCH] cachefilesd: Remove pointer poisoning code as it is likely to fail under ASLR

2023-05-19 Thread David Howells
and, in particular, the poison checking which will cause the process to abort with no message as to why. Signed-off-by: David Howells --- cachefilesd.c | 25 + 1 file changed, 9 insertions(+), 16 deletions(-) diff --git a/cachefilesd.c b/cachefilesd.c index d4d236f

[Linux-cachefs] [PATCH] cachefiles: Allow the cache to be non-root

2023-05-19 Thread David Howells
hown'd to those IDs. Signed-off-by: David Howells cc: David Howells cc: Jeff Layton cc: linux-cachefs@redhat.com cc: linux-er...@lists.ozlabs.org cc: linux-fsde...@vger.kernel.org --- fs/cachefiles/namei.c |3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/cac

Re: [Linux-cachefs] [BUG] fscache writing but not reading

2023-05-19 Thread David Howells
Chris Chilvers wrote: > While testing the fscache performance fixes [1] that were merged into 6.4-rc1 > it appears that the caching no longer works. The client will write to the > cache > but never reads. Can you try reading from afs? You would need to enable CONFIG_AFS_FS in your kernel if it

Re: [Linux-cachefs] [PATCH V5 5/5] cachefiles: add restore command to recover inflight ondemand read requests

2023-04-14 Thread David Howells
Jia Zhu wrote: > +int cachefiles_ondemand_restore(struct cachefiles_cache *cache, char *args) > +{ > + struct cachefiles_req *req; > + > + XA_STATE(xas, &cache->reqs, 0); > + > + if (!test_bit(CACHEFILES_ONDEMAND_MODE, &cache->flags)) > + return -EOPNOTSUPP; > + > + /*

Re: [Linux-cachefs] [PATCH V5 2/5] cachefiles: extract ondemand info field from cachefiles_object

2023-04-14 Thread David Howells
Jia Zhu wrote: > #define CACHEFILES_OBJECT_STATE_FUNCS(_state, _STATE)\ > static inline bool > \ > cachefiles_ondemand_object_is_##_state(const struct cachefiles_object > *object) \ > {

Re: [Linux-cachefs] [PATCH V5 4/5] cachefiles: narrow the scope of triggering EPOLLIN events in ondemand mode

2023-04-14 Thread David Howells
Jia Zhu wrote: > if (cachefiles_in_ondemand_mode(cache)) { > - if (!xa_empty(&cache->reqs)) > - mask |= EPOLLIN; > + if (!xa_empty(xa)) { > + rcu_read_lock(); > + xa_for_each_marked(xa, index, req, CACHEFILE

[Linux-cachefs] [PATCH] netfs: Fix netfs_extract_iter_to_sg() for ITER_UBUF/IOVEC

2023-04-12 Thread David Howells
a scatterlist") Signed-off-by: David Howells Reviewed-by: Jeff Layton cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: linux-cachefs@redhat.com cc: linux-c...@vger.kernel.org cc: linux-fsde...@vger.kernel.org --- fs/netfs/iterator.c |2 +- 1 file changed, 1 insertion(+),

[Linux-cachefs] [PATCH v3 01/55] netfs: Fix netfs_extract_iter_to_sg() for ITER_UBUF/IOVEC

2023-03-31 Thread David Howells
acked iterators. Fixes: 018584697533 ("netfs: Add a function to extract an iterator into a scatterlist") Signed-off-by: David Howells cc: Jeff Layton cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: linux-cachefs@redhat.com cc: linux-c...@vger.kernel.org

[Linux-cachefs] [RFC PATCH v2 01/48] netfs: Fix netfs_extract_iter_to_sg() for ITER_UBUF/IOVEC

2023-03-29 Thread David Howells
acked iterators. Fixes: 018584697533 ("netfs: Add a function to extract an iterator into a scatterlist") Signed-off-by: David Howells cc: Jeff Layton cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: linux-cachefs@redhat.com cc: linux-c...@vger.kernel.org

Re: [Linux-cachefs] [PATCH V4 4/5] cachefiles: narrow the scope of triggering EPOLLIN events in ondemand mode

2023-03-28 Thread David Howells
Jia Zhu wrote: > + if (!xa_empty(xa)) { > + xa_lock(xa); > + xa_for_each_marked(xa, index, req, CACHEFILES_REQ_NEW) { > + if > (!cachefiles_ondemand_is_reopening_read(req)) { > + m

Re: [Linux-cachefs] [PATCH V4 3/5] cachefiles: resend an open request if the read request's object is closed

2023-03-28 Thread David Howells
Jia Zhu wrote: > + struct cachefiles_object *object = > + ((struct cachefiles_ondemand_info *)work)->object; container_of(). > + continue; > + } else if (cachefiles_ondemand_object_is_reopening(object)) { The "else" is unnecessary. > +static voi

Re: [Linux-cachefs] [PATCH V4 2/5] cachefiles: extract ondemand info field from cachefiles_object

2023-03-28 Thread David Howells
Jia Zhu wrote: > @@ -65,10 +71,7 @@ struct cachefiles_object { > enum cachefiles_content content_info:8; /* Info about content > presence */ > unsigned long flags; > #define CACHEFILES_OBJECT_USING_TMPFILE 0 /* Have an > unlinked tmpfile

Re: [Linux-cachefs] [PATCH V4 1/5] cachefiles: introduce object ondemand state

2023-03-28 Thread David Howells
Jia Zhu wrote: > +enum cachefiles_object_state { > + CACHEFILES_ONDEMAND_OBJSTATE_close, /* Anonymous fd closed by daemon or > initial state */ > + CACHEFILES_ONDEMAND_OBJSTATE_open, /* Anonymous fd associated with > object is available */ That looks weird. Maybe make them all-lowerca

[Linux-cachefs] [PATCH 2/2] iov: Fix netfs_extract_user_to_sg()

2023-02-27 Thread David Howells
t;netfs: Add a function to extract an iterator into a scatterlist") Reported-by: kernel test robot Reported-by: Dan Carpenter Signed-off-by: David Howells cc: Steve French cc: Jeff Layton cc: linux-c...@vger.kernel.org cc: linux-cachefs@redhat.com Link: https://lore.kernel.org/oe-k

[Linux-cachefs] [PATCH 09/17] netfs: Add a function to extract an iterator into a scatterlist

2023-02-16 Thread David Howells
manage their lifetime. It cannot be assumed that a ref can be validly taken, particularly in the case of a KVEC iterator. Signed-off-by: David Howells cc: Jeff Layton cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: linux-cachefs@redhat.com cc: linux-c...@vger.kernel.org cc

[Linux-cachefs] [PATCH 08/17] netfs: Add a function to extract a UBUF or IOVEC into a BVEC iterator

2023-02-16 Thread David Howells
tream that's then tricky to recover from. We don't deal with other types of iterator here, leaving it to other mechanisms to retain the pages (eg. PG_locked, PG_writeback and the pipe lock). Signed-off-by: David Howells cc: Jeff Layton cc: Steve French cc: Shyam Prasad N cc: Roh

[Linux-cachefs] [PATCH v6 2/2] mm, netfs, fscache: Stop read optimisation when folio removed from pagecache

2023-02-16 Thread David Howells
achefiles object already contains data when we open it. Reported-by: Rohith Surabattula Suggested-by: Matthew Wilcox Signed-off-by: David Howells cc: Matthew Wilcox cc: Linus Torvalds cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: Dave Wysochanski cc: Dominique

[Linux-cachefs] [PATCH v6 1/2] mm: Merge folio_has_private()/filemap_release_folio() call pairs

2023-02-16 Thread David Howells
ck in fuse. Reported-by: Rohith Surabattula Suggested-by: Matthew Wilcox Signed-off-by: David Howells cc: Matthew Wilcox cc: Linus Torvalds cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: Dave Wysochanski cc: Dominique Martinet cc: Ilya Dryomov cc: "Theodore Ts'o"

[Linux-cachefs] [PATCH v6 0/2] mm, netfs, fscache: Stop read optimisation when folio removed from pagecache

2023-02-16 Thread David Howells
org.uk/ # v4 Link: https://lore.kernel.org/r/167172131368.2334525.8569808925687731937.st...@warthog.procyon.org.uk/ # v5 --- %(shortlog)s %(diffstat)s David Howells (2): mm: Merge folio_has_private()/filemap_release_folio() call pairs mm, netfs, fscache: Stop read optimisation when folio removed from pagecache fs/9

  1   2   3   4   5   6   7   8   9   10   >