[PATCH 19/29] ocfs2: truncate page cache for clone destination file before remapping

2018-10-17 Thread Darrick J. Wong
From: Darrick J. Wong When cloning blocks into another file, truncate the page cache before we start remapping blocks so that concurrent reads wait for us to finish. Signed-off-by: Darrick J. Wong --- fs/ocfs2/refcounttree.c | 10 -- 1 file changed, 4 insertions(+), 6 deletions

[PATCH 20/29] ocfs2: fix pagecache truncation prior to reflink

2018-10-17 Thread Darrick J. Wong
From: Darrick J. Wong Prior to remapping blocks, it is necessary to remove pages from the destination file's page cache. Unfortunately, the truncation is not aggressive enough -- if page size > block size, we'll end up zeroing subpage blocks instead of removing them. So, round the

[PATCH 13/29] vfs: make remap_file_range functions take and return bytes completed

2018-10-17 Thread Darrick J. Wong
From: Darrick J. Wong Change the remap_file_range functions to take a number of bytes to operate upon and return the number of bytes they operated on. This is a requirement for allowing fs implementations to return short clone/dedupe results to the user, which will enable us to obey resource

[PATCH 12/29] vfs: remap helper should update destination inode metadata

2018-10-17 Thread Darrick J. Wong
From: Darrick J. Wong Extend generic_remap_file_range_prep to handle inode metadata updates when remapping into a file. If the operation can possibly alter the file contents, we must update the ctime and mtime and remove security privileges, just like we do for regular file writes. Signed-off

[PATCH 15/29] vfs: plumb remap flags through the vfs dedupe functions

2018-10-17 Thread Darrick J. Wong
From: Darrick J. Wong Plumb a remap_flags argument through the vfs_dedupe_file_range_one functions so that dedupe can take advantage of it. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein --- fs/overlayfs/file.c |3 ++- fs/read_write.c |9 ++--- include/linux/fs.h

[PATCH 14/29] vfs: plumb remap flags through the vfs clone functions

2018-10-17 Thread Darrick J. Wong
From: Darrick J. Wong Plumb a remap_flags argument through the {do,vfs}_clone_file_range functions so that clone can take advantage of it. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein --- fs/ioctl.c |2 +- fs/nfsd/vfs.c |2 +- fs/overlayfs/copy_up.c

[PATCH 11/29] vfs: pass remap flags to generic_remap_checks

2018-10-17 Thread Darrick J. Wong
From: Darrick J. Wong Pass the same remap flags to generic_remap_checks for consistency. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein Reviewed-by: Christoph Hellwig --- fs/read_write.c|2 +- include/linux/fs.h |2 +- mm/filemap.c |4 ++-- 3 files changed

[PATCH 10/29] vfs: pass remap flags to generic_remap_file_range_prep

2018-10-17 Thread Darrick J. Wong
From: Darrick J. Wong Plumb the remap flags through the filesystem from the vfs function dispatcher all the way to the prep function to prepare for behavior changes in subsequent patches. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein Reviewed-by: Christoph Hellwig --- fs/ocfs2

[PATCH 09/29] vfs: combine the clone and dedupe into a single remap_file_range

2018-10-17 Thread Darrick J. Wong
From: Darrick J. Wong Combine the clone_file_range and dedupe_file_range operations into a single remap_file_range file operation dispatch since they're fundamentally the same operation. The differences between the two can be made in the prep functions. Signed-off-by: Darrick J. Wong Rev

[PATCH 06/29] vfs: skip zero-length dedupe requests

2018-10-17 Thread Darrick J. Wong
From: Darrick J. Wong Don't bother calling the filesystem for a zero-length dedupe request; we can return zero and exit. Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig Reviewed-by: Amir Goldstein --- fs/read_write.c |5 + 1 file changed, 5 insertions(+) diff --

[PATCH 05/29] vfs: avoid problematic remapping requests into partial EOF block

2018-10-17 Thread Darrick J. Wong
From: Darrick J. Wong A deduplication data corruption is exposed in XFS and btrfs. It is caused by extending the block match range to include the partial EOF block, but then allowing unknown data beyond EOF to be considered a "match" to data in the destination file because the comparis

[PATCH 07/29] vfs: rename vfs_clone_file_prep to be more descriptive

2018-10-17 Thread Darrick J. Wong
From: Darrick J. Wong The vfs_clone_file_prep is a generic function to be called by filesystem implementations only. Rename the prefix to generic_ and make it more clear that it applies to remap operations, not just clones. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein --- fs

[PATCH 08/29] vfs: rename clone_verify_area to remap_verify_area

2018-10-17 Thread Darrick J. Wong
From: Darrick J. Wong Since we use clone_verify_area for both clone and dedupe range checks, rename the function to make it clear that it's for both. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein --- fs/read_write.c | 10 +- 1 file changed, 5 insertions(+), 5 dele

[PATCH 04/29] vfs: strengthen checking of file range inputs to generic_remap_checks

2018-10-17 Thread Darrick J. Wong
From: Darrick J. Wong File range remapping, if allowed to run past the destination file's EOF, is an optimization on a regular file write. Regular file writes that extend the file length are subject to various constraints which are not checked by range cloning. This is a correctness pr

[PATCH 03/29] vfs: exit early from zero length remap operations

2018-10-17 Thread Darrick J. Wong
From: Darrick J. Wong If a remap caller asks us to remap to the source file's EOF and the source file length leaves us with a zero byte request, exit early. Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig --- fs/read_write.c |2 ++ 1 file changed, 2 insertions(+)

[PATCH 02/29] vfs: check file ranges before cloning files

2018-10-17 Thread Darrick J. Wong
From: Darrick J. Wong Move the file range checks from vfs_clone_file_prep into a separate generic_remap_checks function so that all the checks are collected in a central location. This forms the basis for adding more checks from generic_write_checks that will make cloning's input checking

[PATCH v6 00/29] fs: fixes for serious clone/dedupe problems

2018-10-17 Thread Darrick J. Wong
Hi all, Dave, Eric, and I have been chasing a stale data exposure bug in the XFS reflink implementation, and tracked it down to reflink forgetting to do some of the file-extending activities that must happen for regular writes. We then started auditing the clone, dedupe, and copyfile code and rea

[PATCH 01/29] vfs: vfs_clone_file_prep_inodes should return EINVAL for a clone from beyond EOF

2018-10-17 Thread Darrick J. Wong
From: Darrick J. Wong vfs_clone_file_prep_inodes cannot return 0 if it is asked to remap from a zero byte file because that's what btrfs does. Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig --- fs/read_write.c |3 --- 1 file changed, 3 deletions(-) diff --git

Re: [PATCH 25/26] xfs: support returning partial reflink results

2018-10-17 Thread Darrick J. Wong
On Wed, Oct 17, 2018 at 01:40:02AM -0700, Christoph Hellwig wrote: > > @@ -1415,11 +1419,17 @@ xfs_reflink_remap_range( > > > > trace_xfs_reflink_remap_range(src, pos_in, len, dest, pos_out); > > > > + if (len == 0) { > > + ret = 0; > > + goto out_unlock; > > + } > >

Re: [PATCH 17/26] vfs: enable remap callers that can handle short operations

2018-10-17 Thread Darrick J. Wong
On Wed, Oct 17, 2018 at 01:36:52AM -0700, Christoph Hellwig wrote: > > /* Update inode timestamps and remove security privileges when remapping. > > */ > > @@ -2023,7 +2034,8 @@ loff_t vfs_dedupe_file_range_one(struct file > > *src_file, loff_t src_pos, > > { > > loff_t ret; > > > > - W

[PATCH 25/26] xfs: support returning partial reflink results

2018-10-15 Thread Darrick J. Wong
From: Darrick J. Wong Back when the XFS reflink code only supported clone_file_range, we were only able to return zero or negative error codes to userspace. However, now that copy_file_range (which returns bytes copied) can use XFS' clone_file_range, we have the opportunity to return pa

[PATCH 24/26] xfs: fix pagecache truncation prior to reflink

2018-10-15 Thread Darrick J. Wong
From: Darrick J. Wong Prior to remapping blocks, it is necessary to remove pages from the destination file's page cache. Unfortunately, the truncation is not aggressive enough -- if page size > block size, we'll end up zeroing subpage blocks instead of removing them. So, round the

[PATCH 26/26] xfs: remove redundant remap partial EOF block checks

2018-10-15 Thread Darrick J. Wong
From: Darrick J. Wong Now that we've moved the partial EOF block checks to the VFS helpers, we can remove the redundant functionality from XFS. Signed-off-by: Darrick J. Wong Reviewed-by: Dave Chinner --- fs/xfs/xfs_reflink.c | 19 --- 1 file changed, 19 deletions(-)

[PATCH 23/26] ocfs2: remove ocfs2_reflink_remap_range

2018-10-15 Thread Darrick J. Wong
From: Darrick J. Wong Since ocfs2_remap_file_range is a thin shell around ocfs2_remap_remap_range, move everything from the latter into the former. Signed-off-by: Darrick J. Wong --- fs/ocfs2/file.c | 68 +++- fs/ocfs2/refcounttree.c | 113

[PATCH 22/26] ocfs2: support partial clone range and dedupe range

2018-10-15 Thread Darrick J. Wong
From: Darrick J. Wong Change the ocfs2 remap code to allow for returning partial results. Signed-off-by: Darrick J. Wong --- fs/ocfs2/file.c |7 + fs/ocfs2/refcounttree.c | 72 +-- fs/ocfs2/refcounttree.h | 12 3 files

[PATCH 19/26] vfs: clean up generic_remap_file_range_prep return value

2018-10-15 Thread Darrick J. Wong
From: Darrick J. Wong Since the remap prep function can update the length of the remap request, we can change this function to return the usual return status instead of the odd behavior it has now. Signed-off-by: Darrick J. Wong --- fs/ocfs2/refcounttree.c |2 +- fs/read_write.c

[PATCH 15/26] vfs: plumb remap flags through the vfs clone functions

2018-10-15 Thread Darrick J. Wong
From: Darrick J. Wong Plumb a remap_flags argument through the {do,vfs}_clone_file_range functions so that clone can take advantage of it. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein --- fs/ioctl.c |2 +- fs/nfsd/vfs.c |2 +- fs/overlayfs/copy_up.c

[PATCH 21/26] ocfs2: fix pagecache truncation prior to reflink

2018-10-15 Thread Darrick J. Wong
From: Darrick J. Wong Prior to remapping blocks, it is necessary to remove pages from the destination file's page cache. Unfortunately, the truncation is not aggressive enough -- if page size > block size, we'll end up zeroing subpage blocks instead of removing them. So, round the

[PATCH 16/26] vfs: plumb remap flags through the vfs dedupe functions

2018-10-15 Thread Darrick J. Wong
From: Darrick J. Wong Plumb a remap_flags argument through the vfs_dedupe_file_range_one functions so that dedupe can take advantage of it. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein --- fs/overlayfs/file.c |3 ++- fs/read_write.c |9 ++--- include/linux/fs.h

[PATCH 17/26] vfs: enable remap callers that can handle short operations

2018-10-15 Thread Darrick J. Wong
From: Darrick J. Wong Plumb in a remap flag that enables the filesystem remap handler to shorten remapping requests for callers that can handle it. Now copy_file_range can report partial success (in case we run up against alignment problems, resource limits, etc.). We also enable CAN_SHORTEN

[PATCH 20/26] ocfs2: truncate page cache for clone destination file before remapping

2018-10-15 Thread Darrick J. Wong
From: Darrick J. Wong When cloning blocks into another file, truncate the page cache before we start remapping blocks so that concurrent reads wait for us to finish. Signed-off-by: Darrick J. Wong --- fs/ocfs2/refcounttree.c | 10 -- 1 file changed, 4 insertions(+), 6 deletions

[PATCH 18/26] vfs: hide file range comparison function

2018-10-15 Thread Darrick J. Wong
From: Darrick J. Wong There are no callers of vfs_dedupe_file_range_compare, so we might as well make it a static helper and remove the export. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein Reviewed-by: Christoph Hellwig --- fs/read_write.c| 187

[PATCH 14/26] vfs: make remap_file_range functions take and return bytes completed

2018-10-15 Thread Darrick J. Wong
From: Darrick J. Wong Change the remap_file_range functions to take a number of bytes to operate upon and return the number of bytes they operated on. This is a requirement for allowing fs implementations to return short clone/dedupe results to the user, which will enable us to obey resource

[PATCH 13/26] vfs: create generic_remap_file_range_touch to update inode metadata

2018-10-15 Thread Darrick J. Wong
From: Darrick J. Wong Create a new VFS helper to handle inode metadata updates when remapping into a file. If the operation can possibly alter the file contents, we must update the ctime and mtime and remove security privileges, just like we do for regular file writes. Wire up ocfs2 to ensure

[PATCH 09/26] vfs: rename clone_verify_area to remap_verify_area

2018-10-15 Thread Darrick J. Wong
From: Darrick J. Wong Since we use clone_verify_area for both clone and dedupe range checks, rename the function to make it clear that it's for both. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein --- fs/read_write.c | 10 +- 1 file changed, 5 insertions(+), 5 dele

[PATCH 07/26] vfs: skip zero-length dedupe requests

2018-10-15 Thread Darrick J. Wong
From: Darrick J. Wong Don't bother calling the filesystem for a zero-length dedupe request; we can return zero and exit. Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig Reviewed-by: Amir Goldstein --- fs/read_write.c |5 + 1 file changed, 5 insertions(+) diff --

[PATCH 11/26] vfs: pass remap flags to generic_remap_file_range_prep

2018-10-15 Thread Darrick J. Wong
From: Darrick J. Wong Plumb the remap flags through the filesystem from the vfs function dispatcher all the way to the prep function to prepare for behavior changes in subsequent patches. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein --- fs/ocfs2/file.c |2 +- fs

[PATCH 10/26] vfs: combine the clone and dedupe into a single remap_file_range

2018-10-15 Thread Darrick J. Wong
From: Darrick J. Wong Combine the clone_file_range and dedupe_file_range operations into a single remap_file_range file operation dispatch since they're fundamentally the same operation. The differences between the two can be made in the prep functions. Signed-off-by: Darrick J. Wong Rev

[PATCH 08/26] vfs: rename vfs_clone_file_prep to be more descriptive

2018-10-15 Thread Darrick J. Wong
From: Darrick J. Wong The vfs_clone_file_prep is a generic function to be called by filesystem implementations only. Rename the prefix to generic_ and make it more clear that it applies to remap operations, not just clones. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein --- fs

[PATCH 06/26] vfs: avoid problematic remapping requests into partial EOF block

2018-10-15 Thread Darrick J. Wong
From: Darrick J. Wong A deduplication data corruption is exposed in XFS and btrfs. It is caused by extending the block match range to include the partial EOF block, but then allowing unknown data beyond EOF to be considered a "match" to data in the destination file because the comparis

[PATCH 05/26] vfs: strengthen checking of file range inputs to generic_remap_checks

2018-10-15 Thread Darrick J. Wong
From: Darrick J. Wong File range remapping, if allowed to run past the destination file's EOF, is an optimization on a regular file write. Regular file writes that extend the file length are subject to various constraints which are not checked by range cloning. This is a correctness pr

[PATCH 12/26] vfs: pass remap flags to generic_remap_checks

2018-10-15 Thread Darrick J. Wong
From: Darrick J. Wong Pass the same remap flags to generic_remap_checks for consistency. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein --- fs/read_write.c|2 +- include/linux/fs.h |2 +- mm/filemap.c |4 ++-- 3 files changed, 4 insertions(+), 4 deletions

[PATCH 03/26] vfs: check file ranges before cloning files

2018-10-15 Thread Darrick J. Wong
From: Darrick J. Wong Move the file range checks from vfs_clone_file_prep into a separate generic_remap_checks function so that all the checks are collected in a central location. This forms the basis for adding more checks from generic_write_checks that will make cloning's input checking

[PATCH 01/26] xfs: add a per-xfs trace_printk macro

2018-10-15 Thread Darrick J. Wong
From: Darrick J. Wong Add a "xfs_tprintk" macro so that developers can use trace_printk to print out arbitrary debugging information with the XFS device name attached to the trace output. Signed-off-by: Darrick J. Wong --- fs/xfs/xfs_error.h |6 ++ 1 file changed, 6

[PATCH 04/26] vfs: exit early from zero length remap operations

2018-10-15 Thread Darrick J. Wong
From: Darrick J. Wong If a remap caller asks us to remap to the source file's EOF and the source file has zero bytes, exit early. Signed-off-by: Darrick J. Wong --- fs/read_write.c |2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/read_write.c b/fs/read_write.c index d6e8e24

[PATCH v5 00/26] fs: fixes for serious clone/dedupe problems

2018-10-15 Thread Darrick J. Wong
Hi all, Dave, Eric, and I have been chasing a stale data exposure bug in the XFS reflink implementation, and tracked it down to reflink forgetting to do some of the file-extending activities that must happen for regular writes. We then started auditing the clone, dedupe, and copyfile code and rea

[PATCH 02/26] vfs: vfs_clone_file_prep_inodes should return EINVAL for a clone from beyond EOF

2018-10-15 Thread Darrick J. Wong
From: Darrick J. Wong vfs_clone_file_prep_inodes cannot return 0 if it is asked to remap from a zero byte file because that's what btrfs does. Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig --- fs/read_write.c |3 --- 1 file changed, 3 deletions(-) diff --git

Re: [PATCH 07/25] vfs: combine the clone and dedupe into a single remap_file_range

2018-10-15 Thread Darrick J. Wong
On Mon, Oct 15, 2018 at 05:47:19AM -0700, Christoph Hellwig wrote: > On Mon, Oct 15, 2018 at 09:04:13AM +0300, Amir Goldstein wrote: > > I supposed you figured out the reason already. > > No, I hadn't. > > > It makes it appearance in patch 16/25 as RFR_VFS_FLAGS. > > All those "advisory" flags, w

Re: [PATCH 07/25] vfs: combine the clone and dedupe into a single remap_file_range

2018-10-15 Thread Darrick J. Wong
On Sun, Oct 14, 2018 at 10:19:27AM -0700, Christoph Hellwig wrote: > > unsigned (*mmap_capabilities)(struct file *); > > #endif > > ssize_t (*copy_file_range)(struct file *, loff_t, struct file *, > > loff_t, size_t, unsigned int); > > - int (*clone_file_range)(struct file *, loff_t, st

Re: [PATCH 10/25] vfs: create generic_remap_file_range_touch to update inode metadata

2018-10-15 Thread Darrick J. Wong
On Sun, Oct 14, 2018 at 10:21:31AM -0700, Christoph Hellwig wrote: > > +/* Update inode timestamps and remove security privileges when remapping. > > */ > > +int generic_remap_file_range_touch(struct file *file, bool is_dedupe) > > +{ > > + int ret; > > + > > + /* If can't alter the file conte

Re: [PATCH 24/25] xfs: support returning partial reflink results

2018-10-15 Thread Darrick J. Wong
On Mon, Oct 15, 2018 at 10:05:36AM +1100, Dave Chinner wrote: > On Sun, Oct 14, 2018 at 10:35:46AM -0700, Christoph Hellwig wrote: > > On Fri, Oct 12, 2018 at 05:08:32PM -0700, Darrick J. Wong wrote: > > > From: Darrick J. Wong > > > > > > Back when

Re: [PATCH 11/25] vfs: pass remap flags to generic_remap_file_range_prep

2018-10-15 Thread Darrick J. Wong
On Sun, Oct 14, 2018 at 10:37:38AM -0700, Christoph Hellwig wrote: > > + bool is_dedupe = (remap_flags & RFR_SAME_DATA); > > Btw, I think the code would be cleaner if we dropped this variable. Ok to both. I'll move up the patch to replace is_dedupe with remap_flags to avoid churning the _touch

Re: [PATCH 16/25] vfs: make remapping to source file eof more explicit

2018-10-15 Thread Darrick J. Wong
On Sun, Oct 14, 2018 at 10:24:33AM -0700, Christoph Hellwig wrote: > On Fri, Oct 12, 2018 at 05:07:37PM -0700, Darrick J. Wong wrote: > > From: Darrick J. Wong > > > > Create a RFR_TO_SRC_EOF flag to explicitly declare that the caller wants > > the remap implementation

[PATCH 23/25] xfs: fix pagecache truncation prior to reflink

2018-10-12 Thread Darrick J. Wong
From: Darrick J. Wong Prior to remapping blocks, it is necessary to remove pages from the destination file's page cache. Unfortunately, the truncation is not aggressive enough -- if page size > block size, we'll end up zeroing subpage blocks instead of removing them. So, round the

[PATCH 22/25] ocfs2: support partial clone range and dedupe range

2018-10-12 Thread Darrick J. Wong
From: Darrick J. Wong Change the ocfs2 remap code to allow for returning partial results. Signed-off-by: Darrick J. Wong --- fs/ocfs2/file.c |7 + fs/ocfs2/refcounttree.c | 73 ++- fs/ocfs2/refcounttree.h | 12 3 files

[PATCH 24/25] xfs: support returning partial reflink results

2018-10-12 Thread Darrick J. Wong
From: Darrick J. Wong Back when the XFS reflink code only supported clone_file_range, we were only able to return zero or negative error codes to userspace. However, now that copy_file_range (which returns bytes copied) can use XFS' clone_file_range, we have the opportunity to return pa

[PATCH 25/25] xfs: remove redundant remap partial EOF block checks

2018-10-12 Thread Darrick J. Wong
From: Darrick J. Wong Now that we've moved the partial EOF block checks to the VFS helpers, we can remove the redundantn functionality from XFS. Signed-off-by: Darrick J. Wong Reviewed-by: Dave Chinner --- fs/xfs/xfs_reflink.c | 20 1 file changed, 20 dele

[PATCH 21/25] ocfs2: fix pagecache truncation prior to reflink

2018-10-12 Thread Darrick J. Wong
From: Darrick J. Wong Prior to remapping blocks, it is necessary to remove pages from the destination file's page cache. Unfortunately, the truncation is not aggressive enough -- if page size > block size, we'll end up zeroing subpage blocks instead of removing them. So, round the

[PATCH 16/25] vfs: make remapping to source file eof more explicit

2018-10-12 Thread Darrick J. Wong
From: Darrick J. Wong Create a RFR_TO_SRC_EOF flag to explicitly declare that the caller wants the remap implementation to remap to the end of the source file, once the files are locked. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein --- fs/ioctl.c |3 ++- fs/nfsd

[PATCH 19/25] vfs: implement opportunistic short dedupe

2018-10-12 Thread Darrick J. Wong
From: Darrick J. Wong For a given dedupe request, the bytes_deduped field in the control structure tells userspace if we managed to deduplicate some, but not all of, the requested regions starting from the file offsets supplied. However, due to sloppy coding, the current dedupe code returns

[PATCH 18/25] vfs: hide file range comparison function

2018-10-12 Thread Darrick J. Wong
From: Darrick J. Wong There are no callers of vfs_dedupe_file_range_compare, so we might as well make it a static helper and remove the export. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein --- fs/read_write.c| 191

[PATCH 20/25] ocfs2: truncate page cache for clone destination file before remapping

2018-10-12 Thread Darrick J. Wong
From: Darrick J. Wong When cloning blocks into another file, truncate the page cache before we start remapping blocks so that concurrent reads wait for us to finish. Signed-off-by: Darrick J. Wong --- fs/ocfs2/refcounttree.c | 10 -- 1 file changed, 4 insertions(+), 6 deletions

[PATCH 11/25] vfs: pass remap flags to generic_remap_file_range_prep

2018-10-12 Thread Darrick J. Wong
From: Darrick J. Wong Plumb the remap flags through the filesystem from the vfs function dispatcher all the way to the prep function to prepare for behavior changes in subsequent patches. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein --- fs/ocfs2/file.c |2 +- fs

[PATCH 17/25] vfs: enable remap callers that can handle short operations

2018-10-12 Thread Darrick J. Wong
From: Darrick J. Wong Plumb in a remap flag that enables the filesystem remap handler to shorten remapping requests for callers that can handle it. Now copy_file_range can report partial success (in case we run up against alignment problems, resource limits, etc.). We also enable CAN_SHORTEN

[PATCH 12/25] vfs: pass remap flags to generic_remap_checks

2018-10-12 Thread Darrick J. Wong
From: Darrick J. Wong Pass the same remap flags to generic_remap_checks for consistency. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein --- fs/read_write.c|2 +- include/linux/fs.h |2 +- mm/filemap.c |4 ++-- 3 files changed, 4 insertions(+), 4 deletions

[PATCH 13/25] vfs: make remap_file_range functions take and return bytes completed

2018-10-12 Thread Darrick J. Wong
From: Darrick J. Wong Change the remap_file_range functions to take a number of bytes to operate upon and return the number of bytes they operated on. This is a requirement for allowing fs implementations to return short clone/dedupe results to the user, which will enable us to obey resource

[PATCH 15/25] vfs: plumb RFR_* remap flags through the vfs dedupe functions

2018-10-12 Thread Darrick J. Wong
From: Darrick J. Wong Plumb a remap_flags argument through the vfs_dedupe_file_range_one functions so that dedupe can take advantage of it. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein --- fs/overlayfs/file.c |3 ++- fs/read_write.c |9 ++--- include/linux/fs.h

[PATCH 14/25] vfs: plumb RFR_* remap flags through the vfs clone functions

2018-10-12 Thread Darrick J. Wong
From: Darrick J. Wong Plumb a remap_flags argument through the {do,vfs}_clone_file_range functions so that clone can take advantage of it. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein --- fs/ioctl.c |2 +- fs/nfsd/vfs.c |2 +- fs/overlayfs/copy_up.c

[PATCH 09/25] vfs: rename clone_verify_area to remap_verify_area

2018-10-12 Thread Darrick J. Wong
From: Darrick J. Wong Since we use clone_verify_area for both clone and dedupe range checks, rename the function to make it clear that it's for both. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein --- fs/read_write.c | 10 +- 1 file changed, 5 insertions(+), 5 dele

[PATCH 08/25] vfs: rename vfs_clone_file_prep to be more descriptive

2018-10-12 Thread Darrick J. Wong
From: Darrick J. Wong The vfs_clone_file_prep is a generic function to be called by filesystem implementations only. Rename the prefix to generic_ and make it more clear that it applies to remap operations, not just clones. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein --- fs

[PATCH 06/25] vfs: skip zero-length dedupe requests

2018-10-12 Thread Darrick J. Wong
From: Darrick J. Wong Don't bother calling the filesystem for a zero-length dedupe request; we can return zero and exit. Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig Reviewed-by: Amir Goldstein --- fs/read_write.c |5 + 1 file changed, 5 insertions(+) diff --

[PATCH 03/25] vfs: check file ranges before cloning files

2018-10-12 Thread Darrick J. Wong
From: Darrick J. Wong Move the file range checks from vfs_clone_file_prep into a separate generic_remap_checks function so that all the checks are collected in a central location. This forms the basis for adding more checks from generic_write_checks that will make cloning's input checking

[PATCH 10/25] vfs: create generic_remap_file_range_touch to update inode metadata

2018-10-12 Thread Darrick J. Wong
From: Darrick J. Wong Create a new VFS helper to handle inode metadata updates when remapping into a file. If the operation can possibly alter the file contents, we must update the ctime and mtime and remove security privileges, just like we do for regular file writes. Wire up ocfs2 to ensure

[PATCH 05/25] vfs: avoid problematic remapping requests into partial EOF block

2018-10-12 Thread Darrick J. Wong
From: Darrick J. Wong A deduplication data corruption is exposed in XFS and btrfs. It is caused by extending the block match range to include the partial EOF block, but then allowing unknown data beyond EOF to be considered a "match" to data in the destination file because the comparis

[PATCH 07/25] vfs: combine the clone and dedupe into a single remap_file_range

2018-10-12 Thread Darrick J. Wong
From: Darrick J. Wong Combine the clone_file_range and dedupe_file_range operations into a single remap_file_range file operation dispatch since they're fundamentally the same operation. The differences between the two can be made in the prep functions. Signed-off-by: Darrick J. Wong Rev

[PATCH 04/25] vfs: strengthen checking of file range inputs to generic_remap_checks

2018-10-12 Thread Darrick J. Wong
From: Darrick J. Wong File range remapping, if allowed to run past the destination file's EOF, is an optimization on a regular file write. Regular file writes that extend the file length are subject to various constraints which are not checked by range cloning. This is a correctness pr

[PATCH 02/25] vfs: vfs_clone_file_prep_inodes should return EINVAL for a clone from beyond EOF

2018-10-12 Thread Darrick J. Wong
From: Darrick J. Wong vfs_clone_file_prep_inodes cannot return 0 if it is asked to remap from a zero byte file because that's what btrfs does. Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig --- fs/read_write.c |3 --- 1 file changed, 3 deletions(-) diff --git

[PATCH v4 00/25] fs: fixes for serious clone/dedupe problems

2018-10-12 Thread Darrick J. Wong
Hi all, Dave, Eric, and I have been chasing a stale data exposure bug in the XFS reflink implementation, and tracked it down to reflink forgetting to do some of the file-extending activities that must happen for regular writes. We then started auditing the clone, dedupe, and copyfile code and rea

[PATCH 01/25] xfs: add a per-xfs trace_printk macro

2018-10-12 Thread Darrick J. Wong
From: Darrick J. Wong Add a "xfs_tprintk" macro so that developers can use trace_printk to print out arbitrary debugging information with the XFS device name attached to the trace output. Signed-off-by: Darrick J. Wong --- fs/xfs/xfs_error.h |6 ++ 1 file changed, 6

Re: [PATCH 05/25] vfs: avoid problematic remapping requests into partial EOF block

2018-10-12 Thread Darrick J. Wong
On Fri, Oct 12, 2018 at 11:16:16AM +1100, Dave Chinner wrote: > On Wed, Oct 10, 2018 at 09:12:54PM -0700, Darrick J. Wong wrote: > > From: Darrick J. Wong > > > > A deduplication data corruption is exposed by fstests generic/505 on > > XFS. It is caused by extendi

Re: [PATCH 24/25] xfs: support returning partial reflink results

2018-10-12 Thread Darrick J. Wong
On Fri, Oct 12, 2018 at 12:22:26PM +1100, Dave Chinner wrote: > On Wed, Oct 10, 2018 at 09:15:19PM -0700, Darrick J. Wong wrote: > > From: Darrick J. Wong > > > > Back when the XFS reflink code only supported clone_file_range, we were > > only able to return zero

Re: [PATCH 01/25] xfs: add a per-xfs trace_printk macro

2018-10-11 Thread Darrick J. Wong
On Thu, Oct 11, 2018 at 06:39:34AM -0700, Christoph Hellwig wrote: > On Wed, Oct 10, 2018 at 09:12:23PM -0700, Darrick J. Wong wrote: > > From: Darrick J. Wong > > > > Add a "xfs_tprintk" macro so that developers can use trace_printk to > > print out arbitra

[PATCH v2 17/25] vfs: enable remap callers that can handle short operations

2018-10-11 Thread Darrick J. Wong
From: Darrick J. Wong Plumb in a remap flag that enables the filesystem remap handler to shorten remapping requests for callers that can handle it. Now copy_file_range can report partial success (in case we run up against alignment problems, resource limits, etc.). We also enable CAN_SHORTEN

Re: [PATCH 17/25] vfs: enable remap callers that can handle short operations

2018-10-11 Thread Darrick J. Wong
On Thu, Oct 11, 2018 at 08:15:42AM +0300, Amir Goldstein wrote: > On Thu, Oct 11, 2018 at 7:14 AM Darrick J. Wong > wrote: > > > > From: Darrick J. Wong > > > > Plumb in a remap flag that enables the filesystem remap handler to > > shorten remapping requests

Re: [PATCH v3 00/25] fs: fixes for serious clone/dedupe problems

2018-10-11 Thread Darrick J. Wong
On Thu, Oct 11, 2018 at 11:33:57AM +0300, Amir Goldstein wrote: > On Thu, Oct 11, 2018 at 7:12 AM Darrick J. Wong > wrote: > > > > Hi all, > > > > Dave, Eric, and I have been chasing a stale data exposure bug in the XFS > > reflink implementation, and tracked

Re: [PATCH v3] geneirc/077 fix min size for btrfs

2018-10-10 Thread Darrick J. Wong
On Thu, Oct 11, 2018 at 11:26:00AM +0800, Anand Jain wrote: > If btrfs need to be tested at its default blockgroup which is non-mixed, > then it needs at least 256mb. > > Signed-off-by: Anand Jain > --- > v2->v3: > separated from the patch set of 9. > notrun for the cases where filler

[PATCH 24/25] xfs: support returning partial reflink results

2018-10-10 Thread Darrick J. Wong
From: Darrick J. Wong Back when the XFS reflink code only supported clone_file_range, we were only able to return zero or negative error codes to userspace. However, now that copy_file_range (which returns bytes copied) can use XFS' clone_file_range, we have the opportunity to return pa

[PATCH 25/25] xfs: remove redundant remap partial EOF block checks

2018-10-10 Thread Darrick J. Wong
From: Darrick J. Wong Now that we've moved the partial EOF block checks to the VFS helpers, we can remove the redundantn functionality from XFS. Signed-off-by: Darrick J. Wong --- fs/xfs/xfs_reflink.c | 20 1 file changed, 20 deletions(-) diff --git a/f

[PATCH 22/25] ocfs2: support partial clone range and dedupe range

2018-10-10 Thread Darrick J. Wong
From: Darrick J. Wong Change the ocfs2 remap code to allow for returning partial results. Signed-off-by: Darrick J. Wong --- fs/ocfs2/file.c |7 + fs/ocfs2/refcounttree.c | 73 ++- fs/ocfs2/refcounttree.h | 12 3 files

[PATCH 23/25] xfs: fix pagecache truncation prior to reflink

2018-10-10 Thread Darrick J. Wong
From: Darrick J. Wong Prior to remapping blocks, it is necessary to remove pages from the destination file's page cache. Unfortunately, the truncation is not aggressive enough -- if page size > block size, we'll end up zeroing subpage blocks instead of removing them. So, round the

[PATCH 20/25] ocfs2: truncate page cache for clone destination file before remapping

2018-10-10 Thread Darrick J. Wong
From: Darrick J. Wong When cloning blocks into another file, truncate the page cache before we start remapping blocks so that concurrent reads wait for us to finish. Signed-off-by: Darrick J. Wong --- fs/ocfs2/refcounttree.c | 10 -- 1 file changed, 4 insertions(+), 6 deletions

[PATCH 21/25] ocfs2: fix pagecache truncation prior to reflink

2018-10-10 Thread Darrick J. Wong
From: Darrick J. Wong Prior to remapping blocks, it is necessary to remove pages from the destination file's page cache. Unfortunately, the truncation is not aggressive enough -- if page size > block size, we'll end up zeroing subpage blocks instead of removing them. So, round the

[PATCH 19/25] vfs: implement opportunistic short dedupe

2018-10-10 Thread Darrick J. Wong
From: Darrick J. Wong For a given dedupe request, the bytes_deduped field in the control structure tells userspace if we managed to deduplicate some, but not all of, the requested regions starting from the file offsets supplied. However, due to sloppy coding, the current dedupe code returns

[PATCH 18/25] vfs: hide file range comparison function

2018-10-10 Thread Darrick J. Wong
From: Darrick J. Wong There are no callers of vfs_dedupe_file_range_compare, so we might as well make it a static helper and remove the export. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein --- fs/read_write.c| 191

[PATCH 16/25] vfs: make remapping to source file eof more explicit

2018-10-10 Thread Darrick J. Wong
From: Darrick J. Wong Create a RFR_TO_SRC_EOF flag to explicitly declare that the caller wants the remap implementation to remap to the end of the source file, once the files are locked. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein --- fs/ioctl.c |3 ++- fs/nfsd

[PATCH 17/25] vfs: enable remap callers that can handle short operations

2018-10-10 Thread Darrick J. Wong
From: Darrick J. Wong Plumb in a remap flag that enables the filesystem remap handler to shorten remapping requests for callers that can handle it. Now copy_file_range can report partial success (in case we run up against alignment problems, resource limits, etc.). Signed-off-by: Darrick J

[PATCH 15/25] vfs: plumb RFR_* remap flags through the vfs dedupe functions

2018-10-10 Thread Darrick J. Wong
From: Darrick J. Wong Plumb a remap_flags argument through the vfs_dedupe_file_range_one functions so that dedupe can take advantage of it. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein --- fs/overlayfs/file.c |3 ++- fs/read_write.c |9 ++--- include/linux/fs.h

[PATCH 11/25] vfs: pass remap flags to generic_remap_file_range_prep

2018-10-10 Thread Darrick J. Wong
From: Darrick J. Wong Plumb the remap flags through the filesystem from the vfs function dispatcher all the way to the prep function to prepare for behavior changes in subsequent patches. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein --- fs/ocfs2/file.c |2 +- fs

[PATCH 12/25] vfs: pass remap flags to generic_remap_checks

2018-10-10 Thread Darrick J. Wong
From: Darrick J. Wong Pass the same remap flags to generic_remap_checks for consistency. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein --- fs/read_write.c|2 +- include/linux/fs.h |2 +- mm/filemap.c |4 ++-- 3 files changed, 4 insertions(+), 4 deletions

[PATCH 13/25] vfs: make remap_file_range functions take and return bytes completed

2018-10-10 Thread Darrick J. Wong
From: Darrick J. Wong Change the remap_file_range functions to take a number of bytes to operate upon and return the number of bytes they operated on. This is a requirement for allowing fs implementations to return short clone/dedupe results to the user, which will enable us to obey resource

<    1   2   3   4   5   6   7   8   9   >