[PATCH] btrfs-progs: check: add experimental flag for lowmem mode

2018-07-03 Thread David Disseldorp 
The experimental flag is already carried in the manpage, but was removed
from the btrfs check usage message as part of refactoring via
87c1bd13c1fca430c3dbf0da62e9aa33bde609c8. Add it back.

Signed-off-by: David Disseldorp 
Reviewed-by: Nikolay Borisov 
---
 check/main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/check/main.c b/check/main.c
index 3190b5d4..9aca0c09 100644
--- a/check/main.c
+++ b/check/main.c
@@ -9386,7 +9386,7 @@ const char * const cmd_check_usage[] = {
"original - read inodes and extents to 
memory (requires",
"   more memory, does less IO)",
"lowmem   - try to use less memory but read 
blocks again",
-   "   when needed",
+   "   when needed (experimental)",
"--check-data-csum   verify checksums of data blocks",
"-Q|--qgroup-report  print a report on qgroup consistency",
"-E|--subvol-extents ",
-- 
2.13.7

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH RFC 0/3] btrfs-progs: lowmem: delay before lowmem repair

2018-07-02 Thread David Disseldorp 
On Mon, 2 Jul 2018 12:43:20 +0300, Nikolay Borisov wrote:

> On  2.07.2018 12:28, Su Yue wrote:
> > Since lowmem repair is dangerous, it should remind user more obviously.
> > The patchset add 10 seconds delay like btrfs balance and add am option
> > '--force-repair-lowmem' to skip the delay.  
> 
> IMO this is the wrong way to approach a dangerous option. If it's so
> dangerous it needs to be written in the documentation explicitly this is
> so. If someone wants to use lowmem then they should explicitly set
> --mode lowmem. So I'm inclined to NACK this patch.

AFAICT it's already documented as "experimental" in the manpage, but the
usage flag appears to have been dropped as part of refactoring for
87c1bd13c1fca430c3dbf0da62e9aa33bde609c8 . If nobody's working on a fix,
and lowmem removal isn't an option, then please consider adding the usage
flag back, e.g.
--- a/check/main.c
+++ b/check/main.c
@@ -9386,7 +9386,7 @@ const char * const cmd_check_usage[] = {
"original - read inodes and extents to 
memory (requires",
"   more memory, does less IO)",
"lowmem   - try to use less memory but read 
blocks again",
-   "   when needed",
+   "   when needed (experimental)",
"--check-data-csum   verify checksums of data blocks",
"-Q|--qgroup-report  print a report on qgroup consistency",
"-E|--subvol-extents ",

Cheers, David
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] xfstests: btrfs: add test regarding clearing compression flag/property

2014-09-23 Thread David Disseldorp 
On Thu, 11 Sep 2014 11:45:34 +0100, Filipe Manana wrote:

 Regression test for btrfs where removing the flag FS_COMPR_FL
 (chattr -c) from an inode wouldn't clear its compression property.
 This was fixed in the following linux kernel patch:
 
   Btrfs: add missing compression property remove in btrfs_ioctl_setflags
 
 Signed-off-by: Filipe Manana fdman...@suse.com

I would have expected this to fail with MOUNT_OPTIONS=-o compress=no,
but it looks as though Btrfs stores the attributes unconditionally.
Either way this change looks good to me.
Reviewed-by: David Disseldorp dd...@suse.de

Cheers, David
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2] btrfs-progs: fix typo in subvol list usage

2014-08-18 Thread David Disseldorp 
Signed-off-by: David Disseldorp dd...@suse.de
---
 Documentation/btrfs-subvolume.txt | 2 +-
 cmds-subvolume.c  | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/Documentation/btrfs-subvolume.txt 
b/Documentation/btrfs-subvolume.txt
index a519131..c8b9928 100644
--- a/Documentation/btrfs-subvolume.txt
+++ b/Documentation/btrfs-subvolume.txt
@@ -99,7 +99,7 @@ print the ogeneration of the subvolume, aliases: ogen or 
origin generation.
 -g
 print the generation of the subvolume.
 -o
-print only subvolumes bellow specified path.
+print only subvolumes below the specified path.
 -u
 print the UUID of the subvolume.
 -q
diff --git a/cmds-subvolume.c b/cmds-subvolume.c
index 5e821c7..75a7385 100644
--- a/cmds-subvolume.c
+++ b/cmds-subvolume.c
@@ -390,7 +390,7 @@ static const char * const cmd_subvol_list_usage[] = {
 to the given path,
-c   print the ogeneration of the subvolume,
-g   print the generation of the subvolume,
-   -o   print only subvolumes bellow specified path,
+   -o   print only subvolumes below the specified path,
-u   print the uuid of subvolumes (and snapshots),
-q   print the parent uuid of the snapshots,
-t   print the result as a table,
-- 
1.8.4.5

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 0/2] xfstests: add cifs.ko server-side copy helper

2014-06-11 Thread David Disseldorp 
In preparation for adding cifs.ko support to xfstests, this patch series
extends the cloner binary to support SMB2 server-side copies via
CIFS_IOC_COPYCHUNK_FILE, in addition to the existing Btrfs COW clone
functionality.

This could be split into a separate binary if deemed necessary, but
given the code overlap, I though it suitable to share the same source.

Feedback appreciated.

--

David Disseldorp (2):
  src/cloner: check filesystem type
  src/cloner: add CIFS_IOC_COPYCHUNK_FILE support

 configure.ac |   1 +
 src/cloner.c | 123 
+--
 2 files changed, 114 insertions(+), 10 deletions(-)
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/2] src/cloner: add CIFS_IOC_COPYCHUNK_FILE support

2014-06-11 Thread David Disseldorp 
cifs.ko supports server-side copy offloads via CIFS_IOC_COPYCHUNK_FILE.
In handling the ioctl, the request is split into a series of
SMB2 FSCTL_SRV_COPYCHUNK wire requests, which may be handled by the SMB
server as a local read/write, or COW clone as is the case for Samba with
vfs_btrfs.

Signed-off-by: David Disseldorp dd...@suse.de
---
 configure.ac |  1 +
 src/cloner.c | 78 +++-
 2 files changed, 68 insertions(+), 11 deletions(-)

diff --git a/configure.ac b/configure.ac
index 53459d8..d038f95 100644
--- a/configure.ac
+++ b/configure.ac
@@ -31,6 +31,7 @@ AC_HEADER_STDC
sys/fs/xfs_itable.h \
xfs/platform_defs.h \
btrfs/ioctl.h   \
+   cifs/ioctl.h\
 ])
 
 AC_CHECK_HEADERS([xfs/xfs_log_format.h],,,[#include xfs/libxfs.h])
diff --git a/src/cloner.c b/src/cloner.c
index 6fb40fa..18c44b9 100644
--- a/src/cloner.c
+++ b/src/cloner.c
@@ -1,6 +1,5 @@
 /*
- *  Tiny program to perform file (range) clones using raw Btrfs ioctls.
- *  It should only be needed until btrfs-progs has an xfs_io equivalent.
+ *  Tiny program to perform file (range) clones using raw Btrfs and CIFS 
ioctls.
  *
  *  Copyright (C) 2014 SUSE Linux Products GmbH. All Rights Reserved.
  *
@@ -49,9 +48,21 @@ struct btrfs_ioctl_clone_range_args {
   struct btrfs_ioctl_clone_range_args)
 #endif
 
+#ifdef HAVE_CIFS_IOCTL_H
+#include cifs/ioctl.h
+#else
+
+#define CIFS_IOCTL_MAGIC 0xCF
+#define CIFS_IOC_COPYCHUNK_FILE _IOW(CIFS_IOCTL_MAGIC, 3, int)
+
+#endif
+
 #ifndef BTRFS_SUPER_MAGIC
 #define BTRFS_SUPER_MAGIC0x9123683E
 #endif
+#ifndef CIFS_MAGIC_NUMBER
+#define CIFS_MAGIC_NUMBER0xFE534D42
+#endif
 
 static void
 usage(char *name, const char *msg)
@@ -59,17 +70,19 @@ usage(char *name, const char *msg)
printf(Fatal: %s\n
   Usage:\n
   %s [options] src_file dest_file\n
-  \tA full file clone (reflink) is performed by default, 
-  unless any of the following are specified:\n
+  \tA full file clone is performed by default, 
+  unless any of the following are specified (Btrfs only):\n
   \t-s offset:  source file offset (default = 0)\n
   \t-d offset:  destination file offset (default = 0)\n
-  \t-l length:  length of clone (default = 0)\n,
+  \t-l length:  length of clone (default = 0)\n\n
+  \tBoth Btrfs and CIFS are supported. On Btrfs, a COW clone 
+  is attempted. On CIFS, a server-side copy is requested.\n,
   msg, name);
_exit(1);
 }
 
 static int
-clone_file(int src_fd, int dst_fd)
+clone_file_btrfs(int src_fd, int dst_fd)
 {
int ret = ioctl(dst_fd, BTRFS_IOC_CLONE, src_fd);
if (ret != 0)
@@ -78,8 +91,33 @@ clone_file(int src_fd, int dst_fd)
 }
 
 static int
-clone_file_range(int src_fd, int dst_fd, uint64_t src_off, uint64_t dst_off,
-uint64_t len)
+clone_file_cifs(int src_fd, int dst_fd)
+{
+   int ret = ioctl(dst_fd, CIFS_IOC_COPYCHUNK_FILE, src_fd);
+   if (ret != 0)
+   ret = errno;
+   return ret;
+}
+
+static int
+clone_file(unsigned int fs_type, int src_fd, int dst_fd)
+{
+   switch (fs_type) {
+   case BTRFS_SUPER_MAGIC:
+   return clone_file_btrfs(src_fd, dst_fd);
+   break;
+   case CIFS_MAGIC_NUMBER:
+   return clone_file_cifs(src_fd, dst_fd);
+   break;
+   default:
+   return ENOTSUP;
+   break;
+   }
+}
+
+static int
+clone_file_range_btrfs(int src_fd, int dst_fd, uint64_t src_off,
+  uint64_t dst_off, uint64_t len)
 {
struct btrfs_ioctl_clone_range_args cr_args;
int ret;
@@ -96,6 +134,22 @@ clone_file_range(int src_fd, int dst_fd, uint64_t src_off, 
uint64_t dst_off,
 }
 
 static int
+clone_file_range(unsigned int fs_type, int src_fd, int dst_fd, uint64_t 
src_off,
+uint64_t dst_off, uint64_t len)
+{
+   switch (fs_type) {
+   case BTRFS_SUPER_MAGIC:
+   return clone_file_range_btrfs(src_fd, dst_fd, src_off, dst_off,
+ len);
+   break;
+   case CIFS_MAGIC_NUMBER: /* only supports full file server-side copies */
+   default:
+   return ENOTSUP;
+   break;
+   }
+}
+
+static int
 cloner_check_fs_support(int src_fd, int dest_fd, unsigned int *fs_type)
 {
int ret;
@@ -107,7 +161,8 @@ cloner_check_fs_support(int src_fd, int dest_fd, unsigned 
int *fs_type)
return errno;
}
 
-   if (sfs.f_type != BTRFS_SUPER_MAGIC) {
+   if ((sfs.f_type != BTRFS_SUPER_MAGIC)
+ (sfs.f_type != CIFS_MAGIC_NUMBER)) {
printf(unsupported source FS 0x%x\n,
   (unsigned int

[PATCH 1/2] src/cloner: check filesystem type

2014-06-11 Thread David Disseldorp 
Limit clone requests to Btrfs only for the moment.

Signed-off-by: David Disseldorp dd...@suse.de
---
 src/cloner.c | 47 +++
 1 file changed, 47 insertions(+)

diff --git a/src/cloner.c b/src/cloner.c
index ccc2354..6fb40fa 100644
--- a/src/cloner.c
+++ b/src/cloner.c
@@ -22,6 +22,7 @@
 #include sys/types.h
 #include sys/stat.h
 #include sys/ioctl.h
+#include sys/vfs.h
 #include stdint.h
 #include stdbool.h
 #include fcntl.h
@@ -30,6 +31,7 @@
 #include stdio.h
 #include string.h
 #include errno.h
+#include linux/magic.h
 #ifdef HAVE_BTRFS_IOCTL_H
 #include btrfs/ioctl.h
 #else
@@ -47,6 +49,10 @@ struct btrfs_ioctl_clone_range_args {
   struct btrfs_ioctl_clone_range_args)
 #endif
 
+#ifndef BTRFS_SUPER_MAGIC
+#define BTRFS_SUPER_MAGIC0x9123683E
+#endif
+
 static void
 usage(char *name, const char *msg)
 {
@@ -89,6 +95,41 @@ clone_file_range(int src_fd, int dst_fd, uint64_t src_off, 
uint64_t dst_off,
return ret;
 }
 
+static int
+cloner_check_fs_support(int src_fd, int dest_fd, unsigned int *fs_type)
+{
+   int ret;
+   struct statfs sfs;
+
+   ret = fstatfs(src_fd, sfs);
+   if (ret != 0) {
+   printf(failed to stat source FS\n);
+   return errno;
+   }
+
+   if (sfs.f_type != BTRFS_SUPER_MAGIC) {
+   printf(unsupported source FS 0x%x\n,
+  (unsigned int)sfs.f_type);
+   return ENOTSUP;
+   }
+
+   *fs_type = (unsigned int)sfs.f_type;
+
+   ret = fstatfs(dest_fd, sfs);
+   if (ret != 0) {
+   printf(failed to stat destination FS\n);
+   return errno;
+   }
+
+   if (sfs.f_type != *fs_type) {
+   printf(dest FS type 0x%x does not match source 0x%x\n,
+  (unsigned int)sfs.f_type, *fs_type);
+   return ENOTSUP;
+   }
+
+   return 0;
+}
+
 int
 main(int argc, char **argv)
 {
@@ -102,6 +143,7 @@ main(int argc, char **argv)
int dst_fd;
int ret;
int opt;
+   unsigned int fs_type = 0;
 
while ((opt = getopt(argc, argv, s:d:l:)) != -1) {
char *sval_end;
@@ -162,6 +204,11 @@ main(int argc, char **argv)
goto err_src_close;
}
 
+   ret = cloner_check_fs_support(src_fd, dst_fd, fs_type);
+   if (ret != 0) {
+   goto err_dst_close;
+   }
+
if (full_file) {
ret = clone_file(src_fd, dst_fd);
} else {
-- 
1.8.4.5

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/2 v3] xfstests: add test for btrfs ioctl clone operation

2014-05-26 Thread David Disseldorp 
On Sat, 24 May 2014 17:50:55 +0100, Filipe David Borba Manana wrote:

 This is a test to verify that the btrfs ioctl clone operation is
 able to clone extents of a file to different positions of the file,
 that is, the source and target files are the same. Existing tests
 only cover the case where the source and target files are different.
 
 Signed-off-by: Filipe David Borba Manana fdman...@gmail.com
 ---
 
 V2: Made the test exercise a more complex code path in the btrfs ioctl clone
 code. Now we have extents with different sizes and make the cloner process
 partial extents and split existing extents with smaller ones.
 
 V3: Add tests to verify that after writing to a cloned extent, the original
 extent isn't modified, that defragmenting a file with cloned extents
 doesn't change the file contents and that all the tests have the same
 exact semantics (as observed by an application/user) regardless of the
 following options (and any combination): cow/nodatacow/compression.

Looks good.

Reviewed-by: David Disseldorp dd...@suse.de
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/2] xfstests: add helper require function _require_btrfs_cloner

2014-05-23 Thread David Disseldorp 
On Fri, 23 May 2014 05:05:30 +0100, Filipe David Borba Manana wrote:

 So that the same check (btrfs cloner program presence) can be reused
 by other tests.
 
 Signed-off-by: Filipe David Borba Manana fdman...@gmail.com
 ---
  common/rc   | 7 +++
  tests/btrfs/035 | 4 +---
  2 files changed, 8 insertions(+), 3 deletions(-)
 
 diff --git a/common/rc b/common/rc
 index d1788d1..f27ee53 100644
 --- a/common/rc
 +++ b/common/rc
 @@ -2085,6 +2085,13 @@ _require_fssum()
   [ -x $FSSUM_PROG ] || _notrun fssum not built
  }
  
 +_require_btrfs_cloner()
 +{
 + CLONER_PROG=$here/src/cloner
 + [ -x $CLONER_PROG ] || \
 + _notrun cloner binary not present at $CLONER_PROG
 +}

Would prefer to avoid the reliance on $here, but it appears that other
common/rc functions make the same assumption.

Reviewed-by: David Disseldorp dd...@suse.de

Cheers, David
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/2] xfstests: add test for btrfs ioctl clone operation

2014-05-23 Thread David Disseldorp 
On Fri, 23 May 2014 05:05:31 +0100, Filipe David Borba Manana wrote:

 This is a test to verify that the btrfs ioctl clone operation is
 able to clone extents of a file to different positions of the file,
 that is, the source and target files are the same. Existing tests
 only cover the case where the source and target files are different.
 
 Signed-off-by: Filipe David Borba Manana fdman...@gmail.com

Nice work Filipe.

Reviewed-by: David Disseldorp dd...@suse.de

Cheers, David
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] btrfs/035: update clone test to expect EOPNOTSUPP

2014-05-08 Thread David Disseldorp 
Hi liubo,

On Thu, 8 May 2014 12:11:24 +0800, Liu Bo wrote:

 Something different here, I didn't get EIO on 3.15.0-rc4.

Strange, I'm able to consistently reproduce this on a
vanilla v3.15-rc4-202-g30321c7 kernel.
Does that mean the updated test passes successfully for you?

Cheers, David
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] btrfs/035: update clone test to expect EOPNOTSUPP

2014-05-07 Thread David Disseldorp 
With kernel commit 00fdf13a2e9f313a044288aa59d3b8ec29ff904a, the first
clone-range overwrite attempt now fails with EOPNOTSUPP, rather than
tripping a Btrfs BUG_ON().

This test now trips a new Btrfs bug, in which EIO is returned for
subsequent reads following the second clone range ioctl.

Signed-off-by: David Disseldorp dd...@suse.de
---
 tests/btrfs/035 | 11 +++
 tests/btrfs/035.out |  5 +
 2 files changed, 16 insertions(+)

diff --git a/tests/btrfs/035 b/tests/btrfs/035
index 6808179..c9530f6 100755
--- a/tests/btrfs/035
+++ b/tests/btrfs/035
@@ -57,21 +57,32 @@ src_str=aa
 echo -n $src_str  $SCRATCH_MNT/src
 
 $CLONER_PROG $SCRATCH_MNT/src  $SCRATCH_MNT/src.clone1
+cat $SCRATCH_MNT/src.clone1
+echo
 
 src_str=bbcc
 
 echo -n $src_str  $SCRATCH_MNT/src
 
 $CLONER_PROG $SCRATCH_MNT/src $SCRATCH_MNT/src.clone2
+cat $SCRATCH_MNT/src.clone2
+echo
 
+# Prior to kernel commit 00fdf13a2e9f313a044288aa59d3b8ec29ff904a, this clone
+# resulted in a BUG_ON in __btrfs_drop_extents(). The kernel now returns
+# EOPNOTSUPP up to userspace.
 snap_src_sz=`ls -lah $SCRATCH_MNT/src.clone1 | awk '{print $5}'`
 echo attempting ioctl (src.clone1 src)
 $CLONER_PROG -s 0 -d 0 -l ${snap_src_sz} \
$SCRATCH_MNT/src.clone1 $SCRATCH_MNT/src
+cat $SCRATCH_MNT/src
+echo
 
 snap_src_sz=`ls -lah $SCRATCH_MNT/src.clone2 | awk '{print $5}'`
 echo attempting ioctl (src.clone2 src)
 $CLONER_PROG -s 0 -d 0 -l ${snap_src_sz} \
$SCRATCH_MNT/src.clone2 $SCRATCH_MNT/src
+# BUG: subsequent access attempts currently result in EIO...
+cat $SCRATCH_MNT/src
 
 status=0 ; exit
diff --git a/tests/btrfs/035.out b/tests/btrfs/035.out
index f86cadf..0ea2c4f 100644
--- a/tests/btrfs/035.out
+++ b/tests/btrfs/035.out
@@ -1,3 +1,8 @@
 QA output created by 035
+aa
+bbcc
 attempting ioctl (src.clone1 src)
+clone failed: Operation not supported
+bbcc
 attempting ioctl (src.clone2 src)
+bbcc
-- 
1.8.4.5

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] btrfs-progs: fix typo in subvol list usage

2014-04-16 Thread David Disseldorp 
Signed-off-by: David Disseldorp dd...@suse.de
---
 cmds-subvolume.c | 2 +-
 man/btrfs.8.in   | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/cmds-subvolume.c b/cmds-subvolume.c
index 5e821c7..75a7385 100644
--- a/cmds-subvolume.c
+++ b/cmds-subvolume.c
@@ -390,7 +390,7 @@ static const char * const cmd_subvol_list_usage[] = {
 to the given path,
-c   print the ogeneration of the subvolume,
-g   print the generation of the subvolume,
-   -o   print only subvolumes bellow specified path,
+   -o   print only subvolumes below the specified path,
-u   print the uuid of subvolumes (and snapshots),
-q   print the parent uuid of the snapshots,
-t   print the result as a table,
diff --git a/man/btrfs.8.in b/man/btrfs.8.in
index 8fea115..4221cc2 100644
--- a/man/btrfs.8.in
+++ b/man/btrfs.8.in
@@ -214,7 +214,7 @@ print the ogeneration of the subvolume, aliases: ogen or 
origin generation.
 .IP \fB-g\fP 5
 print the generation of the subvolume.
 .IP \fB-o\fP 5
-print only subvolumes bellow specified path.
+print only subvolumes below the specified path.
 .IP \fB-u\fP 5
 print the UUID of the subvolume.
 .IP \fB-q\fP 5
-- 
1.8.4.5

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] btrfs/035: update clone test to expect EOPNOTSUPP

2014-04-09 Thread David Disseldorp 
With kernel commit 00fdf13a2e9f313a044288aa59d3b8ec29ff904a, the first
clone-range overwrite attempt now fails with EOPNOTSUPP.

FIXME: The second clone-range causes EIO on subsequent read attempts.

Signed-off-by: David Disseldorp dd...@suse.de
---
 tests/btrfs/035 | 10 ++
 tests/btrfs/035.out |  5 +
 2 files changed, 15 insertions(+)

diff --git a/tests/btrfs/035 b/tests/btrfs/035
index 6808179..21a9059 100755
--- a/tests/btrfs/035
+++ b/tests/btrfs/035
@@ -57,21 +57,31 @@ src_str=aa
 echo -n $src_str  $SCRATCH_MNT/src
 
 $CLONER_PROG $SCRATCH_MNT/src  $SCRATCH_MNT/src.clone1
+cat $SCRATCH_MNT/src.clone1
+echo
 
 src_str=bbcc
 
 echo -n $src_str  $SCRATCH_MNT/src
 
 $CLONER_PROG $SCRATCH_MNT/src $SCRATCH_MNT/src.clone2
+cat $SCRATCH_MNT/src.clone2
+echo
 
+# Prior to kernel commit 00fdf13a2e9f313a044288aa59d3b8ec29ff904a, this clone
+# resulted in a BUG_ON in __btrfs_drop_extents(). The kernel now returns EINVAL
+# up to userspace.
 snap_src_sz=`ls -lah $SCRATCH_MNT/src.clone1 | awk '{print $5}'`
 echo attempting ioctl (src.clone1 src)
 $CLONER_PROG -s 0 -d 0 -l ${snap_src_sz} \
$SCRATCH_MNT/src.clone1 $SCRATCH_MNT/src
+cat $SCRATCH_MNT/src
+echo
 
 snap_src_sz=`ls -lah $SCRATCH_MNT/src.clone2 | awk '{print $5}'`
 echo attempting ioctl (src.clone2 src)
 $CLONER_PROG -s 0 -d 0 -l ${snap_src_sz} \
$SCRATCH_MNT/src.clone2 $SCRATCH_MNT/src
+cat $SCRATCH_MNT/src
 
 status=0 ; exit
diff --git a/tests/btrfs/035.out b/tests/btrfs/035.out
index f86cadf..0ea2c4f 100644
--- a/tests/btrfs/035.out
+++ b/tests/btrfs/035.out
@@ -1,3 +1,8 @@
 QA output created by 035
+aa
+bbcc
 attempting ioctl (src.clone1 src)
+clone failed: Operation not supported
+bbcc
 attempting ioctl (src.clone2 src)
+bbcc
-- 
1.8.4.5

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] Btrfs: fix a crash of clone with inline extents's split

2014-04-09 Thread David Disseldorp 
Thanks for the BUG_ON() fix here.
Strangely, I'm now seeing EIO returned for reads following the second
clone-range.

Please see the subsequent xfstests patch.

Cheers, David
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Snapper on Ubuntu

2014-03-15 Thread David Disseldorp 
On Sat, 15 Mar 2014 18:47:49 +0100
Hendrik Friedel hend...@friedels.name wrote:

  I think you may have forgotten to specify the config snapper is supposed
  to use. Try
 
  # snapper -c home create
  # snapper -c Video create  
 
 Thanks, that was it. I would have expected an Error-Message though...

Snapper uses the root config by default. /root snapshots were created
successfully when you ran snapper create without specifying an explicit
config.

Cheers, David
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

xfstests btrfs/035 (was Re: Testing BTRFS)

2014-03-12 Thread David Disseldorp 
Hi Eric,

On Tue, 11 Mar 2014 14:08:02 -0500, Eric Sandeen wrote:

 Indeed, testing 3.8.13-26.2.1.el6uek.x86_64 (which is, I believe,
 the kernel which Avi referred to) via xfstests, I saw
 failures on btrfs/009 and btrfs/022; then the box deadlocked
 on btrfs/024.  I rebooted  resumed, then deadlocked on btrfs/030.
 Rebooted and resumed again, then panicked on btrfs/035.  At that
 point I stopped.

FWIW, Liu Bo recently proposed a fix for the btrfs/035 BUG_ON():

http://permalink.gmane.org/gmane.comp.file-systems.btrfs/33327

Cheers, David
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 0/3] __btrfs_drop_extents() BUG_ON reproducer

2014-02-14 Thread David Disseldorp 
Ping, any Btrfsers get a chance to look at this patch series?
I'd like to get it into the QA tree.

On Fri,  7 Feb 2014 11:35:38 +0100, David Disseldorp wrote:

 This patch-set provides a reproducer for hitting the 3.14.0-rc1 BUG_ON()
 at:
  692 int __btrfs_drop_extents(struct btrfs_trans_handle *trans,
 ...
  839 /*
  840  *  |  range to drop - |
  841  *  |  extent  |
  842  */
  843 if (start = key.offset  end  extent_end) {
  844 BUG_ON(extent_type == BTRFS_FILE_EXTENT_INLINE);
  845 
  846 memcpy(new_key, key, sizeof(new_key));
 
 The first patch adds a small cloner binary which is used by btrfs/035 to
 dispatch BTRFS_IOC_CLONE_RANGE requests.
 
 This workload resembles that of Samba's vfs_btrfs module, when a Windows
 client restores a file from a shadow-copy (snapshot) using server-side
 copy requests.
 
 Changes since V2:
 - Remove explicit write error checks
 
 Changes since V1:
 - Use strtoull instead of atoi
 - Print error conditions in cloner
 - Check for cloner binary before running test
 - Continue test on failure
 - Add cloner to .gitignore
 
 Feedback appreciated.
 
 Cheers, David
 
 
  .gitignore  |   1 +
  configure.ac|   1 +
  src/Makefile|   2 +-
  src/cloner.c| 192 
 +++
  tests/btrfs/035 |  77 
 +
  tests/btrfs/035.out |   3 +++
  tests/btrfs/group   |   1 +
  7 files changed, 276 insertions(+), 1 deletion(-)
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 3/3] btrfs/035: add new clone overwrite regression test

2014-02-07 Thread David Disseldorp 
On Fri, 7 Feb 2014 09:45:09 +1100, Dave Chinner wrote:

 Not exactly what I intended.
 
 If echo fails, it will output some kind of error message, and that
 will cause the golden image mismatch.

Fair enough. I'll resend with the two '  || echo failed to create src'
checks removed. Thanks again for the review Dave.

Cheers, David
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 3/3] btrfs/035: add new clone overwrite regression test

2014-02-07 Thread David Disseldorp 
This test uses the newly added cloner binary to dispatch full file and
range specific clone (reflink) requests.

Signed-off-by: David Disseldorp dd...@suse.de
---
 tests/btrfs/035 | 77 +
 tests/btrfs/035.out |  3 +++
 tests/btrfs/group   |  1 +
 3 files changed, 81 insertions(+)
 create mode 100755 tests/btrfs/035
 create mode 100644 tests/btrfs/035.out

diff --git a/tests/btrfs/035 b/tests/btrfs/035
new file mode 100755
index 000..6808179
--- /dev/null
+++ b/tests/btrfs/035
@@ -0,0 +1,77 @@
+#!/bin/bash
+# FS QA Test No. btrfs/035
+#
+# Regression test for overwriting clones
+#
+#---
+# Copyright (C) 2014 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#---
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo QA output created by $seq
+
+here=`pwd`
+tmp=/tmp/$$
+status=1   # failure is the default!
+
+_cleanup()
+{
+rm -f $tmp.*
+}
+
+trap _cleanup ; exit \$status 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs  /dev/null 21
+_scratch_mount
+
+CLONER_PROG=$here/src/cloner
+[ -x $CLONER_PROG ] || _notrun cloner binary not present at $CLONER_PROG
+
+src_str=aa
+
+echo -n $src_str  $SCRATCH_MNT/src
+
+$CLONER_PROG $SCRATCH_MNT/src  $SCRATCH_MNT/src.clone1
+
+src_str=bbcc
+
+echo -n $src_str  $SCRATCH_MNT/src
+
+$CLONER_PROG $SCRATCH_MNT/src $SCRATCH_MNT/src.clone2
+
+snap_src_sz=`ls -lah $SCRATCH_MNT/src.clone1 | awk '{print $5}'`
+echo attempting ioctl (src.clone1 src)
+$CLONER_PROG -s 0 -d 0 -l ${snap_src_sz} \
+   $SCRATCH_MNT/src.clone1 $SCRATCH_MNT/src
+
+snap_src_sz=`ls -lah $SCRATCH_MNT/src.clone2 | awk '{print $5}'`
+echo attempting ioctl (src.clone2 src)
+$CLONER_PROG -s 0 -d 0 -l ${snap_src_sz} \
+   $SCRATCH_MNT/src.clone2 $SCRATCH_MNT/src
+
+status=0 ; exit
diff --git a/tests/btrfs/035.out b/tests/btrfs/035.out
new file mode 100644
index 000..f86cadf
--- /dev/null
+++ b/tests/btrfs/035.out
@@ -0,0 +1,3 @@
+QA output created by 035
+attempting ioctl (src.clone1 src)
+attempting ioctl (src.clone2 src)
diff --git a/tests/btrfs/group b/tests/btrfs/group
index f9f062f..bee57cb 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -37,3 +37,4 @@
 032 auto quick
 033 auto quick
 034 auto quick
+035 auto quick
-- 
1.8.4.5

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 0/3] __btrfs_drop_extents() BUG_ON reproducer

2014-02-07 Thread David Disseldorp 
This patch-set provides a reproducer for hitting the 3.14.0-rc1 BUG_ON()
at:
 692 int __btrfs_drop_extents(struct btrfs_trans_handle *trans,
...
 839 /*
 840  *  |  range to drop - |
 841  *  |  extent  |
 842  */
 843 if (start = key.offset  end  extent_end) {
 844 BUG_ON(extent_type == BTRFS_FILE_EXTENT_INLINE);
 845 
 846 memcpy(new_key, key, sizeof(new_key));

The first patch adds a small cloner binary which is used by btrfs/035 to
dispatch BTRFS_IOC_CLONE_RANGE requests.

This workload resembles that of Samba's vfs_btrfs module, when a Windows
client restores a file from a shadow-copy (snapshot) using server-side
copy requests.

Changes since V2:
- Remove explicit write error checks

Changes since V1:
- Use strtoull instead of atoi
- Print error conditions in cloner
- Check for cloner binary before running test
- Continue test on failure
- Add cloner to .gitignore

Feedback appreciated.

Cheers, David


 .gitignore  |   1 +
 configure.ac|   1 +
 src/Makefile|   2 +-
 src/cloner.c| 192 
+++
 tests/btrfs/035 |  77 
+
 tests/btrfs/035.out |   3 +++
 tests/btrfs/group   |   1 +
 7 files changed, 276 insertions(+), 1 deletion(-)
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 1/3] btrfs: add small program for clone testing

2014-02-07 Thread David Disseldorp 
The cloner program is capable of cloning files using the BTRFS_IOC_CLONE
and BTRFS_IOC_CLONE_RANGE ioctls.

Signed-off-by: David Disseldorp dd...@suse.de
---
 .gitignore   |   1 +
 src/Makefile |   2 +-
 src/cloner.c | 188 +++
 3 files changed, 190 insertions(+), 1 deletion(-)
 create mode 100644 src/cloner.c

diff --git a/.gitignore b/.gitignore
index ee4cb41..b6f2463 100644
--- a/.gitignore
+++ b/.gitignore
@@ -104,6 +104,7 @@
 /src/aio-dio-regress/aio-free-ring-with-bogus-nr-pages
 /src/aio-dio-regress/aio-io-setup-with-nonwritable-context-pointer
 /src/aio-dio-regress/aiodio_sparse2
+/src/cloner
 
 # dmapi/ binaries
 /dmapi/src/common/cmd/read_invis
diff --git a/src/Makefile b/src/Makefile
index 84c8297..6509f2d 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -18,7 +18,7 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize 
preallo_rw_pattern_reader \
locktest unwritten_mmap bulkstat_unlink_test t_stripealign \
bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable \
stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test \
-   seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec
+   seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner
 
 SUBDIRS =
 
diff --git a/src/cloner.c b/src/cloner.c
new file mode 100644
index 000..dfce837
--- /dev/null
+++ b/src/cloner.c
@@ -0,0 +1,188 @@
+/*
+ *  Tiny program to perform file (range) clones using raw Btrfs ioctls.
+ *  It should only be needed until btrfs-progs has an xfs_io equivalent.
+ *
+ *  Copyright (C) 2014 SUSE Linux Products GmbH. All Rights Reserved.
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation; either version 2 of the License, or
+ *  (at your option) any later version.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with this program; if not, write to the Free Software
+ *  Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
+ */
+
+#include sys/types.h
+#include sys/stat.h
+#include sys/ioctl.h
+#include stdint.h
+#include stdbool.h
+#include fcntl.h
+#include unistd.h
+#include stdlib.h
+#include stdio.h
+#include string.h
+#include errno.h
+
+struct btrfs_ioctl_clone_range_args {
+   int64_t src_fd;
+   uint64_t src_offset;
+   uint64_t src_length;
+   uint64_t dest_offset;
+};
+
+#define BTRFS_IOCTL_MAGIC 0x94
+#define BTRFS_IOC_CLONE   _IOW(BTRFS_IOCTL_MAGIC, 9, int)
+#define BTRFS_IOC_CLONE_RANGE _IOW(BTRFS_IOCTL_MAGIC, 13, \
+  struct btrfs_ioctl_clone_range_args)
+
+static void
+usage(char *name, const char *msg)
+{
+   printf(Fatal: %s\n
+  Usage:\n
+  %s [options] src_file dest_file\n
+  \tA full file clone (reflink) is performed by default, 
+  unless any of the following are specified:\n
+  \t-s offset:  source file offset (default = 0)\n
+  \t-d offset:  destination file offset (default = 0)\n
+  \t-l length:  length of clone (default = 0)\n,
+  msg, name);
+   _exit(1);
+}
+
+static int
+clone_file(int src_fd, int dst_fd)
+{
+   int ret = ioctl(dst_fd, BTRFS_IOC_CLONE, src_fd);
+   if (ret != 0)
+   ret = errno;
+   return ret;
+}
+
+static int
+clone_file_range(int src_fd, int dst_fd, uint64_t src_off, uint64_t dst_off,
+uint64_t len)
+{
+   struct btrfs_ioctl_clone_range_args cr_args;
+   int ret;
+
+   memset(cr_args, 0, sizeof(cr_args));
+   cr_args.src_fd = src_fd;
+   cr_args.src_offset = src_off;
+   cr_args.src_length = len;
+   cr_args.dest_offset = dst_off;
+   ret = ioctl(dst_fd, BTRFS_IOC_CLONE_RANGE, cr_args);
+   if (ret != 0)
+   ret = errno;
+   return ret;
+}
+
+int
+main(int argc, char **argv)
+{
+   bool full_file = true;
+   uint64_t src_off = 0;
+   uint64_t dst_off = 0;
+   uint64_t len = 0;
+   char *src_file;
+   int src_fd;
+   char *dst_file;
+   int dst_fd;
+   int ret;
+   int opt;
+
+   while ((opt = getopt(argc, argv, s:d:l:)) != -1) {
+   char *sval_end;
+   switch (opt) {
+   case 's':
+   errno = 0;
+   src_off = strtoull(optarg, sval_end, 10);
+   if ((errno) || (*sval_end != '\0'))
+   usage(argv[0], invalid source offset);
+   full_file = false;
+   break

[PATCH v3 2/3] src/cloner: use btrfs/ioctl.h header if present

2014-02-07 Thread David Disseldorp 
Check for the btrfsprogs-devel ioctl.h header at configure time. Use it
in src/cloner if present, otherwise fall back to using the copied clone
ioctl definitions.

Signed-off-by: David Disseldorp dd...@suse.de
---
 configure.ac | 1 +
 src/cloner.c | 4 
 2 files changed, 5 insertions(+)

diff --git a/configure.ac b/configure.ac
index bd48fd9..6fba3ad 100644
--- a/configure.ac
+++ b/configure.ac
@@ -30,6 +30,7 @@ AC_HEADER_STDC
 AC_CHECK_HEADERS([ sys/fs/xfs_fsops.h  \
sys/fs/xfs_itable.h \
xfs/platform_defs.h \
+   btrfs/ioctl.h   \
 ])
 
 AC_PACKAGE_NEED_UUIDCOMPARE
diff --git a/src/cloner.c b/src/cloner.c
index dfce837..ccc2354 100644
--- a/src/cloner.c
+++ b/src/cloner.c
@@ -30,6 +30,9 @@
 #include stdio.h
 #include string.h
 #include errno.h
+#ifdef HAVE_BTRFS_IOCTL_H
+#include btrfs/ioctl.h
+#else
 
 struct btrfs_ioctl_clone_range_args {
int64_t src_fd;
@@ -42,6 +45,7 @@ struct btrfs_ioctl_clone_range_args {
 #define BTRFS_IOC_CLONE   _IOW(BTRFS_IOCTL_MAGIC, 9, int)
 #define BTRFS_IOC_CLONE_RANGE _IOW(BTRFS_IOCTL_MAGIC, 13, \
   struct btrfs_ioctl_clone_range_args)
+#endif
 
 static void
 usage(char *name, const char *msg)
-- 
1.8.4.5

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/2] btrfs: add small program for clone testing

2014-02-06 Thread David Disseldorp 
Hi Dave,

On Thu, 6 Feb 2014 10:09:36 +1100, Dave Chinner wrote:

 On Wed, Feb 05, 2014 at 12:16:48PM +0100, David Disseldorp wrote:
  The cloner program is capable of cloning files using the BTRFS_IOC_CLONE
  and BTRFS_IOC_CLONE_RANGE ioctls.
  
  Signed-off-by: David Disseldorp dd...@suse.de
 
 Hi Dave - long time since I've seen your head pop up around here ;)

Indeed, it's been a while. Thanks for the review :)

 
 A few comments below.
 
  +struct btrfs_ioctl_clone_range_args {
  +   int64_t src_fd;
  +   uint64_t src_offset;
  +   uint64_t src_length;
  +   uint64_t dest_offset;
  +};
  +
  +#define BTRFS_IOCTL_MAGIC 0x94
  +#define BTRFS_IOC_CLONE   _IOW(BTRFS_IOCTL_MAGIC, 9, int)
  +#define BTRFS_IOC_CLONE_RANGE _IOW(BTRFS_IOCTL_MAGIC, 13, \
  +  struct btrfs_ioctl_clone_range_args)
 
 Is there some published header file that these belong to? i.e.
 somewhere in the include/linux/uapi/ kernel directory? Normally the
 way to handle this sort of thing is by autoconf - if the header file
 exists, then we include it, otherwise we use the manual definitions.
 This just means that if the public api ever changes, we'll pick it
 up automatically in future...

I'd wanted to avoid the addition of another xfsqa prereq, but I guess
it'll work with the fall-back.
I'll add the autoconf logic to the next round, along with changes
addressing your other remarks.

Cheers, David
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 0/3] __btrfs_drop_extents() BUG_ON reproducer

2014-02-06 Thread David Disseldorp 
This patch-set provides a reproducer for hitting the 3.14.0-rc1 BUG_ON()
at:
 692 int __btrfs_drop_extents(struct btrfs_trans_handle *trans,
...
 839 /*
 840  *  |  range to drop - |
 841  *  |  extent  |
 842  */
 843 if (start = key.offset  end  extent_end) {
 844 BUG_ON(extent_type == BTRFS_FILE_EXTENT_INLINE);
 845 
 846 memcpy(new_key, key, sizeof(new_key));

The first patch adds a small cloner binary which is used by btrfs/035 to
dispatch BTRFS_IOC_CLONE_RANGE requests.

This workload resembles that of Samba's vfs_btrfs module, when a Windows
client restores a file from a shadow-copy (snapshot) using server-side
copy requests.

Changes since V1:
- Use strtoull instead of atoi
- Print error conditions in cloner
- Check for cloner binary before running test
- Continue test on failure
- Add cloner to .gitignore

Feedback appreciated.

Cheers, David


 .gitignore  |   1 +
 configure.ac|   1 +
 src/Makefile|   2 +-
 src/cloner.c| 192 
+++
 tests/btrfs/035 |  77 
+
 tests/btrfs/035.out |   3 +++
 tests/btrfs/group   |   1 +
 7 files changed, 276 insertions(+), 1 deletion(-)
 create mode 100644 src/cloner.c
 create mode 100755 tests/btrfs/035
 create mode 100644 tests/btrfs/035.out
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 2/3] src/cloner: use btrfs/ioctl.h header if present

2014-02-06 Thread David Disseldorp 
Check for the btrfsprogs-devel ioctl.h header at configure time. Use it
in src/cloner if present, otherwise fall back to using the copied clone
ioctl definitions.

Signed-off-by: David Disseldorp dd...@suse.de
---
 configure.ac | 1 +
 src/cloner.c | 4 
 2 files changed, 5 insertions(+)

diff --git a/configure.ac b/configure.ac
index bd48fd9..6fba3ad 100644
--- a/configure.ac
+++ b/configure.ac
@@ -30,6 +30,7 @@ AC_HEADER_STDC
 AC_CHECK_HEADERS([ sys/fs/xfs_fsops.h  \
sys/fs/xfs_itable.h \
xfs/platform_defs.h \
+   btrfs/ioctl.h   \
 ])
 
 AC_PACKAGE_NEED_UUIDCOMPARE
diff --git a/src/cloner.c b/src/cloner.c
index dfce837..ccc2354 100644
--- a/src/cloner.c
+++ b/src/cloner.c
@@ -30,6 +30,9 @@
 #include stdio.h
 #include string.h
 #include errno.h
+#ifdef HAVE_BTRFS_IOCTL_H
+#include btrfs/ioctl.h
+#else
 
 struct btrfs_ioctl_clone_range_args {
int64_t src_fd;
@@ -42,6 +45,7 @@ struct btrfs_ioctl_clone_range_args {
 #define BTRFS_IOC_CLONE   _IOW(BTRFS_IOCTL_MAGIC, 9, int)
 #define BTRFS_IOC_CLONE_RANGE _IOW(BTRFS_IOCTL_MAGIC, 13, \
   struct btrfs_ioctl_clone_range_args)
+#endif
 
 static void
 usage(char *name, const char *msg)
-- 
1.8.4.5

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 1/3] btrfs: add small program for clone testing

2014-02-06 Thread David Disseldorp 
The cloner program is capable of cloning files using the BTRFS_IOC_CLONE
and BTRFS_IOC_CLONE_RANGE ioctls.

Signed-off-by: David Disseldorp dd...@suse.de
---
 .gitignore   |   1 +
 src/Makefile |   2 +-
 src/cloner.c | 188 +++
 3 files changed, 190 insertions(+), 1 deletion(-)
 create mode 100644 src/cloner.c

diff --git a/.gitignore b/.gitignore
index ee4cb41..b6f2463 100644
--- a/.gitignore
+++ b/.gitignore
@@ -104,6 +104,7 @@
 /src/aio-dio-regress/aio-free-ring-with-bogus-nr-pages
 /src/aio-dio-regress/aio-io-setup-with-nonwritable-context-pointer
 /src/aio-dio-regress/aiodio_sparse2
+/src/cloner
 
 # dmapi/ binaries
 /dmapi/src/common/cmd/read_invis
diff --git a/src/Makefile b/src/Makefile
index 84c8297..6509f2d 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -18,7 +18,7 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize 
preallo_rw_pattern_reader \
locktest unwritten_mmap bulkstat_unlink_test t_stripealign \
bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable \
stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test \
-   seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec
+   seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner
 
 SUBDIRS =
 
diff --git a/src/cloner.c b/src/cloner.c
new file mode 100644
index 000..dfce837
--- /dev/null
+++ b/src/cloner.c
@@ -0,0 +1,188 @@
+/*
+ *  Tiny program to perform file (range) clones using raw Btrfs ioctls.
+ *  It should only be needed until btrfs-progs has an xfs_io equivalent.
+ *
+ *  Copyright (C) 2014 SUSE Linux Products GmbH. All Rights Reserved.
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation; either version 2 of the License, or
+ *  (at your option) any later version.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with this program; if not, write to the Free Software
+ *  Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
+ */
+
+#include sys/types.h
+#include sys/stat.h
+#include sys/ioctl.h
+#include stdint.h
+#include stdbool.h
+#include fcntl.h
+#include unistd.h
+#include stdlib.h
+#include stdio.h
+#include string.h
+#include errno.h
+
+struct btrfs_ioctl_clone_range_args {
+   int64_t src_fd;
+   uint64_t src_offset;
+   uint64_t src_length;
+   uint64_t dest_offset;
+};
+
+#define BTRFS_IOCTL_MAGIC 0x94
+#define BTRFS_IOC_CLONE   _IOW(BTRFS_IOCTL_MAGIC, 9, int)
+#define BTRFS_IOC_CLONE_RANGE _IOW(BTRFS_IOCTL_MAGIC, 13, \
+  struct btrfs_ioctl_clone_range_args)
+
+static void
+usage(char *name, const char *msg)
+{
+   printf(Fatal: %s\n
+  Usage:\n
+  %s [options] src_file dest_file\n
+  \tA full file clone (reflink) is performed by default, 
+  unless any of the following are specified:\n
+  \t-s offset:  source file offset (default = 0)\n
+  \t-d offset:  destination file offset (default = 0)\n
+  \t-l length:  length of clone (default = 0)\n,
+  msg, name);
+   _exit(1);
+}
+
+static int
+clone_file(int src_fd, int dst_fd)
+{
+   int ret = ioctl(dst_fd, BTRFS_IOC_CLONE, src_fd);
+   if (ret != 0)
+   ret = errno;
+   return ret;
+}
+
+static int
+clone_file_range(int src_fd, int dst_fd, uint64_t src_off, uint64_t dst_off,
+uint64_t len)
+{
+   struct btrfs_ioctl_clone_range_args cr_args;
+   int ret;
+
+   memset(cr_args, 0, sizeof(cr_args));
+   cr_args.src_fd = src_fd;
+   cr_args.src_offset = src_off;
+   cr_args.src_length = len;
+   cr_args.dest_offset = dst_off;
+   ret = ioctl(dst_fd, BTRFS_IOC_CLONE_RANGE, cr_args);
+   if (ret != 0)
+   ret = errno;
+   return ret;
+}
+
+int
+main(int argc, char **argv)
+{
+   bool full_file = true;
+   uint64_t src_off = 0;
+   uint64_t dst_off = 0;
+   uint64_t len = 0;
+   char *src_file;
+   int src_fd;
+   char *dst_file;
+   int dst_fd;
+   int ret;
+   int opt;
+
+   while ((opt = getopt(argc, argv, s:d:l:)) != -1) {
+   char *sval_end;
+   switch (opt) {
+   case 's':
+   errno = 0;
+   src_off = strtoull(optarg, sval_end, 10);
+   if ((errno) || (*sval_end != '\0'))
+   usage(argv[0], invalid source offset);
+   full_file = false;
+   break

[PATCH v2 3/3] btrfs/035: add new clone overwrite regression test

2014-02-06 Thread David Disseldorp 
This test uses the newly added cloner binary to dispatch full file and
range specific clone (reflink) requests.

Signed-off-by: David Disseldorp dd...@suse.de
---
 tests/btrfs/035 | 77 +
 tests/btrfs/035.out |  3 +++
 tests/btrfs/group   |  1 +
 3 files changed, 81 insertions(+)
 create mode 100755 tests/btrfs/035
 create mode 100644 tests/btrfs/035.out

diff --git a/tests/btrfs/035 b/tests/btrfs/035
new file mode 100755
index 000..49d9ece
--- /dev/null
+++ b/tests/btrfs/035
@@ -0,0 +1,77 @@
+#!/bin/bash
+# FS QA Test No. btrfs/035
+#
+# Regression test for overwriting clones
+#
+#---
+# Copyright (C) 2014 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#---
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo QA output created by $seq
+
+here=`pwd`
+tmp=/tmp/$$
+status=1   # failure is the default!
+
+_cleanup()
+{
+rm -f $tmp.*
+}
+
+trap _cleanup ; exit \$status 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs  /dev/null 21
+_scratch_mount
+
+CLONER_PROG=$here/src/cloner
+[ -x $CLONER_PROG ] || _notrun cloner binary not present at $CLONER_PROG
+
+src_str=aa
+
+echo -n $src_str  $SCRATCH_MNT/src || echo failed to create src
+
+$CLONER_PROG $SCRATCH_MNT/src  $SCRATCH_MNT/src.clone1
+
+src_str=bbcc
+
+echo -n $src_str  $SCRATCH_MNT/src || echo failed to create src
+
+$CLONER_PROG $SCRATCH_MNT/src $SCRATCH_MNT/src.clone2
+
+snap_src_sz=`ls -lah $SCRATCH_MNT/src.clone1 | awk '{print $5}'`
+echo attempting ioctl (src.clone1 src)
+$CLONER_PROG -s 0 -d 0 -l ${snap_src_sz} \
+   $SCRATCH_MNT/src.clone1 $SCRATCH_MNT/src
+
+snap_src_sz=`ls -lah $SCRATCH_MNT/src.clone2 | awk '{print $5}'`
+echo attempting ioctl (src.clone2 src)
+$CLONER_PROG -s 0 -d 0 -l ${snap_src_sz} \
+   $SCRATCH_MNT/src.clone2 $SCRATCH_MNT/src
+
+status=0 ; exit
diff --git a/tests/btrfs/035.out b/tests/btrfs/035.out
new file mode 100644
index 000..f86cadf
--- /dev/null
+++ b/tests/btrfs/035.out
@@ -0,0 +1,3 @@
+QA output created by 035
+attempting ioctl (src.clone1 src)
+attempting ioctl (src.clone2 src)
diff --git a/tests/btrfs/group b/tests/btrfs/group
index f9f062f..bee57cb 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -37,3 +37,4 @@
 032 auto quick
 033 auto quick
 034 auto quick
+035 auto quick
-- 
1.8.4.5

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] ioctl: add note regarding CLONE_RANGE(len=0) behaviour

2014-02-06 Thread David Disseldorp 
A BTRFS_IOC_CLONE_RANGE request with a src_length value of zero has the
effect of cloning all data from src_offset through to end-of-file.

Document this behaviour in the header file for those who (like me)
incorrectly assume that no data is cloned in such a case.

Signed-off-by: David Disseldorp dd...@suse.de
---
 ioctl.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/ioctl.h b/ioctl.h
index a589cd7..f7d435d 100644
--- a/ioctl.h
+++ b/ioctl.h
@@ -506,6 +506,7 @@ static inline char *btrfs_err_str(enum btrfs_err_code 
err_code)
 #define BTRFS_IOC_SCAN_DEV _IOW(BTRFS_IOCTL_MAGIC, 4, \
   struct btrfs_ioctl_vol_args)
 
+/* With a @src_length of zero, the range from @src_offset-EOF is cloned! */
 struct btrfs_ioctl_clone_range_args {
__s64 src_fd;
__u64 src_offset, src_length;
-- 
1.8.4.5

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/2] btrfs/035: add new clone overwrite regression test

2014-02-05 Thread David Disseldorp 
This test uses the newly added cloner binary to dispatch full file and
range specific clone (reflink) requests.

Signed-off-by: David Disseldorp dd...@suse.de
---
 tests/btrfs/035 | 76 +
 tests/btrfs/035.out |  3 +++
 tests/btrfs/group   |  1 +
 3 files changed, 80 insertions(+)
 create mode 100755 tests/btrfs/035
 create mode 100644 tests/btrfs/035.out

diff --git a/tests/btrfs/035 b/tests/btrfs/035
new file mode 100755
index 000..03c2cd3
--- /dev/null
+++ b/tests/btrfs/035
@@ -0,0 +1,76 @@
+#!/bin/bash
+# FS QA Test No. btrfs/035
+#
+# Regression test for overwriting clones
+#
+#---
+# Copyright (C) 2014 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#---
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo QA output created by $seq
+
+here=`pwd`
+tmp=/tmp/$$
+status=1   # failure is the default!
+
+_cleanup()
+{
+rm -f $tmp.*
+}
+
+trap _cleanup ; exit \$status 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs  /dev/null 21
+_scratch_mount
+
+CLONER_PROG=$here/src/cloner
+
+src_str=aa
+
+echo -n $src_str  $SCRATCH_MNT/src || _fail failed to create src
+
+$CLONER_PROG $SCRATCH_MNT/src  $SCRATCH_MNT/src.clone1
+
+src_str=bbcc
+
+echo -n $src_str  $SCRATCH_MNT/src || _fail failed to create src
+
+$CLONER_PROG $SCRATCH_MNT/src $SCRATCH_MNT/src.clone2
+
+snap_src_sz=`ls -lah $SCRATCH_MNT/src.clone1 | awk '{print $5}'`
+echo attempting ioctl (src.clone1 src)
+$CLONER_PROG -s 0 -d 0 -l ${snap_src_sz} \
+   $SCRATCH_MNT/src.clone1 $SCRATCH_MNT/src || _fail ioctl failed
+
+snap_src_sz=`ls -lah $SCRATCH_MNT/src.clone2 | awk '{print $5}'`
+echo attempting ioctl (src.clone2 src)
+$CLONER_PROG -s 0 -d 0 -l ${snap_src_sz} \
+   $SCRATCH_MNT/src.clone2 $SCRATCH_MNT/src || _fail ioctl failed
+
+status=0 ; exit
diff --git a/tests/btrfs/035.out b/tests/btrfs/035.out
new file mode 100644
index 000..f86cadf
--- /dev/null
+++ b/tests/btrfs/035.out
@@ -0,0 +1,3 @@
+QA output created by 035
+attempting ioctl (src.clone1 src)
+attempting ioctl (src.clone2 src)
diff --git a/tests/btrfs/group b/tests/btrfs/group
index f9f062f..bee57cb 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -37,3 +37,4 @@
 032 auto quick
 033 auto quick
 034 auto quick
+035 auto quick
-- 
1.8.4.5

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 0/2] __btrfs_drop_extents() BUG_ON reproducer

2014-02-05 Thread David Disseldorp 
This patch-set provides a reproducer for hitting the 3.14.0-rc1 BUG_ON()
at:
 692 int __btrfs_drop_extents(struct btrfs_trans_handle *trans,
...
 839 /*
 840  *  |  range to drop - |
 841  *  |  extent  |
 842  */
 843 if (start = key.offset  end  extent_end) {
 844 BUG_ON(extent_type == BTRFS_FILE_EXTENT_INLINE);
 845 
 846 memcpy(new_key, key, sizeof(new_key));

The first patch adds a small cloner binary which is used by btrfs/035 to
dispatch BTRFS_IOC_CLONE_RANGE requests.

This workload resembles that of Samba's vfs_btrfs module, when a Windows
client restores a file from a shadow-copy (snapshot) using server-side
copy requests.

Feedback appreciated.

Cheers, David


 src/Makefile|   2 +-
 src/cloner.c| 168 
+++
 tests/btrfs/035 |  76 

 tests/btrfs/035.out |   3 +++
 tests/btrfs/group   |   1 +
 5 files changed, 249 insertions(+), 1 deletion(-)

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/2] btrfs: add small program for clone testing

2014-02-05 Thread David Disseldorp 
The cloner program is capable of cloning files using the BTRFS_IOC_CLONE
and BTRFS_IOC_CLONE_RANGE ioctls.

Signed-off-by: David Disseldorp dd...@suse.de
---
 src/Makefile |   2 +-
 src/cloner.c | 168 +++
 2 files changed, 169 insertions(+), 1 deletion(-)
 create mode 100644 src/cloner.c

diff --git a/src/Makefile b/src/Makefile
index 84c8297..6509f2d 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -18,7 +18,7 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize 
preallo_rw_pattern_reader \
locktest unwritten_mmap bulkstat_unlink_test t_stripealign \
bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable \
stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test \
-   seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec
+   seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner
 
 SUBDIRS =
 
diff --git a/src/cloner.c b/src/cloner.c
new file mode 100644
index 000..59defbb
--- /dev/null
+++ b/src/cloner.c
@@ -0,0 +1,168 @@
+/*
+ *  Tiny program to perform file (range) clones using raw Btrfs ioctls.
+ *  It should only be needed until btrfs-progs has an xfs_io equivalent.
+ *
+ *  Copyright (C) 2014 SUSE Linux Products GmbH. All Rights Reserved.
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation; either version 2 of the License, or
+ *  (at your option) any later version.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with this program; if not, write to the Free Software
+ *  Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
+ */
+
+#include sys/types.h
+#include sys/stat.h
+#include sys/ioctl.h
+#include stdint.h
+#include stdbool.h
+#include fcntl.h
+#include unistd.h
+#include stdlib.h
+#include stdio.h
+#include string.h
+#include errno.h
+
+struct btrfs_ioctl_clone_range_args {
+   int64_t src_fd;
+   uint64_t src_offset;
+   uint64_t src_length;
+   uint64_t dest_offset;
+};
+
+#define BTRFS_IOCTL_MAGIC 0x94
+#define BTRFS_IOC_CLONE   _IOW(BTRFS_IOCTL_MAGIC, 9, int)
+#define BTRFS_IOC_CLONE_RANGE _IOW(BTRFS_IOCTL_MAGIC, 13, \
+  struct btrfs_ioctl_clone_range_args)
+
+static void
+usage(char *name, const char *msg)
+{
+   printf(Fatal: %s\n
+  Usage:\n
+  %s [options] src_file dest_file\n
+  \tA full file clone (reflink) is performed by default, 
+  unless any of the following are specified:\n
+  \t-s offset:  source file offset (default = 0)\n
+  \t-d offset:  destination file offset (default = 0)\n
+  \t-l length:  length of clone (default = 0)\n,
+  msg, name);
+   _exit(1);
+}
+
+static int
+clone_file(int src_fd, int dst_fd)
+{
+   int ret = ioctl(dst_fd, BTRFS_IOC_CLONE, src_fd);
+   if (ret != 0)
+   ret = errno;
+   return ret;
+}
+
+static int
+clone_file_range(int src_fd, int dst_fd, uint64_t src_off, uint64_t dst_off,
+uint64_t len)
+{
+   struct btrfs_ioctl_clone_range_args cr_args;
+   int ret;
+
+   memset(cr_args, 0, sizeof(cr_args));
+   cr_args.src_fd = src_fd;
+   cr_args.src_offset = src_off;
+   cr_args.src_length = len;
+   cr_args.dest_offset = dst_off;
+   ret = ioctl(dst_fd, BTRFS_IOC_CLONE_RANGE, cr_args);
+   if (ret != 0)
+   ret = errno;
+   return ret;
+}
+
+int
+main(int argc, char **argv)
+{
+   bool full_file = true;
+   uint64_t src_off = 0;
+   uint64_t dst_off = 0;
+   uint64_t len = 0;
+   char *src_file;
+   int src_fd;
+   char *dst_file;
+   int dst_fd;
+   int ret;
+   int opt;
+
+   while ((opt = getopt(argc, argv, s:d:l:)) != -1) {
+   switch (opt) {
+   case 's':
+   src_off = atoi(optarg);
+   full_file = false;
+   break;
+   case 'd':
+   dst_off = atoi(optarg);
+   full_file = false;
+   break;
+   case 'l':
+   len = atoi(optarg);
+   full_file = false;
+   break;
+   default:
+   usage(argv[0], invalid argument);
+   }
+   }
+
+   /* should be exactly two args left */
+   if (optind != argc - 2)
+   usage(argv[0], src_file and dst_file arguments are madatory);
+
+   src_file = (char *)strdup(argv