[PATCH] generic/175: disable inline data feature for btrfs

[Wang Xiaoguang]

For btrfs, if compression is enabled, it may generate inline data for a
blocksize data range, this inline data is stored in fs tree, will not have
a individual extent, try to reflink this data range at a not-zero offset
will return EOPNOTSUPP, so here we disable inline data feature for btrfs.

Signed-off-by: Wang Xiaoguang 
---
 tests/generic/175 | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/tests/generic/175 b/tests/generic/175
index 964580c..b3f90dc 100755
--- a/tests/generic/175
+++ b/tests/generic/175
@@ -50,6 +50,13 @@ rm -f "$seqres.full"
 
 echo "Format and mount"
 _scratch_mkfs > "$seqres.full" 2>&1
+# For btrfs, if compression is enabled, it may generate inline data for a
+# blocksize data range, this inline data is stored in fs tree, will not have
+# a individual extent, try to reflink this data range at a not-zero offset
+# will return EOPNOTSUPP, so here we disable inline data feature for btrfs.
+if [ "$FSTYP" = "btrfs" ]; then
+   export MOUNT_OPTIONS="-o max_inline=0 $MOUNT_OPTIONS"
+fi
 _scratch_mount >> "$seqres.full" 2>&1
 
 testdir="$SCRATCH_MNT/test-$seq"
-- 
2.9.0



--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] generic: make 17[1-4] work well when btrfs compression is enabled

[Darrick J. Wong]

On Mon, Oct 10, 2016 at 11:49:03AM +0800, Wang Xiaoguang wrote:
> hi,
> 
> On 10/10/2016 05:04 AM, Darrick J. Wong wrote:
> >On Sat, Oct 08, 2016 at 01:36:10AM +1100, Dave Chinner wrote:
> >>On Fri, Oct 07, 2016 at 03:00:42PM +0800, Wang Xiaoguang wrote:
> >>>When enabling btrfs compression, original codes can not fill fs
> >>>correctly, fix this.
> >>>
> >>>Signed-off-by: Wang Xiaoguang 
> >>>---
> >>>  tests/generic/171 | 4 +---
> >>>  tests/generic/172 | 2 +-
> >>>  tests/generic/173 | 4 +---
> >>>  tests/generic/174 | 4 +---
> >>>  4 files changed, 4 insertions(+), 10 deletions(-)
> >>>
> >>>diff --git a/tests/generic/171 b/tests/generic/171
> >>>index f391685..d2ae8e4 100755
> >>>--- a/tests/generic/171
> >>>+++ b/tests/generic/171
> >>>@@ -75,9 +75,7 @@ _cp_reflink $testdir/bigfile $testdir/clonefile
> >>>  sync
> >>>  echo "Allocate the rest of the space"
> >>>-nr_free=$(stat -f -c '%f' $testdir)
> >>>-touch $testdir/file0 $testdir/file1
> >Why remove this line which ensures that the inode we're going to use (file1)
> >later in the test has been allocated /before/ we try to eat all the space?
> Sorry, I don't understand :)
> In generic/171, generic/172, generic/173, generic/174, you created file0 and
> file1, but I don't see test cases use them and they are empty.
> 
> >
> >>>-_pwrite_byte 0x61 0 $((blksz * nr_free)) $testdir/eat_my_space >> 
> >>>$seqres.full 2>&1
> >>>+dd if=/dev/zero of=$testdir/eat_my_space >> $seqres.full 2>&1
> >Uh... isn't a bunch of zeroes just as compressible as 0x61?
> 0x61 is also compressible.
> >I suppose
> >the point here is to write until write returns ENOSPC, since in btrfs
> >land we can't assume that writing $nr blocks will use up at least that
> >much space.
> Yes.

Ok, just making sure I figured it out correctly. :)

> >
> >Does XFS reflink still pass this test with this patch?
> Could you please give a stable xfs repotory that supports xfs reflink,
> thanks.

The for-next branch of Dave Chinner's kernel tree:
https://git.kernel.org/cgit/linux/kernel/git/dgc/linux-xfs.git/?h=for-next

And userspace programs in my repo:
https://github.com/djwong/xfsprogs/tree/for-dave-for-4.9-10

(I wouldn't call them stable, but they are what's submitted for 4.9.)

> >>Please don't replace xfs_io writes using a specific data pattern
> >>with dd calls that write zeros. Indeed, we don't use dd for new
> >>tests anymore - xfs_io should be used.
> >>
> >>Write a function that fills all the remaining free space (one may
> >>already exist) and call it in all these places.
> >Yeah, there already are a few of these...
> OK, I'll use it, thanks.

No problem! :)

--D

> 
> Regards,
> Xiaoguang Wang
> >
> >--D
> >
> >>Cheers,
> >>
> >>Dave.
> >>-- 
> >>Dave Chinner
> >>da...@fromorbit.com
> >>--
> >>To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> >>the body of a message to majord...@vger.kernel.org
> >>More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >
> 
> 
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] generic: make 17[1-4] work well when btrfs compression is enabled

[Wang Xiaoguang]

hi,

On 10/10/2016 05:04 AM, Darrick J. Wong wrote:

On Sat, Oct 08, 2016 at 01:36:10AM +1100, Dave Chinner wrote:

On Fri, Oct 07, 2016 at 03:00:42PM +0800, Wang Xiaoguang wrote:

When enabling btrfs compression, original codes can not fill fs
correctly, fix this.

Signed-off-by: Wang Xiaoguang 
---
  tests/generic/171 | 4 +---
  tests/generic/172 | 2 +-
  tests/generic/173 | 4 +---
  tests/generic/174 | 4 +---
  4 files changed, 4 insertions(+), 10 deletions(-)

diff --git a/tests/generic/171 b/tests/generic/171
index f391685..d2ae8e4 100755
--- a/tests/generic/171
+++ b/tests/generic/171
@@ -75,9 +75,7 @@ _cp_reflink $testdir/bigfile $testdir/clonefile
  sync
  
  echo "Allocate the rest of the space"

-nr_free=$(stat -f -c '%f' $testdir)
-touch $testdir/file0 $testdir/file1

Why remove this line which ensures that the inode we're going to use (file1)
later in the test has been allocated /before/ we try to eat all the space?

Sorry, I don't understand :)
In generic/171, generic/172, generic/173, generic/174, you created file0 and
file1, but I don't see test cases use them and they are empty.




-_pwrite_byte 0x61 0 $((blksz * nr_free)) $testdir/eat_my_space >> $seqres.full 
2>&1
+dd if=/dev/zero of=$testdir/eat_my_space >> $seqres.full 2>&1

Uh... isn't a bunch of zeroes just as compressible as 0x61?

0x61 is also compressible.

I suppose
the point here is to write until write returns ENOSPC, since in btrfs
land we can't assume that writing $nr blocks will use up at least that
much space.

Yes.



Does XFS reflink still pass this test with this patch?
Could you please give a stable xfs repotory that supports xfs reflink, 
thanks.





Please don't replace xfs_io writes using a specific data pattern
with dd calls that write zeros. Indeed, we don't use dd for new
tests anymore - xfs_io should be used.

Write a function that fills all the remaining free space (one may
already exist) and call it in all these places.

Yeah, there already are a few of these...

OK, I'll use it, thanks.

Regards,
Xiaoguang Wang


--D


Cheers,

Dave.
--
Dave Chinner
da...@fromorbit.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html






--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: btrfs_direct_IO oops

[Al Viro]

On Sun, Oct 09, 2016 at 11:11:06AM -0400, Dave Jones wrote:
> idx = 0, offset = 0

== starts at slot 0, nothing put into it yet, but

> curbuf = 0, nrbufs = 1, buffers = 16
> [9fa21100 ea00065f6d80 0 4096]

the underlying pipe has something stored into slot 0.  That definitely
should not happen.

> WARNING: CPU: 0 PID: 29610 at lib/iov_iter.c:327 sanity+0x102/0x150
> CPU: 0 PID: 29610 Comm: trinity-c9 Not tainted 4.8.0-think+ #8 
>  c97b7ae8
>  9f3e2011
>  
>  

>  9fc1e22b
>  9f3fa2f2
>  c97b7b28
>  9f08b010
> 
>  014734c3d60f
>  9fc1e22b
>  0147
>  0010

*blink*

where have those come from?
 
> Call Trace:
>  [] dump_stack+0x6c/0x9b
>  [] ? sanity+0x102/0x150
>  [] __warn+0x110/0x130
>  [] warn_slowpath_null+0x2c/0x40
>  [] sanity+0x102/0x150
>  [] copy_page_to_iter+0x2be/0x480
>  [] ? pagecache_get_page+0xba/0x4f0
>  [] generic_file_read_iter+0x245/0xd30
>  [] generic_file_splice_read+0xfd/0x230
>  [] ? pipe_to_user+0x40/0x40
>  [] do_splice_to+0x98/0xd0
>  [] splice_direct_to_actor+0xd4/0x2c0
>  [] ? generic_pipe_buf_nosteal+0x10/0x10
>  [] do_splice_direct+0xc5/0x110
>  [] do_sendfile+0x242/0x470
>  [] SyS_sendfile64+0x7d/0xf0
>  [] do_syscall_64+0x7f/0x200
>  [] entry_SYSCALL64_slow_path+0x25/0x25
> ---[ end trace 2c7bd411d4aa0491 ]---

Very interesting.  Could you slap something like
diff --git a/lib/iov_iter.c b/lib/iov_iter.c
index 0ce3411..1ef00e7 100644
--- a/lib/iov_iter.c
+++ b/lib/iov_iter.c
@@ -682,8 +682,9 @@ static void pipe_advance(struct iov_iter *i, size_t size)
 {
struct pipe_inode_info *pipe = i->pipe;
struct pipe_buffer *buf;
-   int idx = i->idx;
-   size_t off = i->iov_offset;
+   int old_idx = idx = i->idx;
+   size_t old_off = off = i->iov_offset;
+   size_t old_size = size;

if (unlikely(i->count < size))
size = i->count;
@@ -713,6 +714,9 @@ static void pipe_advance(struct iov_iter *i, size_t size)
pipe->nrbufs--;
}
}
+   if (!sanity(i))
+   printk(KERN_ERR "buggered pipe_advance(%zd) [%d,%zd]",
+   old_size, old_idx, old_off);
 }
 
 void iov_iter_advance(struct iov_iter *i, size_t size)

in there and try to reproduce that one?
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/4] btrfs-progs: Make btrfs-debug-tree print all readable strings for inode flags

[Qu Wenruo]

Before this patch, only 3 inode flags have readable string: NODATACOW,
NODATASUM, READONLY.

This patch will output all readable strings for remaining inode flags,
making debug-tree tool more handy.

Signed-off-by: Qu Wenruo 
---
 print-tree.c | 46 ++
 1 file changed, 26 insertions(+), 20 deletions(-)

diff --git a/print-tree.c b/print-tree.c
index a14..f015646 100644
--- a/print-tree.c
+++ b/print-tree.c
@@ -851,29 +851,35 @@ static void print_uuid_item(struct extent_buffer *l, 
unsigned long offset,
}
 }
 
-/* Caller should ensure sizeof(*ret) >= 29 "NODATASUM|NODATACOW|READONLY" */
+#define copy_one_inode_flag(flags, name, empty, dst) ({
\
+   if (flags & BTRFS_INODE_##name) {   \
+   if (!empty) \
+   strcat(dst, "|");   \
+   strcat(dst, #name); \
+   empty = 0;  \
+   }   \
+})
+
+/*
+ * Caller should ensure sizeof(*ret) >= 102: all charactors plus '|' of
+ * BTRFS_INODE_* flags
+ */
 static void inode_flags_to_str(u64 flags, char *ret)
 {
int empty = 1;
 
-   if (flags & BTRFS_INODE_NODATASUM) {
-   empty = 0;
-   strcpy(ret, "NODATASUM");
-   }
-   if (flags & BTRFS_INODE_NODATACOW) {
-   if (!empty) {
-   empty = 0;
-   strcat(ret, "|");
-   }
-   strcat(ret, "NODATACOW");
-   }
-   if (flags & BTRFS_INODE_READONLY) {
-   if (!empty) {
-   empty = 0;
-   strcat(ret, "|");
-   }
-   strcat(ret, "READONLY");
-   }
+   copy_one_inode_flag(flags, NODATASUM, empty, ret);
+   copy_one_inode_flag(flags, NODATACOW, empty, ret);
+   copy_one_inode_flag(flags, READONLY, empty, ret);
+   copy_one_inode_flag(flags, NOCOMPRESS, empty, ret);
+   copy_one_inode_flag(flags, PREALLOC, empty, ret);
+   copy_one_inode_flag(flags, SYNC, empty, ret);
+   copy_one_inode_flag(flags, IMMUTABLE, empty, ret);
+   copy_one_inode_flag(flags, APPEND, empty, ret);
+   copy_one_inode_flag(flags, NODUMP, empty, ret);
+   copy_one_inode_flag(flags, NOATIME, empty, ret);
+   copy_one_inode_flag(flags, DIRSYNC, empty, ret);
+   copy_one_inode_flag(flags, COMPRESS, empty, ret);
if (empty)
strcat(ret, "none");
 }
@@ -902,7 +908,7 @@ void btrfs_print_leaf(struct btrfs_root *root, struct 
extent_buffer *l)
u32 nr = btrfs_header_nritems(l);
u64 objectid;
u32 type;
-   char flags_str[32];
+   char flags_str[128];
 
printf("leaf %llu items %d free space %d generation %llu owner %llu\n",
(unsigned long long)btrfs_header_bytenr(l), nr,
-- 
2.10.0



--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 4/4] btrfs-progs: convert-test: Add test case for common inode flags

[Qu Wenruo]

Add a new test case to check if btrfs-convert copies common inode flags
like append(a), immutable(i).

Signed-off-by: Qu Wenruo 
---
 tests/convert-tests/009-common-inode-flags/test.sh | 34 ++
 1 file changed, 34 insertions(+)
 create mode 100755 tests/convert-tests/009-common-inode-flags/test.sh

diff --git a/tests/convert-tests/009-common-inode-flags/test.sh 
b/tests/convert-tests/009-common-inode-flags/test.sh
new file mode 100755
index 000..8bbac22
--- /dev/null
+++ b/tests/convert-tests/009-common-inode-flags/test.sh
@@ -0,0 +1,34 @@
+#!/bin/bash
+# Check if btrfs-convert can copy common inode flags like SYNC/IMMUTABLE
+
+source $TOP/tests/common
+source $TOP/tests/common.convert
+
+setup_root_helper
+prepare_test_dev 512M
+check_prereq btrfs-convert
+
+fail=0
+default_mke2fs="mke2fs -t ext4 -b 4096"
+convert_test_preamble '' 'common inode flags test' 16k "$default_mke2fs"
+convert_test_prep_fs $default_mke2fs
+
+# create file with specific flags
+$SUDO_HELPER run_check touch $TEST_MNT/flag_test
+$SUDO_HELPER run_check chattr +aSidA $TEST_MNT/flag_test
+
+run_check_umount_test_dev
+convert_test_do_convert
+run_check_mount_test_dev
+
+# Above flags should be copied to btrfs flags, and lsattr should get them
+run_check_stdout lsattr $TEST_MNT/flag_test | cut -f1 -d\  > tmp_output
+grep -e "S" -e "i" -e "a" -e "d" -e "A" -q tmp_output
+if [ $? -ne 0 ]; then
+   rm tmp_output
+   _fail "no common inode flags are copied after convert"
+fi
+rm tmp_output
+
+run_check_umount_test_dev
+convert_test_post_rollback
-- 
2.10.0



--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 0/4] Btrfs-convert: Add support to copy common inode flags

[Qu Wenruo]

Branch can be fetched from:
https://github.com/adam900710/btrfs-progs/tree/convert_inode_flags

Thanks for the report from Lakshmipathi.G, we exposed a bug that
btrfs-convert never copies common inode flags like SYNC/IMMUTABLE/APPEND.

The root cause is quite awkward, we didn't even have these flags defined
in ctree.h.

This patchset will copy related flags to btrfs-progs header and make
btrfs-convert to copy these flags.(Only some ext2 flags are support)
And enhance btrfs-debug-tree to handle these flags.

Finally, adds test case to prevent such problem happens again.

Qu Wenruo (4):
  btrfs-progs: Copy btrfs inode flags from kernel header
  btrfs-progs: Make btrfs-debug-tree print all readable strings for
inode flags
  btrfs-progs: convert: Convert ext inode flags to btrfs inode flags
  btrfs-progs: convert-test: Add test case for common inode flags

 btrfs-convert.c| 25 
 ctree.h|  9 +
 print-tree.c   | 46 --
 tests/convert-tests/009-common-inode-flags/test.sh | 34 
 4 files changed, 94 insertions(+), 20 deletions(-)
 create mode 100755 tests/convert-tests/009-common-inode-flags/test.sh

-- 
2.10.0



--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 3/4] btrfs-progs: convert: Convert ext inode flags to btrfs inode flags

[Qu Wenruo]

Before this patch, btrfs-convert never copy ext* inode flags to
corresponding btrfs inode flags.

This makes common flags like APPEND/SYNC/SYNCDIR/IMMUTABLE not copied to
btrfs inode.

This patch introduces ext2_convert_inode_flags() function to handle the
convert, so btrfs-convert can copy as many inode flags as possible.

Reported-by: Lakshmipathi.G 
Signed-off-by: Qu Wenruo 
---
 btrfs-convert.c | 25 +
 1 file changed, 25 insertions(+)

diff --git a/btrfs-convert.c b/btrfs-convert.c
index 542179e..160a635 100644
--- a/btrfs-convert.c
+++ b/btrfs-convert.c
@@ -2201,6 +2201,30 @@ static int ext2_check_state(struct btrfs_convert_context 
*cctx)
return 0;
 }
 
+#define copy_one_ext2_flag(flags, ext2_inode, name) ({ \
+   if (ext2_inode->i_flags & EXT2_##name##_FL) \
+   flags |= BTRFS_INODE_##name;\
+})
+
+/*
+ * Convert EXT2_*_FL to corresponding BTRFS_INODE_* flags
+ *
+ * Only a subset of EXT_*_FL is supported in btrfs.
+ */
+static void ext2_convert_inode_flags(struct btrfs_inode_item *dst,
+struct ext2_inode *src)
+{
+   u64 flags = 0;
+
+   copy_one_ext2_flag(flags, src, APPEND);
+   copy_one_ext2_flag(flags, src, SYNC);
+   copy_one_ext2_flag(flags, src, IMMUTABLE);
+   copy_one_ext2_flag(flags, src, NODUMP);
+   copy_one_ext2_flag(flags, src, NOATIME);
+   copy_one_ext2_flag(flags, src, DIRSYNC);
+   btrfs_set_stack_inode_flags(dst, flags);
+}
+
 /*
  * copy a single inode. do all the required works, such as cloning
  * inode item, creating file extents and creating directory entries.
@@ -2223,6 +2247,7 @@ static int ext2_copy_single_inode(struct 
btrfs_trans_handle *trans,
BTRFS_INODE_NODATASUM;
btrfs_set_stack_inode_flags(_inode, flags);
}
+   ext2_convert_inode_flags(_inode, ext2_inode);
 
switch (ext2_inode->i_mode & S_IFMT) {
case S_IFREG:
-- 
2.10.0



--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/4] btrfs-progs: Copy btrfs inode flags from kernel header

[Qu Wenruo]

Btrfs-progs header lacks quite a lot inode flags.
Copy them from kernel for later use.

Signed-off-by: Qu Wenruo 
---
 ctree.h | 9 +
 1 file changed, 9 insertions(+)

diff --git a/ctree.h b/ctree.h
index e0dd2e4..c76b1f1 100644
--- a/ctree.h
+++ b/ctree.h
@@ -1246,6 +1246,15 @@ struct btrfs_root {
 #define BTRFS_INODE_NODATASUM  (1 << 0)
 #define BTRFS_INODE_NODATACOW  (1 << 1)
 #define BTRFS_INODE_READONLY   (1 << 2)
+#define BTRFS_INODE_NOCOMPRESS (1 << 3)
+#define BTRFS_INODE_PREALLOC   (1 << 4)
+#define BTRFS_INODE_SYNC   (1 << 5)
+#define BTRFS_INODE_IMMUTABLE  (1 << 6)
+#define BTRFS_INODE_APPEND (1 << 7)
+#define BTRFS_INODE_NODUMP (1 << 8)
+#define BTRFS_INODE_NOATIME(1 << 9)
+#define BTRFS_INODE_DIRSYNC(1 << 10)
+#define BTRFS_INODE_COMPRESS   (1 << 11)
 
 #define read_eb_member(eb, ptr, type, member, result) (
\
read_extent_buffer(eb, (char *)(result),\
-- 
2.10.0



--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: raid levels and NAS drives

[Tomasz Kusmierz]

On 10 October 2016 at 02:01, ronnie sahlberg  wrote:
> (without html this time.)
>
> Nas drives are more expensive but also more durable than the normal consumer
> drives, but not as durable as enterprise drives.
> They are meant for near continous use, compared to consumer/backup drives
> that are meant for only occasional use and meant to spend the majority of
> time spinned down.
>
>
> They fall in-between consumer and enterprise gear.
>

Again, you read a marketing flyer ...

Historically enterprise drives did equal to a drive with SCSI, after
that it started to equal to a drive with more exotic interfaces like
SAS or FATA ... nowadays this means more in line "high [seak]
performance, for which you pay extra extra extra buck" (10k, 15k
arrays of 10 disks with databases on it that are serving plenty of
people ?).
Currently, customer = low end drive where you will not pay twice the
price for 10% performance increase.

There is nothing there about reliability !!!
Now every [sane] storage engineer will chose a "customer" 5.4k drives
for a cold storage / slow IO storage. In high demand, very random seek
patterns everybody will go for extreme fast disk that will die in 12
months, because cost * effort or replacing a failed disk is still less
than assembling a like array from 7.2k disk (extra controller, extra
bays, extra power, extra everything !).

So:
1.
Stop reading a marketing material that is designed to suck money out
of you pocket. Read technical datasheet.
Stop reading a paid for articles from so called "specialists", my
company pays those people to put in articles that I write to sound
more technical so I can tell you how much "horse" those are.

2.
hdd:
faster rpm = better seek + better sequential read write
slower rpm = survives longer + takes less power + better $ per GB

3.
what you need to use it for:
a remote nas box ? a single 5.1k hdd will saturate your gigabit lan, 7
will saturate your SFP+ - go for best $ per GB
local storage ? 4 x 7.1k hdd in raid10 and you're talking a good
performance ! putting more disks in and you can drop down to 5.1k
a high demand database with thousands of people punching milions of
queries a second ? 15k as many as you can!

4.
For time being on btrfs give raid 5 & 6 a wide berth for time being
... unless you back up your data [very] regularly than, have fun :)
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: raid levels and NAS drives

[Tomasz Kusmierz]

And what exactly are NAS drives ?

Are you talking marketing by any chance ? Please, tell me you got the pun.

On 10 October 2016 at 00:12, Charles Zeitler  wrote:
> Is there any advantage to using NAS drives
> under RAID levels,  as oppposed to regular
> 'desktop' drives for BTRFS?
>
> Charles Zeitler
>
> --
>  The Perfect Is The Enemy Of
>  The Good Enough
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Is is possible to submit binary image as fstest test case?

[Dave Chinner]

On Fri, Oct 07, 2016 at 06:05:51PM +0200, David Sterba wrote:
> On Fri, Oct 07, 2016 at 08:18:38PM +1100, Dave Chinner wrote:
> > On Thu, Oct 06, 2016 at 04:12:56PM +0800, Qu Wenruo wrote:
> > > So I'm wondering if I can just upload a zipped raw image as part of
> > > the test case?
> > 
> > Preferably not. We've managed to avoid pre-built images in xfstests
> > for 15 years, so there'd have to be a really good reason to start
> > doing this, especially as once we open that floodgate we'll end up
> > with everyone wanting to do this and it will blow out the size of
> > the repository in now time.
> > 
> > If the issue is just timing or being unable to trigger an error
> > at the right time, this is what error injection frameworks or
> > debug-only sysfs hooks are for. The XFS kernel code has both,
> > xfstests use both, and they pretty much do away with the need for
> > custom binary filesystem images for such testing...
> 
> Error injection framework may not be alwasy available, eg. in kernels
> built for production. Yet I'd like to make the test possible.

Why would you ever enable error injection on a production kernel?
We simply don't run the error injection tests when the
infrastructure is not there, jsut like we do with all other tests
that are depenent on optional kernel/fs features

> Also, I find it useful to keep the exact images that are attached to a
> report and not necessarily try to recreate it to trigger a bug. If the
> images are small, hosting them with the sources makes testing easy.
> Big images would probably go to own repository and be linked.
> 
> I don't really expect fstests to store crafted images and would advise
> Qu to not even try to propose that, because the answer was quite
> predictable.

You say that like it's a bad thing?  What's the guarantee that a
specific corrupt image will always be sufficient to trigger the
problem the test is supposed to check? i.e. we could change a
different part of the FS code and that could mask the bug the image
used to trigger. The test then passes, but we haven't actually fix
the bug that the test used to exercise. i.e. we've got a false "we
fixed the problem" report, when all we did is prevent a specific
vector from tripping over it.

Error injection and sysfs hooks into debug code are much more
reliable ways of testing that the root cause of a problem is fixed
and stays fixed.  The reproducing trigger cannot be masked by
changes in other code layers, so we know that if the error
injection/sysfs hook test handles the problem correctly, we have
actually fixed the underlying bug and not just masked it...

Cheers,

Dave.
-- 
Dave Chinner
da...@fromorbit.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Making statfs return qgroup info (for container use case etc.)

[Dave Chinner]

On Fri, Oct 07, 2016 at 06:58:47PM +0200, David Sterba wrote:
> On Fri, Oct 07, 2016 at 09:40:11AM +1100, Dave Chinner wrote:
> > XFS does this with directory tree quotas. It was implmented 10 years
> > ago or so, IIRC...
> 
> Sometimes, the line between a historical remark and trolling is thin

All I've done is quickly point out that we've already got an
implementation of the exact functionality being asked for and given
a rough indication of when the commits hit the tree. i.e. so anyone
wanting to implment this in btrfs can search for it easily and work
out how to use the existing VFS infrastructure to report this
information.

You can take that as trolling if you want, but if you think I have
time for playing stupid, petty, idiotic games like that then you
need to ask yourself why you have had such a defensive and
paranoid reaction...

Cheers,

Dave.
-- 
Dave Chinner
da...@fromorbit.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

raid levels and NAS drives

[Charles Zeitler]

Is there any advantage to using NAS drives
under RAID levels,  as oppposed to regular
'desktop' drives for BTRFS?

Charles Zeitler

-- 
 The Perfect Is The Enemy Of
 The Good Enough
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Data total size shrinking during btrfs balance

[Jeremy Yoder]

Interesting.  I've always noticed that value to be fairly close to the
total drive size, so I just assumed it was supposed to match.

Thanks!
Jeremy

On Sun, Oct 9, 2016 at 5:08 PM, Hugo Mills  wrote:
> On Sun, Oct 09, 2016 at 04:59:04PM -0400, Jeremy Yoder wrote:
>> I didn't notice this until it was nearly too late.  I had run out of
>> metadata space so I ran
>>  btrfs balance start -v -dusage=0 /mnt/btrfsroot
>>
>> That went fine, but I was still low on metadata space, so I tried a
>> few variations:
>>  btrfs balance start -v -dusage=1 /mnt/btrfsroot
>>  btrfs balance start -v -dusage=5 /mnt/btrfsroot
>>  btrfs balance start -v -dusage=10 /mnt/btrfsroot
>>
>> I didn't run any of these to completion, I cancelled them after a few hours.
>>
>> I captured the output of a couple commands every once in a while:
>>  btrfs fi df /mnt/btrfsroot
>>  btrfs fi show /dev/sda5
>>
>> What I didn't notice (because I was looking at the metadata space) was
>> the Data total size.
>>
>> It starts out as:
>>
>> Data, single: total=4.61TiB, used=2.50TiB
>> System, DUP: total=8.00MiB, used=576.00KiB
>> System, single: total=4.00MiB, used=0.00
>> Metadata, DUP: total=46.50GiB, used=42.82GiB
>> Metadata, single: total=8.00MiB, used=0.00
>> unknown, single: total=512.00MiB, used=0.00
>>
>> Label: none  uuid: e6818780-7d28-41fe-b7a8-e38b61a98621
>> Total devices 1 FS bytes used 2.54TiB
>> devid1 size 5.35TiB used 4.70TiB path /dev/sda5
>>
>> Over time however, the Data total begins to drop:
>> Data, single: total=4.56TiB, used=2.50TiB
>> Data, single: total=4.41TiB, used=2.50TiB
>> Data, single: total=4.23TiB, used=2.51TiB
>> Data, single: total=4.00TiB, used=2.51TiB
>> Data, single: total=3.59TiB, used=2.51TiB
>> Data, single: total=3.35TiB, used=2.51TiB
>>
>> The drop corresponds to a drop in the "show" used amounts:
>> devid1 size 5.35TiB used 4.65TiB path /dev/sda5
>> devid1 size 5.35TiB used 4.50TiB path /dev/sda5
>> devid1 size 5.35TiB used 4.32TiB path /dev/sda5
>> devid1 size 5.35TiB used 4.09TiB path /dev/sda5
>> devid1 size 5.35TiB used 3.68TiB path /dev/sda5
>> devid1 size 5.35TiB used 3.44TiB path /dev/sda5
>>
>> System info:
>> Ubuntu 14.04
>> Linux server4 4.4.0-38-generic #57~14.04.1-Ubuntu SMP Tue Sep 6
>> 17:20:43 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
>>
>> I had been running the 14.04 btrfs-progs version 3.12-1ubuntu0.1
>>
>> I just upgraded to btrfs-progs 4.4.1~ubuntu14.04.1~ppa1 from a PPA,
>> which allowed me to run
>> # btrfs fi usage /mnt/btrfsroot/
>> Overall:
>> Device size:   5.35TiB
>> Device allocated:  3.44TiB
>> Device unallocated:1.91TiB
>> Device missing:  0.00B
>> Used:  2.59TiB
>> Free (estimated):  2.75TiB  (min: 1.80TiB)
>> Data ratio:   1.00
>> Metadata ratio:   2.00
>> Global reserve:  512.00MiB  (used: 0.00B)
>>
>> Data,single: Size:3.35TiB, Used:2.51TiB
>>/dev/sda5   3.35TiB
>>
>> Metadata,DUP: Size:46.50GiB, Used:42.07GiB
>>/dev/sda5  93.00GiB
>>
>> System,DUP: Size:32.00MiB, Used:416.00KiB
>>/dev/sda5  64.00MiB
>>
>> Unallocated:
>>/dev/sda5   1.91TiB
>>
>> Any suggestions?
>
>Do nothing. The FS is working as designed. :)
>
>The "Data, total" value you're worried about is the amount of space
> currently allocated for use as data. The balance operation takes a
> chunk of data allocation, and moves any data in it to other chunks.
> Once the chunk is empty, it's freed up. Now, when the data is moved,
> it will typically go to any free allocated space first; only if there
> isn't any free space allocated for data will more space be allocated.
> So, the normal behaviour is that when you run a balance, the data is
> effectively compacted into fewer chunks, and the unused chunks are
> freed up. If the FS needs more chunks allocated, it will do so
> automatically, so you're not losing anything here.
>
>Hugo.
>
> --
> Hugo Mills | You know... I'm sure this code would seem a lot
> hugo@... carfax.org.uk | better if I never tried running it.
> http://carfax.org.uk/  |
> PGP: E2AB1DE4  |
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Data total size shrinking during btrfs balance

[Hugo Mills]

On Sun, Oct 09, 2016 at 04:59:04PM -0400, Jeremy Yoder wrote:
> I didn't notice this until it was nearly too late.  I had run out of
> metadata space so I ran
>  btrfs balance start -v -dusage=0 /mnt/btrfsroot
> 
> That went fine, but I was still low on metadata space, so I tried a
> few variations:
>  btrfs balance start -v -dusage=1 /mnt/btrfsroot
>  btrfs balance start -v -dusage=5 /mnt/btrfsroot
>  btrfs balance start -v -dusage=10 /mnt/btrfsroot
> 
> I didn't run any of these to completion, I cancelled them after a few hours.
> 
> I captured the output of a couple commands every once in a while:
>  btrfs fi df /mnt/btrfsroot
>  btrfs fi show /dev/sda5
> 
> What I didn't notice (because I was looking at the metadata space) was
> the Data total size.
> 
> It starts out as:
> 
> Data, single: total=4.61TiB, used=2.50TiB
> System, DUP: total=8.00MiB, used=576.00KiB
> System, single: total=4.00MiB, used=0.00
> Metadata, DUP: total=46.50GiB, used=42.82GiB
> Metadata, single: total=8.00MiB, used=0.00
> unknown, single: total=512.00MiB, used=0.00
> 
> Label: none  uuid: e6818780-7d28-41fe-b7a8-e38b61a98621
> Total devices 1 FS bytes used 2.54TiB
> devid1 size 5.35TiB used 4.70TiB path /dev/sda5
> 
> Over time however, the Data total begins to drop:
> Data, single: total=4.56TiB, used=2.50TiB
> Data, single: total=4.41TiB, used=2.50TiB
> Data, single: total=4.23TiB, used=2.51TiB
> Data, single: total=4.00TiB, used=2.51TiB
> Data, single: total=3.59TiB, used=2.51TiB
> Data, single: total=3.35TiB, used=2.51TiB
> 
> The drop corresponds to a drop in the "show" used amounts:
> devid1 size 5.35TiB used 4.65TiB path /dev/sda5
> devid1 size 5.35TiB used 4.50TiB path /dev/sda5
> devid1 size 5.35TiB used 4.32TiB path /dev/sda5
> devid1 size 5.35TiB used 4.09TiB path /dev/sda5
> devid1 size 5.35TiB used 3.68TiB path /dev/sda5
> devid1 size 5.35TiB used 3.44TiB path /dev/sda5
> 
> System info:
> Ubuntu 14.04
> Linux server4 4.4.0-38-generic #57~14.04.1-Ubuntu SMP Tue Sep 6
> 17:20:43 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
> 
> I had been running the 14.04 btrfs-progs version 3.12-1ubuntu0.1
> 
> I just upgraded to btrfs-progs 4.4.1~ubuntu14.04.1~ppa1 from a PPA,
> which allowed me to run
> # btrfs fi usage /mnt/btrfsroot/
> Overall:
> Device size:   5.35TiB
> Device allocated:  3.44TiB
> Device unallocated:1.91TiB
> Device missing:  0.00B
> Used:  2.59TiB
> Free (estimated):  2.75TiB  (min: 1.80TiB)
> Data ratio:   1.00
> Metadata ratio:   2.00
> Global reserve:  512.00MiB  (used: 0.00B)
> 
> Data,single: Size:3.35TiB, Used:2.51TiB
>/dev/sda5   3.35TiB
> 
> Metadata,DUP: Size:46.50GiB, Used:42.07GiB
>/dev/sda5  93.00GiB
> 
> System,DUP: Size:32.00MiB, Used:416.00KiB
>/dev/sda5  64.00MiB
> 
> Unallocated:
>/dev/sda5   1.91TiB
> 
> Any suggestions?

   Do nothing. The FS is working as designed. :)

   The "Data, total" value you're worried about is the amount of space
currently allocated for use as data. The balance operation takes a
chunk of data allocation, and moves any data in it to other chunks.
Once the chunk is empty, it's freed up. Now, when the data is moved,
it will typically go to any free allocated space first; only if there
isn't any free space allocated for data will more space be allocated.
So, the normal behaviour is that when you run a balance, the data is
effectively compacted into fewer chunks, and the unused chunks are
freed up. If the FS needs more chunks allocated, it will do so
automatically, so you're not losing anything here.

   Hugo.

-- 
Hugo Mills | You know... I'm sure this code would seem a lot
hugo@... carfax.org.uk | better if I never tried running it.
http://carfax.org.uk/  |
PGP: E2AB1DE4  |


signature.asc
Description: Digital signature

Re: [PATCH] generic: make 17[1-4] work well when btrfs compression is enabled

[Darrick J. Wong]

On Sat, Oct 08, 2016 at 01:36:10AM +1100, Dave Chinner wrote:
> On Fri, Oct 07, 2016 at 03:00:42PM +0800, Wang Xiaoguang wrote:
> > When enabling btrfs compression, original codes can not fill fs
> > correctly, fix this.
> > 
> > Signed-off-by: Wang Xiaoguang 
> > ---
> >  tests/generic/171 | 4 +---
> >  tests/generic/172 | 2 +-
> >  tests/generic/173 | 4 +---
> >  tests/generic/174 | 4 +---
> >  4 files changed, 4 insertions(+), 10 deletions(-)
> > 
> > diff --git a/tests/generic/171 b/tests/generic/171
> > index f391685..d2ae8e4 100755
> > --- a/tests/generic/171
> > +++ b/tests/generic/171
> > @@ -75,9 +75,7 @@ _cp_reflink $testdir/bigfile $testdir/clonefile
> >  sync
> >  
> >  echo "Allocate the rest of the space"
> > -nr_free=$(stat -f -c '%f' $testdir)
> > -touch $testdir/file0 $testdir/file1

Why remove this line which ensures that the inode we're going to use (file1)
later in the test has been allocated /before/ we try to eat all the space?

> > -_pwrite_byte 0x61 0 $((blksz * nr_free)) $testdir/eat_my_space >> 
> > $seqres.full 2>&1
> > +dd if=/dev/zero of=$testdir/eat_my_space >> $seqres.full 2>&1

Uh... isn't a bunch of zeroes just as compressible as 0x61?  I suppose
the point here is to write until write returns ENOSPC, since in btrfs
land we can't assume that writing $nr blocks will use up at least that
much space.

Does XFS reflink still pass this test with this patch?

> Please don't replace xfs_io writes using a specific data pattern
> with dd calls that write zeros. Indeed, we don't use dd for new
> tests anymore - xfs_io should be used.
> 
> Write a function that fills all the remaining free space (one may
> already exist) and call it in all these places.

Yeah, there already are a few of these...

--D

> 
> Cheers,
> 
> Dave.
> -- 
> Dave Chinner
> da...@fromorbit.com
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Data total size shrinking during btrfs balance

[Jeremy Yoder]

I didn't notice this until it was nearly too late.  I had run out of
metadata space so I ran
 btrfs balance start -v -dusage=0 /mnt/btrfsroot

That went fine, but I was still low on metadata space, so I tried a
few variations:
 btrfs balance start -v -dusage=1 /mnt/btrfsroot
 btrfs balance start -v -dusage=5 /mnt/btrfsroot
 btrfs balance start -v -dusage=10 /mnt/btrfsroot

I didn't run any of these to completion, I cancelled them after a few hours.

I captured the output of a couple commands every once in a while:
 btrfs fi df /mnt/btrfsroot
 btrfs fi show /dev/sda5

What I didn't notice (because I was looking at the metadata space) was
the Data total size.

It starts out as:

Data, single: total=4.61TiB, used=2.50TiB
System, DUP: total=8.00MiB, used=576.00KiB
System, single: total=4.00MiB, used=0.00
Metadata, DUP: total=46.50GiB, used=42.82GiB
Metadata, single: total=8.00MiB, used=0.00
unknown, single: total=512.00MiB, used=0.00

Label: none  uuid: e6818780-7d28-41fe-b7a8-e38b61a98621
Total devices 1 FS bytes used 2.54TiB
devid1 size 5.35TiB used 4.70TiB path /dev/sda5

Over time however, the Data total begins to drop:
Data, single: total=4.56TiB, used=2.50TiB
Data, single: total=4.41TiB, used=2.50TiB
Data, single: total=4.23TiB, used=2.51TiB
Data, single: total=4.00TiB, used=2.51TiB
Data, single: total=3.59TiB, used=2.51TiB
Data, single: total=3.35TiB, used=2.51TiB

The drop corresponds to a drop in the "show" used amounts:
devid1 size 5.35TiB used 4.65TiB path /dev/sda5
devid1 size 5.35TiB used 4.50TiB path /dev/sda5
devid1 size 5.35TiB used 4.32TiB path /dev/sda5
devid1 size 5.35TiB used 4.09TiB path /dev/sda5
devid1 size 5.35TiB used 3.68TiB path /dev/sda5
devid1 size 5.35TiB used 3.44TiB path /dev/sda5

System info:
Ubuntu 14.04
Linux server4 4.4.0-38-generic #57~14.04.1-Ubuntu SMP Tue Sep 6
17:20:43 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

I had been running the 14.04 btrfs-progs version 3.12-1ubuntu0.1

I just upgraded to btrfs-progs 4.4.1~ubuntu14.04.1~ppa1 from a PPA,
which allowed me to run
# btrfs fi usage /mnt/btrfsroot/
Overall:
Device size:   5.35TiB
Device allocated:  3.44TiB
Device unallocated:1.91TiB
Device missing:  0.00B
Used:  2.59TiB
Free (estimated):  2.75TiB  (min: 1.80TiB)
Data ratio:   1.00
Metadata ratio:   2.00
Global reserve:  512.00MiB  (used: 0.00B)

Data,single: Size:3.35TiB, Used:2.51TiB
   /dev/sda5   3.35TiB

Metadata,DUP: Size:46.50GiB, Used:42.07GiB
   /dev/sda5  93.00GiB

System,DUP: Size:32.00MiB, Used:416.00KiB
   /dev/sda5  64.00MiB

Unallocated:
   /dev/sda5   1.91TiB

Any suggestions?
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Btrfs progs pre-release 4.8.1-rc1

[David Sterba]

On Fri, Oct 07, 2016 at 12:57:58PM -0700, Justin Maggard wrote:
> This is falling over on my 32-bit ARM platform.  Using stock Debian
> Jessie armel or armhf, it builds now, but the send ioctl is unhappy:
> 
> # btrfs send -p /data/test/.snapshots/1/snapshot/
> /data/test/.snapshots/2/snapshot/ > /dev/null
> At subvol /data/test/.snapshots/2/snapshot/
> ERROR: send ioctl failed with -25: Inappropriate ioctl for device
> 
> If I remove the packed attribute from the btrfs_ioctl_send_args struct
> and change the BUILD_ASSERT to expect a 72-byte structure, it all
> starts working again.  I'm using a 4.1 kernel built with the standard
> armhf toolchain from Wheezy (gcc 4.7.2).

Thanks for the report, I think I'll drop the additional changes and
release just the build fixes. We'd need to validate that all ioctls
really work on the target build architecture. The potential explanation
for the problem you found is that 32bit arm has 4 byte pointers but
needs 8 byte alignment.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

btrfs-convert: migrating files with flags set using chattr

[Lakshmipathi.G]

Hi.

Looks like btrfs-progs/btrfs-convert doesn't retain file attributes
like immutable flags, is this expected behavior ?

//  test.img is ext2
[root@ mnt]# touch a b
[root@ mnt]# chattr +a a
[root@ mnt]# chattr +i b
[root@ mnt]# lsattr a b
-a-- a
i--- b


// after btrfs-convert  test.img

[root@ mnt]# lsattr a b
 a
 b




Cheers,
Lakshmipathi.G
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH]btrfs-progs: Add fast,slow symlinks,fifo types to convert test

[Lakshmipathi.G]

Signed-off-by: Lakshmipathi.G 
---
 tests/common.convert | 20 +---
 1 file changed, 17 insertions(+), 3 deletions(-)

diff --git a/tests/common.convert b/tests/common.convert
index 1e00d38..5083e65 100644
--- a/tests/common.convert
+++ b/tests/common.convert
@@ -25,10 +25,10 @@ generate_dataset() {
done
;;
 
-   symlink)
+   fast_symlink)
for num in $(seq 1 $DATASET_SIZE); do
run_check $SUDO_HELPER touch 
$dirpath/$dataset_type.$num
-   run_check $SUDO_HELPER ln -s 
$dirpath/$dataset_type.$num $dirpath/slink.$num
+   run_check $SUDO_HELPER cd $dirpath && ln -s 
$dataset_type.$num $dirpath/slink.$num && cd /
done
;;
 
@@ -71,12 +71,26 @@ generate_dataset() {
run_check $SUDO_HELPER setfattr -n user.foo -v 
bar$num $dirpath/$dataset_type.$num
done
;;
+
+   fifo)
+   for num in $(seq 1 $DATASET_SIZE); do
+   run_check $SUDO_HELPER mkfifo 
$dirpath/$dataset_type.$num
+   done
+   ;;
+
+   slow_symlink)
+   long_filename=`date +%s | sha256sum | cut -f1 -d'-'`
+   run_check $SUDO_HELPER touch $dirpath/$long_filename
+   for num in $(seq 1 $DATASET_SIZE); do
+   run_check $SUDO_HELPER ln -s 
$dirpath/$long_filename $dirpath/slow_slink.$num
+   done
+   ;;
esac
 }
 
 populate_fs() {
 
-for dataset_type in 'small' 'hardlink' 'symlink' 'brokenlink' 'perm' 
'sparse' 'acls'; do
+for dataset_type in 'small' 'hardlink' 'fast_symlink' 'brokenlink' 
'perm' 'sparse' 'acls' 'fifo' 'slow_symlink'; do
generate_dataset "$dataset_type"
done
 }
-- 
1.9.3

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: btrfs_direct_IO oops

[Dave Jones]

On Sat, Oct 08, 2016 at 07:20:08PM -0400, Dave Jones wrote:
 > On Sat, Oct 08, 2016 at 07:29:03PM +0100, Al Viro wrote:
 >  > On Sat, Oct 08, 2016 at 02:08:06PM -0400, Dave Jones wrote:
 >  > > That code: matches this dissembly:
 >  > > 
 >  > > for (i = seg + 1; i < iter->nr_segs; i++) {
 >  > 
 >  > *whoa*
 >  > 
 >  > OK, that loop in check_direct_IO() should be done *ONLY* for
 >  > iovec iter - even for a bvec one it's completely bogus, and
 >  > for pipe ones it blows up immediately.
 >  > 
 >  > Sorry, I'd missed that bogosity - replace
 >  > if (iov_iter_rw(iter) == WRITE)
 >  >   return 0;
 >  > with
 >  >   if (iov_iter_rw(iter) != READ || !iter_is_iovec(iter))
 >  >   return 0;
 >  > in there; that should fix the damn thing.
 > 
 > Yep, seems to do the trick. Have been running the last six hours
 > without seeing it or anything similar since.

Overnight, I did hit another iov related warning..

idx = 0, offset = 0
curbuf = 0, nrbufs = 1, buffers = 16
[9fa21100 ea00065f6d80 0 4096]
[  (null)   (null) 0 0]
[  (null)   (null) 0 0]
[  (null)   (null) 0 0]
[  (null)   (null) 0 0]
[  (null)   (null) 0 0]
[  (null)   (null) 0 0]
[  (null)   (null) 0 0]
[  (null)   (null) 0 0]
[  (null)   (null) 0 0]
[  (null)   (null) 0 0]
[  (null)   (null) 0 0]
[  (null)   (null) 0 0]
[  (null)   (null) 0 0]
[  (null)   (null) 0 0]
[  (null)   (null) 0 0]

WARNING: CPU: 0 PID: 29610 at lib/iov_iter.c:327 sanity+0x102/0x150
CPU: 0 PID: 29610 Comm: trinity-c9 Not tainted 4.8.0-think+ #8 
 c97b7ae8
 9f3e2011
 
 

 9fc1e22b
 9f3fa2f2
 c97b7b28
 9f08b010

 014734c3d60f
 9fc1e22b
 0147
 0010

Call Trace:
 [] dump_stack+0x6c/0x9b
 [] ? sanity+0x102/0x150
 [] __warn+0x110/0x130
 [] warn_slowpath_null+0x2c/0x40
 [] sanity+0x102/0x150
 [] copy_page_to_iter+0x2be/0x480
 [] ? pagecache_get_page+0xba/0x4f0
 [] generic_file_read_iter+0x245/0xd30
 [] generic_file_splice_read+0xfd/0x230
 [] ? pipe_to_user+0x40/0x40
 [] do_splice_to+0x98/0xd0
 [] splice_direct_to_actor+0xd4/0x2c0
 [] ? generic_pipe_buf_nosteal+0x10/0x10
 [] do_splice_direct+0xc5/0x110
 [] do_sendfile+0x242/0x470
 [] SyS_sendfile64+0x7d/0xf0
 [] do_syscall_64+0x7f/0x200
 [] entry_SYSCALL64_slow_path+0x25/0x25
---[ end trace 2c7bd411d4aa0491 ]---

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html