Re: [PATCH] btrfs-progs: change mans to describe the third copy of superblock

[Satoru Takeuchi]

At Thu, 19 Oct 2017 17:05:18 +0800,
Qu Wenruo wrote:
> 
> 
> 
> On 2017年10月19日 16:34, Misono, Tomohiro wrote:
> > On 2017/10/19 16:45, Satoru Takeuchi wrote:
> >> Some tools can select which superblock these commands use by "-s 
> >> "
> >> option. Although this option says the valid values are 0-2, we can set 3
> >> if filesystem is very large.
> >>
> > 
> > Hello, 
> > Wiki says there are 4 superblocks. However in the implementation 
> > BTRFS_SUPER_MIROR_MAX
> > is 3 and 0 indicates the block at 64K (disk-io.h of btrfs-progs), therefore 
> > I think
> > there is no 4th superblock actually.
> 
> Kernel implementation also shows that it will only update up to 3
> superblocks:
> 
> ---
>   if (max_mirrors == 0)
>   max_mirrors = BTRFS_SUPER_MIRROR_MAX;
> 
>   for (i = 0; i < max_mirrors; i++) {
>   bytenr = btrfs_sb_offset(i);
>   if (bytenr + BTRFS_SUPER_INFO_SIZE >=
>   device->commit_total_bytes)
>   break;
> ---
> 
> And BTRFS_SUPER_MIRROR_MAX is 3:
> ---
> #define BTRFS_SUPER_MIRROR_MAX 3
> ---
> 
> So even you can set any value and btrfs_sb_offset() can calculate the
> super block offset, you will just read out some garbage.

My fault, sorry. I should read source more carefully. And thank you both
to let me know my mistake.

Thanks,
Satoru

> 
> Thanks,
> Qu
> > 
> > Regards,
> > Tomohiro
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> > the body of a message to majord...@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Unmountable fs - missing generation?

[Satoru Takeuchi]

At Thu, 19 Oct 2017 12:03:08 +0900,
satoru takeuchi wrote:
> 
> Resend it since I forgot to CC linux-btrfs ML >Larkin
> 
> On Oct 17, 2017, at 0:16, Larkin Lowrey <llow...@nuclearwinter.com>
> wrote:
> 
> I am unable to mount one my my filesystems. The superblock thinks
> the latest generation is 2220927 but I can't seem to find a root
> with that number. I can find 2220926 and 2220928 but not 2220927.
> Is there anything that I can do to recover this FS?
> 
> 
> `btrfs-select-super` may help you. Please see the following steps.
> 
> 1. Backup the current unmountable fs image if possible.
> 2. Salvage your files as much as possilbe with reading the following
> document
> if possible.
> 
> https://btrfs.wiki.kernel.org/index.php/Restore
> 
> 3. Execute `btrfs-select-super -s 1 /dev/Cached/Backups`. Please note
> that
> this command changes the contents of /dev/Cached/Backups. So if this
> command fails. Things would get worse.

I forgot to tell one important point. You can only run above mentioned
command iff 1st copy of superblock is valid. It can be confirmed
by `btrfs inspect-internal dump-super (or btrfs-dump-super)` as follows.

* valid case

```
$ sudo btrfs inspect dump-super -a /dev/sdb4
...
superblock: bytenr=67108864, device=/dev/sdb4   # 1st copy of superblock
-
csum0x423bcd19 [match]
bytenr  67108864
flags   0x1
...
```

* invalid case

```
$ sudo btrfs inspect dump-super -a /dev/sdb4
...
superblock: bytenr=67108864, device=/dev/sdb4
-
ERROR: bad magic on superblock on /dev/sdb4 at 67108864
...
```

Thanks,
Satoru

> Thanks,
> Satoru
> 
> 
> # btrfs check /dev/Cached/Backups
> checksum verify failed on 159057884594176 found 15284E33 wanted
> C8C5B54E
> checksum verify failed on 159057884594176 found 15284E33 wanted
> C8C5B54E
> checksum verify failed on 159057884594176 found 472037C9 wanted
> 9ACDCCB4
> checksum verify failed on 159057884594176 found 472037C9 wanted
> 9ACDCCB4
> Csum didn't match
> Couldn't setup extent tree
> Couldn't open file system
> 
> # btrfs-find-root -g 2220927 /dev/Cached/Backups
> Couldn't setup extent tree
> Couldn't setup device tree
> Superblock thinks the generation is 2220927
> Superblock thinks the level is 2
> 
> Found tree root at 159057884577792 gen 2220927 level 2
> Well block 101489031790592(gen: 2220928 level: 2) seems good, but
> generation/level doesn't match, want gen: 2220927 level: 2
> 
> # btrfs check --tree-root 159057884577792 /dev/Cached/Backups
> checksum verify failed on 159057884594176 found 15284E33 wanted
> C8C5B54E
> checksum verify failed on 159057884594176 found 15284E33 wanted
> C8C5B54E
> checksum verify failed on 159057884594176 found 472037C9 wanted
> 9ACDCCB4
> checksum verify failed on 159057884594176 found 472037C9 wanted
> 9ACDCCB4
> Csum didn't match
> Couldn't setup extent tree
> Couldn't open file system
> 
> # btrfs check --tree-root 101489031790592 /dev/Cached/Backups
> parent transid verify failed on 101489031790592 wanted 2220927
> found 2220928
> parent transid verify failed on 101489031790592 wanted 2220927
> found 2220928
> parent transid verify failed on 101489031790592 wanted 2220927
> found 2220928
> parent transid verify failed on 101489031790592 wanted 2220927
> found 2220928
> Ignoring transid failure
> parent transid verify failed on 159057595138048 wanted 2220927
> found 2220920
> parent transid verify failed on 159057595138048 wanted 2220927
> found 2220920
> parent transid verify failed on 159057595138048 wanted 2220927
> found 2220920
> parent transid verify failed on 159057595138048 wanted 2220927
> found 2220920
> Ignoring transid failure
> parent transid verify failed on 158652658122752 wanted 2220927
> found 2220911
> parent transid verify failed on 158652658122752 wanted 2220927
> found 2220911
> parent transid verify failed on 158652658122752 wanted 2220927
> found 2220911
> parent transid verify failed on 158652658122752 wanted 2220927
> found 2220911
> Ignoring transid failure
> Checking filesystem on /dev/Cached/Backups
> UUID: 1b213dfd-6486-47d8-8459-bc5825882023
> checking extents
> parent transid verify failed on 116329711550464 wanted 2220928
> found 2220921
> parent transid verify failed on 116329711550464 wanted 2220928
> fo

[PATCH] btrfs-progs: change mans to describe the third copy of superblock

[Satoru Takeuchi]

Some tools can select which superblock these commands use by "-s "
option. Although this option says the valid values are 0-2, we can set 3
if filesystem is very large.

Signed-off-by: Satoru Takeuchi <satoru.takeu...@gmail.com>
---
 Documentation/btrfs-check.asciidoc| 2 +-
 Documentation/btrfs-restore.asciidoc  | 2 +-
 Documentation/btrfs-select-super.asciidoc | 3 ++-
 3 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/Documentation/btrfs-check.asciidoc 
b/Documentation/btrfs-check.asciidoc
index fbf4884..a557cff 100644
--- a/Documentation/btrfs-check.asciidoc
+++ b/Documentation/btrfs-check.asciidoc
@@ -74,7 +74,7 @@ run in read-only mode, this option exists to calm potential 
panic when users
 are going to run the checker
 
 -s|--super ::
-use 'superblock'th superblock copy, valid values are 0, 1 or 2 if the
+use 'superblock'th superblock copy, valid values are 0, 1, 2 or 3 if the
 respective superblock offset is within the device size
 +
 This can be used to use a different starting point if some of the primary
diff --git a/Documentation/btrfs-restore.asciidoc 
b/Documentation/btrfs-restore.asciidoc
index 090dcc5..c19e0e2 100644
--- a/Documentation/btrfs-restore.asciidoc
+++ b/Documentation/btrfs-restore.asciidoc
@@ -63,7 +63,7 @@ use  to read the root tree
 only restore files that are under specified subvolume root pointed by 
 
 -u|--super ::
-use given superblock mirror identified by , it can be 0,1 or 2
+use given superblock mirror identified by , it can be 0, 1, 2 or 3
 
 -r|--root ::
 only restore files that are under a specified subvolume whose objectid is 

diff --git a/Documentation/btrfs-select-super.asciidoc 
b/Documentation/btrfs-select-super.asciidoc
index 6e94a03..7f96bd8 100644
--- a/Documentation/btrfs-select-super.asciidoc
+++ b/Documentation/btrfs-select-super.asciidoc
@@ -32,13 +32,14 @@ Superblock copies exist in the following offsets on the 
device:
 - primary: '64KiB' (65536)
 - 1st copy: '64MiB' (67108864)
 - 2nd copy: '256GiB' (274877906944)
+- 3rd copy: '1PiB' (1125899906842624)
 
 A superblock size is '4KiB' (4096).
 
 OPTIONS
 ---
 -s|--super ::
-use 'superblock'th superblock copy, valid values are 0 1 or 2 if the
+use 'superblock'th superblock copy, valid values are 0, 1, 2 or 3 if the
 respective superblock offset is within the device size
 
 SEE ALSO
-- 
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] btrfs-progs: allow "none" to disable compression for convenience

[Satoru Takeuchi]

At Tue, 19 Sep 2017 17:14:27 +0200,
David Sterba wrote:
> 
> On Mon, Sep 18, 2017 at 09:41:17AM +0900, Satoru Takeuchi wrote:
> > At Sun, 17 Sep 2017 14:08:40 +0100,
> > Mike Fleetwood wrote:
> > > 
> > > On 17 September 2017 at 01:36, Satoru Takeuchi
> > > <satoru.takeu...@gmail.com> wrote:
> > > > It's messy to use "" to disable compression. Introduce the new value 
> > > > "no"
> > > > which can also be used for this purpose.
> > > 
> > > From an English language point of view, "none" would be better.  None
> > > says the absence of, where as no is more general negative.
> > 
> > Thank you for your comment. How about is it?
> > 
> > ---
> > It's messy to use "" to disable compression. Introduce the new value "none"
> > which can also be used for this purpose.
> 
> I'd allow both values, 'no' and 'none', similar to the mount options,
> that also accept both (technically, the 'no' + anything is accepted for
> disabling compression).

But only "no" is writtein in man 5 btrfs.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH RESEND] btrfs-progs: allow "no" to disable compression for convenience

[Satoru Takeuchi]

At Sat, 14 Oct 2017 18:19:14 +0200,
Koen Kooi wrote:
> 
> Op 14-10-17 om 14:54 schreef Satoru Takeuchi:
> > It's messy to use "" to disable compression. Introduce the new value "no"
> > which can also be used for this purpose.
> 
> Wouldn't 'none' be a better fit?

I consider "no" is better because we use `nocompress` or `compress=no`
mount option to disable compression. I want to unify the expression.

Thanks,
Satoru

> 
> regards,
> 
> Koen
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] btrfs: use BLK_STS defines where needed

[Satoru Takeuchi]

At Sat, 14 Oct 2017 08:35:56 +0800,
Anand Jain wrote:
> 
> At few places we could use BLK_STS_OK and BLK_STS_NOSUPP.
> 
> Signed-off-by: Anand Jain 

Reviewed-by: Satoru Taekeuchi 

> ---
>  fs/btrfs/compression.c | 3 ++-
>  fs/btrfs/inode.c   | 4 ++--
>  fs/btrfs/volumes.c | 2 +-
>  3 files changed, 5 insertions(+), 4 deletions(-)
> 
> diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c
> index b51d23f5cafa..43d4b51556ec 100644
> --- a/fs/btrfs/compression.c
> +++ b/fs/btrfs/compression.c
> @@ -239,7 +239,8 @@ static void end_compressed_bio_write(struct bio *bio)
>cb->start,
>cb->start + cb->len - 1,
>NULL,
> -  bio->bi_status ? 0 : 1);
> +  bio->bi_status ?
> +  BLK_STS_OK : BLK_STS_NOTSUPP);
>   cb->compressed_pages[0]->mapping = NULL;
>  
>   end_compressed_writeback(inode, cb);
> diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
> index 128f3e58634f..9aa03c89ad86 100644
> --- a/fs/btrfs/inode.c
> +++ b/fs/btrfs/inode.c
> @@ -8360,7 +8360,7 @@ static void btrfs_endio_direct_read(struct bio *bio)
>   if (dip->flags & BTRFS_DIO_ORIG_BIO_SUBMITTED) {
>   err = btrfs_subio_endio_read(inode, io_bio, err);
>   if (!err)
> - bio->bi_status = 0;
> + bio->bi_status = BLK_STS_OK;
>   }
>  
>   unlock_extent(_I(inode)->io_tree, dip->logical_offset,
> @@ -8478,7 +8478,7 @@ static void btrfs_end_dio_bio(struct bio *bio)
>   if (dip->errors) {
>   bio_io_error(dip->orig_bio);
>   } else {
> - dip->dio_bio->bi_status = 0;
> + dip->dio_bio->bi_status = BLK_STS_OK;
>   bio_endio(dip->orig_bio);
>   }
>  out:
> diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
> index a61405c2cd31..15e017af756c 100644
> --- a/fs/btrfs/volumes.c
> +++ b/fs/btrfs/volumes.c
> @@ -6020,7 +6020,7 @@ static void btrfs_end_bio(struct bio *bio)
>* this bio is actually up to date, we didn't
>* go over the max number of errors
>*/
> - bio->bi_status = 0;
> + bio->bi_status = BLK_STS_OK;
>   }
>  
>   btrfs_end_bbio(bbio, bio);
> -- 
> 2.13.1
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH RESEND] btrfs-progs: allow "no" to disable compression for convenience

[Satoru Takeuchi]

It's messy to use "" to disable compression. Introduce the new value "no"
which can also be used for this purpose.

Signed-off-by: Satoru Takeuchi <satoru.takeu...@gmail.com>
---
 Documentation/btrfs-property.asciidoc | 2 +-
 props.c   | 6 --
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/Documentation/btrfs-property.asciidoc 
b/Documentation/btrfs-property.asciidoc
index 7ed6a7d..97b90d6 100644
--- a/Documentation/btrfs-property.asciidoc
+++ b/Documentation/btrfs-property.asciidoc
@@ -43,7 +43,7 @@ read-only flag of subvolume: true or false
 label
 label of device
 compression
-compression setting for an inode: lzo, zlib, zstd, or "" (empty string)
+compression setting for an inode: lzo, zlib, zstd, no, or "" (empty string). 
Both no and "" are for disabling compression.
 
 *list* [-t ] ::
 Lists available properties with their descriptions for the given object.
diff --git a/props.c b/props.c
index a7e3e96..a2df868 100644
--- a/props.c
+++ b/props.c
@@ -142,9 +142,11 @@ static int prop_compression(enum prop_object_type type,
memcpy(xattr_name + XATTR_BTRFS_PREFIX_LEN, name, strlen(name));
xattr_name[XATTR_BTRFS_PREFIX_LEN + strlen(name)] = '\0';
 
-   if (value)
+   if (value) {
+   if (!strcmp(value, "no"))
+   value = "";
sret = fsetxattr(fd, xattr_name, value, strlen(value), 0);
-   else
+   } else
sret = fgetxattr(fd, xattr_name, NULL, 0);
if (sret < 0) {
ret = -errno;
-- 
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] btrfs-progs: doc: update help/document of btrfs device remove

[Satoru Takeuchi]

At Tue, 3 Oct 2017 17:12:39 +0900,
Misono, Tomohiro wrote:
> 
> This patch updates help/document of "btrfs device remove" in two points:
> 
> 1. Add explanation of 'missing' for 'device remove'. This is only
> written in wikipage currently.
> (https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices)
> 
> 2. Add example of device removal in the man document. This is because
> that explanation of "remove" says "See the example section below", but
> there is no example of removal currently.
> 
> Signed-off-by: Tomohiro Misono 
> ---
>  Documentation/btrfs-device.asciidoc | 19 +++
>  cmds-device.c   | 10 +-
>  2 files changed, 28 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/btrfs-device.asciidoc 
> b/Documentation/btrfs-device.asciidoc
> index 88822ec..dc523a9 100644
> --- a/Documentation/btrfs-device.asciidoc
> +++ b/Documentation/btrfs-device.asciidoc
> @@ -75,6 +75,10 @@ The operation can take long as it needs to move all data 
> from the device.
>  It is possible to delete the device that was used to mount the filesystem. 
> The
>  device entry in mount table will be replaced by another device name with the
>  lowest device id.
> ++
> +If device is mounted as degraded mode (-o degraded), special term "missing"
> +can be used for . In that case, the first device that is described by
> +the filesystem metadata, but not presented at the mount time will be removed.
>  
>  *delete* | [|...] ::
>  Alias of remove kept for backward compatibility
> @@ -206,6 +210,21 @@ data or the block groups occupy the whole first device.
>  The device size of '/dev/sdb' as seen by the filesystem remains unchanged, 
> but
>  the logical space from 50-100GiB will be unused.
>  
> + REMOVE DEVICE 

It's a part of "TYPICAL USECASES" section. So it's also necessary to modify
the following sentence

===
See the example section below.
===

to as follow.

===
See the *TYPICAL USECASES* section below.
===

Or just removing the above mentioned sentence is also OK since there is
"See the section *TYPICAL USECASES* for some examples." in "DEVICE MANAGEMENT"
section.

> +
> +Device removal must satisfy the profile constraints, otherwise the command
> +fails. For example:
> +
> + $ btrfs device remove /dev/sda /mnt
> + $ ERROR: error removing device '/dev/sda': unable to go below two devices 
> on raid1

s/^$  ERROR/ERROR/

> +
> +
> +In order to remove a device, you need to convert profile in this case:
> +
> + $ btrfs balance start -mconvert=dup /mnt
> + $ btrfs balance start -dconvert=single /mnt

It's simpler to convert both the RAID configuration of data and metadata
by the following one command.

$ btrfs balance -mconvert=dup -dconvert=single /mnt

> + $ btrfs device remove /dev/sda /mnt
> +
>  DEVICE STATS
>  
>  
> diff --git a/cmds-device.c b/cmds-device.c
> index 4337eb2..6cb53ff 100644
> --- a/cmds-device.c
> +++ b/cmds-device.c
> @@ -224,9 +224,16 @@ static int _cmd_device_remove(int argc, char **argv,
>   return !!ret;
>  }
>  
> +#define COMMON_USAGE_REMOVE_DELETE \
> + "", \
> + "If 'missing' is specified for , the first device that is", \
> + "described by the filesystem metadata, but not presented at the", \
> + "mount time will be removed."
> +
>  static const char * const cmd_device_remove_usage[] = {
>   "btrfs device remove | [|...] ",
>   "Remove a device from a filesystem",
> + COMMON_USAGE_REMOVE_DELETE,
>   NULL
>  };
>  
> @@ -237,7 +244,8 @@ static int cmd_device_remove(int argc, char **argv)
>  
>  static const char * const cmd_device_delete_usage[] = {
>   "btrfs device delete | [|...] ",
> - "Remove a device from a filesystem",
> + "Remove a device from a filesystem (alias of \"btrfs device remove\")",
> + COMMON_USAGE_REMOVE_DELETE,
>   NULL
>  };

This snippet is not related to the description of this patch.
Dividing this patch is better.

Thanks,
Satoru

>  
> -- 
> 2.9.5
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: USB upgrade fun

[Satoru Takeuchi]

At Sun, 08 Oct 2017 17:58:10 +0800,
Kai Hendry wrote:
> 
> Hi there,
> 
> My /mnt/raid1 suddenly became full somewhat expectedly, so I bought 2
> new USB 4TB hard drives (one WD, one Seagate) to upgrade to.
> 
> After adding sde and sdd I started to see errors in dmesg [2].
> https://s.natalian.org/2017-10-07/raid1-newdisks.txt
> [2] https://s.natalian.org/2017-10-07/btrfs-errors.txt

These messages are harmless. Qu is tuckling on this problem.

> 
> Sidenote: I've since learnt that removing a drive actually deletes the
> contents of the drive? I don't want that. I was hoping to put that drive
> into cold storage. How do I remove a drive without losing data from a
> RAID1 configuration?

Please let me clarify what you said. Do you worry about losing filesystem data
in removed device, in this case /dev/sdc1? To be more specific,
if /mnt/raid1/file is in /dev/sdc1 and lose this file by removing this device?
If so, don't worry. When removing /dev/sdc1, the filesystem data exists in this
device is moved to other devices, /dev/sdb1, /dev/sdd1, or /dev/sde1.

Just FYI, `btrfs replace /dev/sdc1 /dev/sdd1 /mnt/raid1` is more suitable
in your case.

> 
> 
> I assumed it had to perhaps with the USB bus on my NUC5CPYB being maxed
> out, and to expedite the sync, I tried to remove one of the older 2TB
> sdc1.  However the load went crazy and my system went completely
> unstable. I shutdown the machine and after an hour I hard powered it
> down since it seemed to hang (it's headless).

Because all data in /dev/sdc1, in this case totally

 1.81TiB(data) + 6.00GiB(metadata) + 32MiB(system)

should be moved to remaining devices.

> 
> 
> After a reboot it failed, namely because "nofail" wasn't in my fstab and
> systemd is pedantic by default. After managing to get it booting into my
> system without /mnt/raid1 I faced these "open ctree failed" issues. 
> After running btrfs check on all the drives and getting nowhere, I
> decided to unplug the new drives and I discovered that when I take out
> the new 4TB WD drive, I could mount it with -o degraded.
> 
> dmesg errors with the WD include "My Passport" Wrong diagnostic page;
> asked for 1 got 8 "Failed to get diagnostic page 0xffea" which
> raised my suspicions. The model number btw is WDBYFT0040BYI-WESN
> 
> Anyway, I'm back up and running with 2x2TB  (one of them didn't finish
> removing, I don't know which) & 1x4TB.
> 
> [1] https://s.natalian.org/2017-10-08/usb-btrfs.txt
> 
> I've decided to send the WD back for a refund. I've decided I want keep
> the 2x2TB and RAID1 with the new 1x4TB disk. So 4TB total. btrfs
> complains now of "Some devices missing" [1]. How do I fix this
> situation?

Probably `btrfs device remove missing /mnt/raid1` works.

Thanks,
Satoru

> 
> Any tips how to name this individual disks? hdparm -I isn't a lot to go
> on.
> 
> [hendry@nuc ~]$ sudo hdparm -I /dev/sdb1 | grep Model
> Model Number:   ST4000LM024-2AN17V
> [hendry@nuc ~]$ sudo hdparm -I /dev/sdc1 | grep Model
> Model Number:   ST2000LM003 HN-M201RAD
> [hendry@nuc ~]$ sudo hdparm -I /dev/sdd1 | grep Model
> Model Number:   ST2000LM005 HN-M201AAD
> 
> 
> Ok, thanks. Hope you can guide me,
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] btrfs: Remove WARN_ON for unaligned device created before v4.13 and adds more user friendly output

[Satoru Takeuchi]

At Sat, 23 Sep 2017 10:19:26 +0900,
Satoru Takeuchi wrote:
> 
> At Wed, 20 Sep 2017 15:18:43 +0900,
> Qu Wenruo wrote:
> > 
> > Commit 7dfb8be11b5d ("btrfs: Round down values which are written for
> > total_bytes_size") is fixing the unaligned device size caused by
> > adding/shrinking device.
> > 
> > It added a new WARN_ON() when device size is unaligned.
> > This is fine for new device added to btrfs using v4.13 kernel, but not
> > existing device whose total_bytes is already unaligned.
> > 
> > And the WARN_ON() will get triggered every time a block group get
> > created/removed on the unaligned device.
> > 
> > This patch will remove the WARN_ON(), and warn user more gently what's
> > happening and how to fix it.
> > 
> > Reported-by: Rich Rauenzahn <r...@shroop.net>
> > Fixes: 7dfb8be11b5d ("btrfs: Round down values which are written for
> > total_bytes_size")
> > Signed-off-by: Qu Wenruo <quwenruo.bt...@gmx.com>
> > ---
> >  fs/btrfs/ctree.h   |  1 -
> >  fs/btrfs/volumes.c | 16 
> >  2 files changed, 12 insertions(+), 5 deletions(-)
> > 
> > diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
> > index 5a8933da39a7..4de9269e435a 100644
> > --- a/fs/btrfs/ctree.h
> > +++ b/fs/btrfs/ctree.h
> > @@ -1562,7 +1562,6 @@ static inline void 
> > btrfs_set_device_total_bytes(struct extent_buffer *eb,
> >  {
> > BUILD_BUG_ON(sizeof(u64) !=
> >  sizeof(((struct btrfs_dev_item *)0))->total_bytes);
> > -   WARN_ON(!IS_ALIGNED(val, eb->fs_info->sectorsize));
> > btrfs_set_64(eb, s, offsetof(struct btrfs_dev_item, total_bytes), val);
> >  }
> >  
> > diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
> > index 0e8f16c305df..afae25df6a8c 100644
> > --- a/fs/btrfs/volumes.c
> > +++ b/fs/btrfs/volumes.c
> > @@ -6472,15 +6472,23 @@ static int read_one_chunk(struct btrfs_fs_info 
> > *fs_info, struct btrfs_key *key,
> > return 0;
> >  }
> >  
> > -static void fill_device_from_item(struct extent_buffer *leaf,
> > -struct btrfs_dev_item *dev_item,
> > -struct btrfs_device *device)
> > +static void fill_device_from_item(struct btrfs_fs_info *fs_info,
> > + struct extent_buffer *leaf,
> > + struct btrfs_dev_item *dev_item,
> > + struct btrfs_device *device)
> >  {
> > unsigned long ptr;
> >  
> > device->devid = btrfs_device_id(leaf, dev_item);
> > device->disk_total_bytes = btrfs_device_total_bytes(leaf, dev_item);
> > device->total_bytes = device->disk_total_bytes;
> > +   if (!IS_ALIGNED(device->total_bytes, fs_info->sectorsize)) {
> > +   btrfs_warn(fs_info,
> > +  "devid %llu has unaligned total bytes %llu",
> > +  device->devid, device->disk_total_bytes);
> > +   btrfs_warn(fs_info,
> > +  "please shrink the device a little and resize back 
> > to fix it");
> > +   }
> 
> How about telling uses to know device->total_bytes should be alligned
> to fs_info->sectorsize here?
> 
> Thanks,

I should make my comment clearer, sorry.

===
+   if (!IS_ALIGNED(device->total_bytes, fs_info->sectorsize)) {
+   btrfs_warn(fs_info,
+  "devid %llu: total bytes %llu should be aligned to 
%u bytes",
+  device->devid, device->disk_total_bytes, 
fs_info->sectorsize);
+   btrfs_warn(fs_info,
+  "please shrink the device a little and resize back 
to fix it");
+   }
===

Thanks,
Satoru

> Satoru
> 
> > device->commit_total_bytes = device->disk_total_bytes;
> > device->bytes_used = btrfs_device_bytes_used(leaf, dev_item);
> > device->commit_bytes_used = device->bytes_used;
> > @@ -6625,7 +6633,7 @@ static int read_one_dev(struct btrfs_fs_info *fs_info,
> > return -EINVAL;
> > }
> >  
> > -   fill_device_from_item(leaf, dev_item, device);
> > +   fill_device_from_item(fs_info, leaf, dev_item, device);
> > device->in_fs_metadata = 1;
> > if (device->writeable && !device->is_tgtdev_for_dev_replace) {
> > device->fs_devices->total_rw_bytes += device->total_bytes;
> > -- 
> > 2.14.1
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> > the body of a message to majord...@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] btrfs: Remove WARN_ON for unaligned device created before v4.13 and adds more user friendly output

[Satoru Takeuchi]

At Wed, 20 Sep 2017 15:18:43 +0900,
Qu Wenruo wrote:
> 
> Commit 7dfb8be11b5d ("btrfs: Round down values which are written for
> total_bytes_size") is fixing the unaligned device size caused by
> adding/shrinking device.
> 
> It added a new WARN_ON() when device size is unaligned.
> This is fine for new device added to btrfs using v4.13 kernel, but not
> existing device whose total_bytes is already unaligned.
> 
> And the WARN_ON() will get triggered every time a block group get
> created/removed on the unaligned device.
> 
> This patch will remove the WARN_ON(), and warn user more gently what's
> happening and how to fix it.
> 
> Reported-by: Rich Rauenzahn 
> Fixes: 7dfb8be11b5d ("btrfs: Round down values which are written for
> total_bytes_size")
> Signed-off-by: Qu Wenruo 
> ---
>  fs/btrfs/ctree.h   |  1 -
>  fs/btrfs/volumes.c | 16 
>  2 files changed, 12 insertions(+), 5 deletions(-)
> 
> diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
> index 5a8933da39a7..4de9269e435a 100644
> --- a/fs/btrfs/ctree.h
> +++ b/fs/btrfs/ctree.h
> @@ -1562,7 +1562,6 @@ static inline void btrfs_set_device_total_bytes(struct 
> extent_buffer *eb,
>  {
>   BUILD_BUG_ON(sizeof(u64) !=
>sizeof(((struct btrfs_dev_item *)0))->total_bytes);
> - WARN_ON(!IS_ALIGNED(val, eb->fs_info->sectorsize));
>   btrfs_set_64(eb, s, offsetof(struct btrfs_dev_item, total_bytes), val);
>  }
>  
> diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
> index 0e8f16c305df..afae25df6a8c 100644
> --- a/fs/btrfs/volumes.c
> +++ b/fs/btrfs/volumes.c
> @@ -6472,15 +6472,23 @@ static int read_one_chunk(struct btrfs_fs_info 
> *fs_info, struct btrfs_key *key,
>   return 0;
>  }
>  
> -static void fill_device_from_item(struct extent_buffer *leaf,
> -  struct btrfs_dev_item *dev_item,
> -  struct btrfs_device *device)
> +static void fill_device_from_item(struct btrfs_fs_info *fs_info,
> +   struct extent_buffer *leaf,
> +   struct btrfs_dev_item *dev_item,
> +   struct btrfs_device *device)
>  {
>   unsigned long ptr;
>  
>   device->devid = btrfs_device_id(leaf, dev_item);
>   device->disk_total_bytes = btrfs_device_total_bytes(leaf, dev_item);
>   device->total_bytes = device->disk_total_bytes;
> + if (!IS_ALIGNED(device->total_bytes, fs_info->sectorsize)) {
> + btrfs_warn(fs_info,
> +"devid %llu has unaligned total bytes %llu",
> +device->devid, device->disk_total_bytes);
> + btrfs_warn(fs_info,
> +"please shrink the device a little and resize back 
> to fix it");
> + }

How about telling uses to know device->total_bytes should be alligned
to fs_info->sectorsize here?

Thanks,
Satoru

>   device->commit_total_bytes = device->disk_total_bytes;
>   device->bytes_used = btrfs_device_bytes_used(leaf, dev_item);
>   device->commit_bytes_used = device->bytes_used;
> @@ -6625,7 +6633,7 @@ static int read_one_dev(struct btrfs_fs_info *fs_info,
>   return -EINVAL;
>   }
>  
> - fill_device_from_item(leaf, dev_item, device);
> + fill_device_from_item(fs_info, leaf, dev_item, device);
>   device->in_fs_metadata = 1;
>   if (device->writeable && !device->is_tgtdev_for_dev_replace) {
>   device->fs_devices->total_rw_bytes += device->total_bytes;
> -- 
> 2.14.1
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Btrfs: use btrfs_op instead of bio_op in __btrfs_map_block

[Satoru Takeuchi]

At Tue, 19 Sep 2017 17:50:09 -0600,
Liu Bo wrote:
> 
> This seems to be a leftover of commit cf8cddd38bab ("btrfs: don't
> abuse REQ_OP_* flags for btrfs_map_block").
> 
> It should use btrfs_op() helper to provide one of 'enum btrfs_map_op'
> types.
> 
> Signed-off-by: Liu Bo <bo.li....@oracle.com>

Reviewed-by: Satoru Takeuchi <satoru.takeu...@gmail.com>

I checked all callers of the following functions and there is no leftover.

- btrfs_map_block
- btrfs_map_sblock
- __btrfs_map_block

Thanks,
Satoru

> ---
>  fs/btrfs/volumes.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
> index bd679bc..bd7b75d3 100644
> --- a/fs/btrfs/volumes.c
> +++ b/fs/btrfs/volumes.c
> @@ -6229,7 +6229,7 @@ blk_status_t btrfs_map_bio(struct btrfs_fs_info 
> *fs_info, struct bio *bio,
>   map_length = length;
>  
>   btrfs_bio_counter_inc_blocked(fs_info);
> - ret = __btrfs_map_block(fs_info, bio_op(bio), logical,
> + ret = __btrfs_map_block(fs_info, btrfs_op(bio), logical,
>   _length, , mirror_num, 1);
>   if (ret) {
>   btrfs_bio_counter_dec(fs_info);
> -- 
> 2.9.4
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] btrfs-progs: subvolume: outputs message only when operation succeeds

[Satoru Takeuchi]

At Tue, 19 Sep 2017 16:41:52 +0900,
Misono, Tomohiro wrote:
> 
> "btrfs subvolume create/delete" outputs the message of "Create/Delete
> subvolume ..." even when an operation fails.
> Since it is confusing, let's outputs the message only when an operation 
> succeeds.
> 
> Signed-off-by: Tomohiro Misono <misono.tomoh...@jp.fujitsu.com>

Current message as follows is odd as you said. 

```
Create subvolume './test'
ERROR: cannot create subvolume: No such file or directory
```

It's ambiguous for users to know whether creating subvolume succeeded or not.

I tested this patch with injecting error on ioctl() for subvol 
creation/deletion.

Reviewed-by: Satoru Takeuchi <satoru.takeu...@gmail.com>
Tested-by: Satoru Takeuchi <satoru.takeu...@gmail.com>

> ---
>  cmds-subvolume.c | 8 
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/cmds-subvolume.c b/cmds-subvolume.c
> index 666f6e0..6d4b0fe 100644
> --- a/cmds-subvolume.c
> +++ b/cmds-subvolume.c
> @@ -189,7 +189,6 @@ static int cmd_subvol_create(int argc, char **argv)
>   if (fddst < 0)
>   goto out;
>  
> - printf("Create subvolume '%s/%s'\n", dstdir, newname);
>   if (inherit) {
>   struct btrfs_ioctl_vol_args_v2  args;
>  
> @@ -213,6 +212,7 @@ static int cmd_subvol_create(int argc, char **argv)
>   error("cannot create subvolume: %s", strerror(errno));
>   goto out;
>   }
> + printf("Create subvolume '%s/%s'\n", dstdir, newname);
>  
>   retval = 0; /* success */
>  out:
> @@ -337,9 +337,6 @@ again:
>   goto out;
>   }
>  
> - printf("Delete subvolume (%s): '%s/%s'\n",
> - commit_mode == 2 || (commit_mode == 1 && cnt + 1 == argc)
> - ? "commit" : "no-commit", dname, vname);
>   memset(, 0, sizeof(args));
>   strncpy_null(args.name, vname);
>   res = ioctl(fd, BTRFS_IOC_SNAP_DESTROY, );
> @@ -349,6 +346,9 @@ again:
>   ret = 1;
>   goto out;
>   }
> + printf("Delete subvolume (%s): '%s/%s'\n",
> + commit_mode == 2 || (commit_mode == 1 && cnt + 1 == argc)
> + ? "commit" : "no-commit", dname, vname);
>  
>   if (commit_mode == 1) {
>   res = wait_for_commit(fd);
> -- 
> 2.9.5
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] btrfs-progs: allow "none" to disable compression for convenience

[Satoru Takeuchi]

At Tue, 19 Sep 2017 17:14:27 +0200,
David Sterba wrote:
> 
> On Mon, Sep 18, 2017 at 09:41:17AM +0900, Satoru Takeuchi wrote:
> > At Sun, 17 Sep 2017 14:08:40 +0100,
> > Mike Fleetwood wrote:
> > > 
> > > On 17 September 2017 at 01:36, Satoru Takeuchi
> > > <satoru.takeu...@gmail.com> wrote:
> > > > It's messy to use "" to disable compression. Introduce the new value 
> > > > "no"
> > > > which can also be used for this purpose.
> > > 
> > > From an English language point of view, "none" would be better.  None
> > > says the absence of, where as no is more general negative.
> > 
> > Thank you for your comment. How about is it?
> > 
> > ---
> > It's messy to use "" to disable compression. Introduce the new value "none"
> > which can also be used for this purpose.
> 
> I'd allow both values, 'no' and 'none', similar to the mount options,
> that also accept both (technically, the 'no' + anything is accepted for
> disabling compression).

As a result of reading "man 5 btrfs", now I prefer "no". It's used
to mean disabling compression there. On the other hand, "none" is
not used at all.

From man 5 btrfs:
===
...
FILE ATTRIBUTES
...
   compress, compress=type, compress-force, compress-force=type
   (default: off)

   Control BTRFS file data compression. Type may be specified as zlib, 
lzo or no (for no compression, used for remounting). If no type is specified, 
zlib is used. If
   compress-force is specified, all files will be compressed, whether 
or not they compress well.
...
   X
   no compression, permanently turn off compression on the given file, 
other compression mount options will not affect that
...
===

So David, please apply my v1 patcth if it looks good for you.

Thanks,
Satoru
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Btrfs Issues

[Satoru Takeuchi]

At Wed, 13 Sep 2017 11:53:35 -0400,
Ruoxin Jiang wrote:
> 
> [1  ]
> Hello,
> 
> We are researchers from Columbia University, New York. As part of our
> current research we have found some semantic discrepancies between
> btrfs and other popular filesystems.
> 
> We have attached two cases. The first one involves an invalid O_DIRECT
> write() that fails back to buffered write instead of failing with
> error EINVAL. In directory 2, we discovered that btrfs calculates
> write_bytes in __btrfs_buffered_write differently from that in
> generic_perform_writes in fs/mmap.c. This can cause inconsistent
> behavior between btrfs and other filesystems when program invokes the
> same writev/write() syscall.
> 
> In each directory, you will find a Readme.md describing the issue and
> pointing to the code that may cause the problem. For your convenience,
> we also included test programs (min.cpp) and instructions in Readme to
> help reproduce the issues.
> 
> We would appreciate very much if you could confirm the two cases at
> your conveniences.

I took a look at your test programs, btrfs_issues/{1,2}/min.cpp. It looks
very hard to read since you call syscalls in odd ways and all flags are
hardcoded as literal hexadecimal numbers. Could rewrite these program
to improve readability?

In addition, I have two questions about btrfs_issues/1/min.cpp.

1. Why you set 'filename' as the 1st argument of mmap()?
2. What's the purpose of mmap() call? I guess mmap() is not related to issue 1.

Thanks,
Satoru

> 
> Thanks,
> Amy
> [2 btrfs_issues.tar.gz ]
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] btrfs-progs: Output time elapsed for each major tree it checked

[Satoru Takeuchi]

At Mon, 18 Sep 2017 09:47:29 +0900,
Qu Wenruo wrote:
> 
> Marc reported that "btrfs check --repair" runs much faster than "btrfs
> check", which is quite weird.
> 
> This patch will add time elapsed for each major tree it checked, for
> both original mode and lowmem mode, so we can have a clue what's going
> wrong.
> 
> Reported-by: Marc MERLIN <m...@merlins.org>
> Signed-off-by: Qu Wenruo <quwenruo.bt...@gmx.com>
> ---
> v2:
>   Add prefix for each time report, as error message may make the output
>   hard to understand. Suggested by Satoru.

Reviewed-by: Satoru Takeuchi <satoru.takeu...@gmail.com>

> ---
>  cmds-check.c | 21 +++--
>  utils.h  | 24 
>  2 files changed, 43 insertions(+), 2 deletions(-)
> 
> diff --git a/cmds-check.c b/cmds-check.c
> index 006edbde..f1074c73 100644
> --- a/cmds-check.c
> +++ b/cmds-check.c
> @@ -5318,13 +5318,16 @@ static int do_check_fs_roots(struct btrfs_fs_info 
> *fs_info,
> struct cache_tree *root_cache)
>  {
>   int ret;
> + struct timer timer;
>  
>   if (!ctx.progress_enabled)
>   fprintf(stderr, "checking fs roots\n");
> + start_timer();
>   if (check_mode == CHECK_MODE_LOWMEM)
>   ret = check_fs_roots_v2(fs_info);
>   else
>   ret = check_fs_roots(fs_info, root_cache);
> + printf("fs roots checked in %d seconds\n", stop_timer());
>  
>   return ret;
>  }
> @@ -11584,14 +11587,16 @@ out:
>  static int do_check_chunks_and_extents(struct btrfs_fs_info *fs_info)
>  {
>   int ret;
> + struct timer timer;
>  
>   if (!ctx.progress_enabled)
>   fprintf(stderr, "checking extents\n");
> + start_timer();
>   if (check_mode == CHECK_MODE_LOWMEM)
>   ret = check_chunks_and_extents_v2(fs_info);
>   else
>   ret = check_chunks_and_extents(fs_info);
> -
> + printf("extents checked in %d seconds\n", stop_timer());
>   return ret;
>  }
>  
> @@ -12772,6 +12777,7 @@ int cmd_check(int argc, char **argv)
>   int qgroups_repaired = 0;
>   unsigned ctree_flags = OPEN_CTREE_EXCLUSIVE;
>   int force = 0;
> + struct timer timer;
>  
>   while(1) {
>   int c;
> @@ -12953,8 +12959,11 @@ int cmd_check(int argc, char **argv)
>   if (repair)
>   ctree_flags |= OPEN_CTREE_PARTIAL;
>  
> + printf("opening btrfs filesystem\n");
> + start_timer();
>   info = open_ctree_fs_info(argv[optind], bytenr, tree_root_bytenr,
> chunk_root_bytenr, ctree_flags);
> + printf("btrfs filesystem opened in %d seconds\n", stop_timer());
>   if (!info) {
>   error("cannot open file system");
>   ret = -EIO;
> @@ -13115,8 +13124,10 @@ int cmd_check(int argc, char **argv)
>   else
>   fprintf(stderr, "checking free space cache\n");
>   }
> + start_timer();
>   ret = check_space_cache(root);
>   err |= !!ret;
> + printf("free space cache (tree) checked in %d seconds\n", 
> stop_timer());
>   if (ret) {
>   if (btrfs_fs_compat_ro(info, FREE_SPACE_TREE))
>   error("errors found in free space tree");
> @@ -13140,18 +13151,22 @@ int cmd_check(int argc, char **argv)
>   }
>  
>   fprintf(stderr, "checking csums\n");
> + start_timer();
>   ret = check_csums(root);
>   err |= !!ret;
> + printf("csums tree checked in %d seconds\n", stop_timer());
>   if (ret) {
>   error("errors found in csum tree");
>   goto out;
>   }
>  
> - fprintf(stderr, "checking root refs\n");
>   /* For low memory mode, check_fs_roots_v2 handles root refs */
>   if (check_mode != CHECK_MODE_LOWMEM) {
> + fprintf(stderr, "checking root refs\n");
> + start_timer();
>   ret = check_root_refs(root, _cache);
>   err |= !!ret;
> + printf("root refs checked in %d seconds\n", stop_timer());
>   if (ret) {
>   error("errors found in root refs");
>   goto out;
> @@ -13186,8 +13201,10 @@ int cmd_check(int argc, char **argv)
>  
>   if (info->quota_enabled) {
>   fprintf(stderr, "checking quota groups\n");
> + start_timer();
>   ret = qgroup_verify_all(info);
&

[PATCH v2] btrfs-progs: allow "none" to disable compression for convenience

[Satoru Takeuchi]

At Sun, 17 Sep 2017 14:08:40 +0100,
Mike Fleetwood wrote:
> 
> On 17 September 2017 at 01:36, Satoru Takeuchi
> <satoru.takeu...@gmail.com> wrote:
> > It's messy to use "" to disable compression. Introduce the new value "no"
> > which can also be used for this purpose.
> 
> From an English language point of view, "none" would be better.  None
> says the absence of, where as no is more general negative.

Thank you for your comment. How about is it?

---
It's messy to use "" to disable compression. Introduce the new value "none"
which can also be used for this purpose.

Signed-off-by: Satoru Takeuchi <satoru.takeu...@gmail.com>
---
 Documentation/btrfs-property.asciidoc | 2 +-
 props.c   | 6 --
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/Documentation/btrfs-property.asciidoc 
b/Documentation/btrfs-property.asciidoc
index 7ed6a7d..786af9b 100644
--- a/Documentation/btrfs-property.asciidoc
+++ b/Documentation/btrfs-property.asciidoc
@@ -43,7 +43,7 @@ read-only flag of subvolume: true or false
 label
 label of device
 compression
-compression setting for an inode: lzo, zlib, zstd, or "" (empty string)
+compression setting for an inode: lzo, zlib, zstd, none, or "" (empty string). 
Both none and "" are for disabling compression.
 
 *list* [-t ] ::
 Lists available properties with their descriptions for the given object.
diff --git a/props.c b/props.c
index a7e3e96..8d85181 100644
--- a/props.c
+++ b/props.c
@@ -142,9 +142,11 @@ static int prop_compression(enum prop_object_type type,
memcpy(xattr_name + XATTR_BTRFS_PREFIX_LEN, name, strlen(name));
xattr_name[XATTR_BTRFS_PREFIX_LEN + strlen(name)] = '\0';
 
-   if (value)
+   if (value) {
+   if (!strcmp(value, "none"))
+   value = "";
sret = fsetxattr(fd, xattr_name, value, strlen(value), 0);
-   else
+   } else
sret = fgetxattr(fd, xattr_name, NULL, 0);
if (sret < 0) {
ret = -errno;
-- 
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Btrfs: cleanup 'start' subtraction from try uncompressed inline extent

[Satoru Takeuchi]

At Fri, 15 Sep 2017 01:57:26 +0300,
Timofey Titovets wrote:
> 
> Was added in:
>   c8b978188c9a0fd3d535c13debd19d522b726f1f
>   "Btrfs: Add zlib compression support"
> Survive to near time (from 08.10.2008).
> 
> Because 'start' checked for zero before branch, so it's
> safe to remove that subtraction.
> 
> Signed-off-by: Timofey Titovets <nefelim...@gmail.com>

Reviewed-by: Satoru Takeuchi <satoru.takeu...@gmail.com>

> ---
>  fs/btrfs/inode.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
> index 02ef32149c15..81123408e82e 100644
> --- a/fs/btrfs/inode.c
> +++ b/fs/btrfs/inode.c
> @@ -570,7 +570,7 @@ static noinline void compress_file_range(struct inode 
> *inode,
>  cont:
>   if (start == 0) {
>   /* lets try to make an inline extent */
> - if (ret || total_in < (actual_end - start)) {
> + if (ret || total_in < actual_end) {
>   /* we didn't compress the entire range, try
>* to make an uncompressed inline extent.
>*/
> -- 
> 2.14.1
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] btrfs-progs: Output time elapsed for each major tree it checked

[Satoru Takeuchi]

At Mon, 11 Sep 2017 14:26:23 +0900,
Qu Wenruo wrote:
> 
> Marc reported that "btrfs check --repair" runs much faster than "btrfs
> check", which is quite weird.
> 
> This patch will add time elapsed for each major tree it checked, for
> both original mode and lowmem mode, so we can have a clue what's going
> wrong.
> 
> Reported-by: Marc MERLIN 
> Signed-off-by: Qu Wenruo 
> ---
>  cmds-check.c | 21 +++--
>  utils.h  | 24 
>  2 files changed, 43 insertions(+), 2 deletions(-)
> 
> diff --git a/cmds-check.c b/cmds-check.c
> index 006edbde..fee806cd 100644
> --- a/cmds-check.c
> +++ b/cmds-check.c
> @@ -5318,13 +5318,16 @@ static int do_check_fs_roots(struct btrfs_fs_info 
> *fs_info,
> struct cache_tree *root_cache)
>  {
>   int ret;
> + struct timer timer;
>  
>   if (!ctx.progress_enabled)
>   fprintf(stderr, "checking fs roots\n");
> + start_timer();
>   if (check_mode == CHECK_MODE_LOWMEM)
>   ret = check_fs_roots_v2(fs_info);
>   else
>   ret = check_fs_roots(fs_info, root_cache);
> + printf("done in %d seconds\n", stop_timer());

How about adding the prefixes to show which part of check/repair is done
before showing the elaplsed time, like "checking fs roots done in XX seconds"
and "checking extents done in XX seconds"?

Current message, "done in XX seconds" for all parts, is hard to understand
which part takes such the time, especially when something wrong happens
in some parts. For example, please see the following example of repairing
the image based on tests/fsck-tests/002-bad-transid/default_case.img

===
...
opening btrfs filesystem
parent transid verify failed on 29360128 wanted 9 found 755944791
parent transid verify failed on 29360128 wanted 9 found 755944791
parent transid verify failed on 29360128 wanted 9 found 755944791
parent transid verify failed on 29360128 wanted 9 found 755944791
Ignoring transid failure
done in 0 seconds
...
checking free space cache
cache and super generation don't match, space cache will be invalidated
done in 0 seconds
...
===

Thanks,
Satoru

>  
>   return ret;
>  }
> @@ -11584,14 +11587,16 @@ out:
>  static int do_check_chunks_and_extents(struct btrfs_fs_info *fs_info)
>  {
>   int ret;
> + struct timer timer;
>  
>   if (!ctx.progress_enabled)
>   fprintf(stderr, "checking extents\n");
> + start_timer();
>   if (check_mode == CHECK_MODE_LOWMEM)
>   ret = check_chunks_and_extents_v2(fs_info);
>   else
>   ret = check_chunks_and_extents(fs_info);
> -
> + printf("done in %d seconds\n", stop_timer());
>   return ret;
>  }
>  
> @@ -12772,6 +12777,7 @@ int cmd_check(int argc, char **argv)
>   int qgroups_repaired = 0;
>   unsigned ctree_flags = OPEN_CTREE_EXCLUSIVE;
>   int force = 0;
> + struct timer timer;
>  
>   while(1) {
>   int c;
> @@ -12953,8 +12959,11 @@ int cmd_check(int argc, char **argv)
>   if (repair)
>   ctree_flags |= OPEN_CTREE_PARTIAL;
>  
> + printf("opening btrfs filesystem\n");
> + start_timer();
>   info = open_ctree_fs_info(argv[optind], bytenr, tree_root_bytenr,
> chunk_root_bytenr, ctree_flags);
> + printf("done in %d seconds\n", stop_timer());
>   if (!info) {
>   error("cannot open file system");
>   ret = -EIO;
> @@ -13115,8 +13124,10 @@ int cmd_check(int argc, char **argv)
>   else
>   fprintf(stderr, "checking free space cache\n");
>   }
> + start_timer();
>   ret = check_space_cache(root);
>   err |= !!ret;
> + printf("done in %d seconds\n", stop_timer());
>   if (ret) {
>   if (btrfs_fs_compat_ro(info, FREE_SPACE_TREE))
>   error("errors found in free space tree");
> @@ -13140,18 +13151,22 @@ int cmd_check(int argc, char **argv)
>   }
>  
>   fprintf(stderr, "checking csums\n");
> + start_timer();
>   ret = check_csums(root);
>   err |= !!ret;
> + printf("done in %d seconds\n", stop_timer());
>   if (ret) {
>   error("errors found in csum tree");
>   goto out;
>   }
>  
> - fprintf(stderr, "checking root refs\n");
>   /* For low memory mode, check_fs_roots_v2 handles root refs */
>   if (check_mode != CHECK_MODE_LOWMEM) {
> + fprintf(stderr, "checking root refs\n");
> + start_timer();
>   ret = check_root_refs(root, _cache);
>   err |= !!ret;
> + printf("done in %d seconds\n", stop_timer());
>   if (ret) {
>   error("errors found in root refs");
>   goto out;
> @@ -13186,8 +13201,10 @@ int cmd_check(int argc, char **argv)
>  
>   if (info->quota_enabled) {
>   fprintf(stderr, "checking 

[PATCH] btrfs-progs: allow "no" to disable compression for convenience

[Satoru Takeuchi]

It's messy to use "" to disable compression. Introduce the new value "no"
which can also be used for this purpose.

Signed-off-by: Satoru Takeuchi <satoru.takeu...@gmail.com>
---
 Documentation/btrfs-property.asciidoc | 2 +-
 props.c   | 6 --
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/Documentation/btrfs-property.asciidoc 
b/Documentation/btrfs-property.asciidoc
index 7ed6a7d..97b90d6 100644
--- a/Documentation/btrfs-property.asciidoc
+++ b/Documentation/btrfs-property.asciidoc
@@ -43,7 +43,7 @@ read-only flag of subvolume: true or false
 label
 label of device
 compression
-compression setting for an inode: lzo, zlib, zstd, or "" (empty string)
+compression setting for an inode: lzo, zlib, zstd, no, or "" (empty string). 
Both no and "" are for disabling compression.
 
 *list* [-t ] ::
 Lists available properties with their descriptions for the given object.
diff --git a/props.c b/props.c
index a7e3e96..a2df868 100644
--- a/props.c
+++ b/props.c
@@ -142,9 +142,11 @@ static int prop_compression(enum prop_object_type type,
memcpy(xattr_name + XATTR_BTRFS_PREFIX_LEN, name, strlen(name));
xattr_name[XATTR_BTRFS_PREFIX_LEN + strlen(name)] = '\0';
 
-   if (value)
+   if (value) {
+   if (!strcmp(value, "no"))
+   value = "";
sret = fsetxattr(fd, xattr_name, value, strlen(value), 0);
-   else
+   } else
sret = fgetxattr(fd, xattr_name, NULL, 0);
if (sret < 0) {
ret = -errno;
-- 
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] btrfs: prevent to set invalid default subvolid

[Satoru Takeuchi]

`btrfs sub set-default` succeeds to set an ID which isn't corresponding to any
fs/file tree. If such the bad ID is set to a filesystem, we can't mount this
filesystem without specifying `subvol` or `subvolid` mount options.

Signed-off-by: Satoru Takeuchi <satoru.takeu...@gmail.com>
---
 fs/btrfs/ioctl.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
index ae8fbf9..8e1bc19 100644
--- a/fs/btrfs/ioctl.c
+++ b/fs/btrfs/ioctl.c
@@ -4057,6 +4057,10 @@ static long btrfs_ioctl_default_subvol(struct file 
*file, void __user *argp)
ret = PTR_ERR(new_root);
goto out;
}
+   if (!is_fstree(new_root->objectid)) {
+   ret = -ENOENT;
+   goto out;
+   }
 
path = btrfs_alloc_path();
if (!path) {
-- 
2.7.4
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] btrfs: convert all mount option checking code to use btrfs_test_opt

[Satoru Takeuchi]

Signed-off-by: Satoru Takeuchi <satoru.takeu...@gmail.com>
---
 fs/btrfs/extent-tree.c | 2 +-
 fs/btrfs/super.c   | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index e2d7e86..ea9b0d2 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -652,7 +652,7 @@ static int cache_block_group(struct btrfs_block_group_cache 
*cache,
cache->cached = BTRFS_CACHE_FAST;
spin_unlock(>lock);
 
-   if (fs_info->mount_opt & BTRFS_MOUNT_SPACE_CACHE) {
+   if (btrfs_test_opt(fs_info, SPACE_CACHE)) {
mutex_lock(_ctl->mutex);
ret = load_free_space_cache(fs_info, cache);
 
diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
index 0b7a1d8..26f0a38 100644
--- a/fs/btrfs/super.c
+++ b/fs/btrfs/super.c
@@ -292,7 +292,7 @@ void __btrfs_panic(struct btrfs_fs_info *fs_info, const 
char *function,
vaf.va = 
 
errstr = btrfs_decode_error(errno);
-   if (fs_info && (fs_info->mount_opt & BTRFS_MOUNT_PANIC_ON_FATAL_ERROR))
+   if (fs_info && (btrfs_test_opt(fs_info, PANIC_ON_FATAL_ERROR)))
panic(KERN_CRIT "BTRFS panic (device %s) in %s:%d: %pV 
(errno=%d %s)\n",
s_id, function, line, , errno, errstr);
 
-- 
2.7.4
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] fix inconsistent device between /proc/pid/maps and stat

[Satoru Takeuchi]

2017-03-24 20:58 GMT+09:00 David Sterba <dste...@suse.cz>:
> On Tue, Mar 21, 2017 at 10:53:09AM +0900, Satoru Takeuchi wrote:
>> There have been some discussions about inconsistent device between 
>> /proc/pid/maps and stat(2).
>>
>> http://thr3ads.net/btrfs-devel/2011/05/2346176-RFC-PATCH-0-2-btrfs-vfs-Return-same-device-in-stat-2-and-proc-pid-maps
>> https://www.spinics.net/lists/linux-btrfs/msg09044.html
>> https://patchwork.kernel.org/patch/2825842/
>> https://patchwork.kernel.org/patch/2831525/
>>
>> It's important since it breaks user programs like lsof(8). There was a 
>> patche by Mark to fix this problem.
>> However, unfortunately, that patch is not merged so far.
>
> And no variant of the get_map_dev will ever be merged. Reworking this
> requires extensions to the superblock and subvolume structures, making
> it more generic and suitable for VFS.

Thank you for your comment. I'm going to reconsider how to fix this problem.

Thanks,
Satoru
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] fix inconsistent device between /proc/pid/maps and stat

[Satoru Takeuchi]

There have been some discussions about inconsistent device between 
/proc/pid/maps and stat(2).

http://thr3ads.net/btrfs-devel/2011/05/2346176-RFC-PATCH-0-2-btrfs-vfs-Return-same-device-in-stat-2-and-proc-pid-maps
https://www.spinics.net/lists/linux-btrfs/msg09044.html
https://patchwork.kernel.org/patch/2825842/
https://patchwork.kernel.org/patch/2831525/

It's important since it breaks user programs like lsof(8). There was a patche 
by Mark to fix this problem.
However, unfortunately, that patch is not merged so far.

  NOTE:
  1) This patch is inspired by the Mark's one and I tweak it as follows.
 - move a flag for this fix from sb->s_flags to kernel internal sb->s_iflags
 - change that flag's name from MS_STAT_FOR_DEV to SB_I_LOGICALVOL, to make 
its meaning clearer
  2) This patch is for Chris's for-linus-4.11 (commit 
e1699d2d7bf6e6cce3e1baff19f9dd4595a58664 ("")). It should modify to apply to 
4.11-rcX because of statx patches changes
 struct inode_operations->getattr() interface.

For more information about this problem, please refer to the patchat the end of 
this mail.

---
Subject: [PATCH] fix inconsistent devie between /proc/pid/maps and stat

/proc/pid/maps returns each device as follows.

===
dev = inode->i_sb->s_dev;
===

However, stat(2) returns it as follows.

===
stat->dev = BTRFS_I(inode)->root->anon_dev;
===

It results in device mismatch and this inconsistency breaks users programs like 
lsof(8) as follows.

Without this patch:
===
$ mount | grep /home/vagrant/mnt
/dev/vda5 on /home/vagrant/mnt type btrfs 
(rw,relatime,noacl,space_cache,subvolid=5,subvol=/)
$ /home/vagrant/mnt/lsof | grep /home/vagrant/mnt
lsof  19292  root  txt   REG   0,44   
163136257 /home/sat/mnt/lsof
lsof  19292  root  mem   REG   0,42 
257 /home/sat/mnt/lsof (path dev=0,44)
lsof  19294  root  txt   REG   0,44   
163136257 /home/sat/mnt/lsof
lsof  19294  root  mem   REG   0,42 
257 /home/sat/mnt/lsof (path dev=0,44)
===

With this patch:
===
$ mount | grep /home/vagrant/mnt
/dev/vda5 on /home/vagrant/mnt type btrfs 
(rw,relatime,noacl,space_cache,subvolid=5,subvol=/)
$ /home/vagrant/mnt/lsof | grep /home/vagrant/mnt
lsof  752 root  txt   REG   0,44   163224   
 263 /home/vagrant/mnt/lsof
lsof  754 root  txt   REG   0,44   163224   
 263 /home/vagrant/mnt/lsof
===

Signed-off-by: Satoru Takeuchi <satoru.takeu...@gmail.com>
CC: Signed-off-by: Mark Fasheh <mfas...@suse.de>
---
 fs/btrfs/super.c |  1 +
 fs/proc/generic.c| 13 +
 fs/proc/internal.h   |  1 +
 fs/proc/nommu.c  |  2 +-
 fs/proc/task_mmu.c   |  2 +-
 fs/proc/task_nommu.c |  2 +-
 include/linux/fs.h   |  1 +
 7 files changed, 19 insertions(+), 3 deletions(-)

diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
index da687dc79cce..2ccfdb107ba0 100644
--- a/fs/btrfs/super.c
+++ b/fs/btrfs/super.c
@@ -1133,6 +1133,7 @@ static int btrfs_fill_super(struct super_block *sb,
 #endif
sb->s_flags |= MS_I_VERSION;
sb->s_iflags |= SB_I_CGROUPWB;
+   sb->s_iflags |= SB_I_LOGICALVOL;
err = open_ctree(sb, fs_devices, (char *)data);
if (err) {
btrfs_err(fs_info, "open_ctree failed");
diff --git a/fs/proc/generic.c b/fs/proc/generic.c
index f6a01f09f79d..d38cd77b297c 100644
--- a/fs/proc/generic.c
+++ b/fs/proc/generic.c
@@ -646,3 +646,16 @@ void *PDE_DATA(const struct inode *inode)
return __PDE_DATA(inode);
 }
 EXPORT_SYMBOL(PDE_DATA);
+
+dev_t proc_get_map_dev(struct dentry *dentry)
+{
+   struct inode *inode = dentry->d_inode;
+   struct kstat kstat;
+
+   if (inode->i_sb->s_iflags & SB_I_LOGICALVOL &&
+   inode->i_op->getattr &&
+   inode->i_op->getattr(NULL, dentry, ) == 0)
+   return kstat.dev;
+
+   return inode->i_sb->s_dev;
+}
diff --git a/fs/proc/internal.h b/fs/proc/internal.h
index 2de5194ba378..bf0f11fc209b 100644
--- a/fs/proc/internal.h
+++ b/fs/proc/internal.h
@@ -190,6 +190,7 @@ static inline struct proc_dir_entry *pde_get(struct 
proc_dir_entry *pde)
return pde;
 }
 extern void pde_put(struct proc_dir_entry *);
+dev_t proc_get_map_dev(struct dentry *);
 
 static inline bool is_empty_pde(const struct proc_dir_entry *pde)
 {
diff --git a/fs/proc/nommu.c b/fs/proc/nommu.c
index 75634379f82e..e9c29864a50e 100644
--- a/fs/proc/nommu.c
+++ b/fs/proc/nommu.c
@@ -46,7 +46,7 @@ static int nommu_region_show(struct seq_file *m, struct 
vm_region *region)
 
if (file) {
struct inode *inode = file_inode(region->vm_file);
-   dev = inode->i_sb->s_dev;
+   dev = proc_get_map_dev(vma->vm_file

Re: [PATCH v12.1 01/15] btrfs: expand cow_file_range() to support in-band dedup and subpage-blocksize

[Satoru Takeuchi]

On 2016/07/12 1:41, David Sterba wrote:
> On Mon, Jul 11, 2016 at 11:05:29AM +0800, Qu Wenruo wrote:
>> From: Wang Xiaoguang 
>>
>> Extract cow_file_range() new parameters for both in-band dedupe and
>> subpage sector size patchset.
>>
>> This should make conflict of both patchset to minimal, and reduce the
>> effort needed to rebase them.
> 
> Great, thanks. I did a test merge, there are still conflicts but they
> seem to be resolvable more easily, picking the changes from the right
> patchset. I haven't tested it so there's more work, but the point was to
> get the conflict surface down.
> 
> There's another candidate, btrfs_set_extent_delalloc as it adds a
> parameter, can you please send a similar patch for that?

He's going to attend LinuxCon Japan (Jul 13 - Jul 15).
So I guess he will reply several days later (next week?).

Thanks,
Satoru

> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] btrfs-progs: add option to run balance as daemon

[Satoru Takeuchi]

On 2016/06/22 0:16, Austin S. Hemmelgarn wrote:
> Currently, balance operations are run synchronously in the foreground.
> This is nice for interactive management, but is kind of crappy when you
> start looking at automation and similar things.
> 
> This patch adds an option to `btrfs balance start` to tell it to
> daemonize prior to running the balance operation, thus allowing us to
> preform balances asynchronously.  The two biggest use cases I have for
> this are starting a balance on a remote server without establishing a
> full shell session, and being able to background the balance in a
> recovery shell (which usually has no job control) so I can still get
> progress information.
> 
> Because it simply daemonizes prior to calling the balance ioctl, this
> doesn't actually need any kernel support.
> 
> Signed-off-by: Austin S. Hemmelgarn 
> ---
> This works as is, but there are two specific things I would love to
> eventually fix but don't have the time to fix right now:
> * There is no way to get any feedback from the balance operation.
> * Because of how everything works, trying to start a new balance with
>   --background while one iw already running won't return an error but
>   won't queue or start a new balance either.
> 
> The first one is more a utility item than anything else, and probably
> would not be hard to add.  Ideally, it should be output to a user
> specified file, and this should work even for a normal foreground balance.
> 
> The second is very much a UX issue, but can't be easily sovled without
> doing some creative process monitoring from the parrent processes.
> 
>  Documentation/btrfs-balance.asciidoc |  2 ++
>  cmds-balance.c   | 43 
> +++-
>  2 files changed, 44 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/btrfs-balance.asciidoc 
> b/Documentation/btrfs-balance.asciidoc
> index 7df40b9..f487dbb 100644
> --- a/Documentation/btrfs-balance.asciidoc
> +++ b/Documentation/btrfs-balance.asciidoc
> @@ -85,6 +85,8 @@ act on system chunks (requires '-f'), see `FILTERS` section 
> for details about 'f
>  be verbose and print balance filter arguments
>  -f
>  force reducing of metadata integrity, eg. when going from 'raid1' to 'single'
> +--background
> +run the balance operation asynchronously in the background
>  
>  *status* [-v] ::
>  Show status of running or paused balance.
> diff --git a/cmds-balance.c b/cmds-balance.c
> index 708bbf4..66169b7 100644
> --- a/cmds-balance.c
> +++ b/cmds-balance.c
> @@ -20,6 +20,9 @@
>  #include 
>  #include 
>  #include 
> +#include 
> +#include 
> +#include 
>  #include 
>  
>  #include "kerncompat.h"
> @@ -510,6 +513,7 @@ static const char * const cmd_balance_start_usage[] = {
>   "-v be verbose",
>   "-f force reducing of metadata integrity",
>   "--full-balance do not print warning and do not delay start",
> + "--background   run the balance as a background process",
>   NULL
>  };
>  
> @@ -520,6 +524,7 @@ static int cmd_balance_start(int argc, char **argv)
>   , NULL };
>   int force = 0;
>   int verbose = 0;
> + int background = 0;
>   unsigned start_flags = 0;
>   int i;
>  
> @@ -527,7 +532,8 @@ static int cmd_balance_start(int argc, char **argv)
>  
>   optind = 1;
>   while (1) {
> - enum { GETOPT_VAL_FULL_BALANCE = 256 };
> + enum { GETOPT_VAL_FULL_BALANCE = 256,
> + GETOPT_VAL_BACKGROUND = 257 };
>   static const struct option longopts[] = {
>   { "data", optional_argument, NULL, 'd'},
>   { "metadata", optional_argument, NULL, 'm' },
> @@ -536,6 +542,8 @@ static int cmd_balance_start(int argc, char **argv)
>   { "verbose", no_argument, NULL, 'v' },
>   { "full-balance", no_argument, NULL,
>   GETOPT_VAL_FULL_BALANCE },
> + { "background", no_argument, NULL,
> + GETOPT_VAL_BACKGROUND },
>   { NULL, 0, NULL, 0 }
>   };
>  
> @@ -574,6 +582,9 @@ static int cmd_balance_start(int argc, char **argv)
>   case GETOPT_VAL_FULL_BALANCE:
>   start_flags |= BALANCE_START_NOWARN;
>   break;
> + case GETOPT_VAL_BACKGROUND:
> + background = 1;
> + break;
>   default:
>   usage(cmd_balance_start_usage);
>   }
> @@ -626,6 +637,36 @@ static int cmd_balance_start(int argc, char **argv)
>   args.flags |= BTRFS_BALANCE_FORCE;
>   if (verbose)
>   dump_ioctl_balance_args();
> + if (background) {
> + switch (fork()) {
> + case (-1):
> + error("Unable to fork to run balance in 

[PATCH] btrfs-progs: dedupe-inband enable/reconfigure: force option does not take argument

[Satoru Takeuchi]

---
This patch can be applied to integration-20160704(2355a7e5dcdf122d1924)
---
 cmds-dedupe-ib.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/cmds-dedupe-ib.c b/cmds-dedupe-ib.c
index 342784c..dbb30ab 100644
--- a/cmds-dedupe-ib.c
+++ b/cmds-dedupe-ib.c
@@ -132,7 +132,7 @@ static int enable_reconfig_dedupe(int argc, char **argv, 
int reconf)
{ "hash-algorithm", required_argument, NULL, 'a'},
{ "limit-hash", required_argument, NULL, 'l'},
{ "limit-memory", required_argument, NULL, 'm'},
-   { "force", required_argument, NULL, 'f'},
+   { "force", no_argument, NULL, 'f'},
{ NULL, 0, NULL, 0}
};

-- 
2.5.5
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] btrfs-progs: fi show: print error message if no valid Btrfs is specified

[Satoru Takeuchi]

* Before this patch

 ===
 # ./btrfs fi show foo  # "foo" doesn't mean any valid Btrfs
 #  # no error message
 # echo $?
 1
 ===

* After this patch

 ===
 # ./btrfs fi show foo
 ERROR: foo is not a valid Btrfs
 #
 # echo $?
 1
 ===

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
 cmds-filesystem.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/cmds-filesystem.c b/cmds-filesystem.c
index 88867a3..90f3c49 100644
--- a/cmds-filesystem.c
+++ b/cmds-filesystem.c
@@ -898,9 +898,10 @@ devs_only:
list_for_each_entry(fs_devices, _uuids, list)
print_one_uuid(fs_devices, unit_mode);

-   if (search && !found)
+   if (search && !found) {
+   error("%s is not a valid Btrfs", search);
ret = 1;
-
+   }
while (!list_empty(_uuids)) {
fs_devices = list_entry(all_uuids.next,
struct btrfs_fs_devices, list);
-- 
2.5.5
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v11 00/13] Btrfs dedupe framework

[Satoru Takeuchi]

disk backend bug
>>   Add handle for multiple hash on same bytenr corner case to fix abort
>>   trans error
>>   Increase dedup rate by enhancing delayed ref handler for both backend.
>>   Move dedup_add() to run_delayed_ref() time, to fix abort trans error.
>>   Increase dedup block size up limit to 8M.
>> v4:
>>   Add dedup prop for disabling dedup for given files/dirs.
>>   Merge inmem_search() and ondisk_search() into generic_search() to save
>>   some code
>>   Fix another delayed_ref related bug.
>>   Use the same mutex for both inmem and ondisk backend.
>>   Move dedup_add() back to btrfs_finish_ordered_io() to increase dedup
>>   rate.
>> v5:
>>   Reuse compress routine for much simpler dedup function.
>>   Slightly improved performance due to above modification.
>>   Fix race between dedup enable/disable
>>   Fix for false ENOSPC report
>> v6:
>>   Further enable/disable race window fix.
>>   Minor format change according to checkpatch.
>> v7:
>>   Fix one concurrency bug with balance.
>>   Slightly modify return value from -EINVAL to -EOPNOTSUPP for
>>   btrfs_dedup_ioctl() to allow progs to distinguish unsupported commands
>>   and wrong parameter.
>>   Rebased to integration-4.6.
>> v8:
>>   Rename 'dedup' to 'dedupe'.
>>   Add support to allow dedupe and compression work at the same time.
>>   Fix several balance related bugs. Special thanks to Satoru Takeuchi,
>>   who exposed most of them.
>>   Small dedupe hit case performance improvement.
>> v9:
>>   Re-order the patchset to completely separate pure in-memory and any
>>   on-disk format change.
>>   Fold bug fixes into its original patch.
>> v10:
>>   Adding back missing bug fix patch.
>>   Reduce on-disk item size.
>>   Hide dedupe ioctl under CONFIG_BTRFS_DEBUG.
>> v11:
>>   Remove other backend and props support to focus on the framework and
>>   in-memory backend. Suggested by David.
>>   Better disable and buffered write race protection.
>>   Comprehensive fix to dedupe metadata ENOSPC problem.
>>
>> Qu Wenruo (3):
>>   btrfs: delayed-ref: Add support for increasing data ref under spinlock
>>   btrfs: dedupe: Inband in-memory only de-duplication implement
>>   btrfs: relocation: Enhance error handling to avoid BUG_ON
>>
>> Wang Xiaoguang (10):
>>   btrfs: dedupe: Introduce dedupe framework and its header
>>   btrfs: dedupe: Introduce function to initialize dedupe info
>>   btrfs: dedupe: Introduce function to add hash into in-memory tree
>>   btrfs: dedupe: Introduce function to remove hash from in-memory tree
>>   btrfs: dedupe: Introduce function to search for an existing hash
>>   btrfs: dedupe: Implement btrfs_dedupe_calc_hash interface
>>   btrfs: ordered-extent: Add support for dedupe
>>   btrfs: dedupe: Add ioctl for inband dedupelication
>>   btrfs: improve inode's outstanding_extents computation
>>   btrfs: dedupe: fix false ENOSPC
>>
>>  fs/btrfs/Makefile   |   2 +-
>>  fs/btrfs/ctree.h|  25 +-
>>  fs/btrfs/dedupe.c   | 710 
>> 
>>  fs/btrfs/dedupe.h   | 210 +
>>  fs/btrfs/delayed-ref.c  |  30 +-
>>  fs/btrfs/delayed-ref.h  |   8 +
>>  fs/btrfs/disk-io.c  |   4 +
>>  fs/btrfs/extent-tree.c  |  83 +-
>>  fs/btrfs/extent_io.c|  63 +++-
>>  fs/btrfs/extent_io.h|  15 +-
>>  fs/btrfs/file.c |  26 +-
>>  fs/btrfs/free-space-cache.c |   5 +-
>>  fs/btrfs/inode-map.c|   4 +-
>>  fs/btrfs/inode.c| 434 ++-
>>  fs/btrfs/ioctl.c|  80 -
>>  fs/btrfs/ordered-data.c |  46 ++-
>>  fs/btrfs/ordered-data.h |  14 +
>>  fs/btrfs/relocation.c   |  46 ++-
>>  fs/btrfs/sysfs.c|   2 +
>>  include/uapi/linux/btrfs.h  |  41 +++
>>  20 files changed, 1701 insertions(+), 147 deletions(-)
>>  create mode 100644 fs/btrfs/dedupe.c
>>  create mode 100644 fs/btrfs/dedupe.h
>>
> 
> 
> -- 
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Confusing output from fi us/df

[Satoru Takeuchi]

On 2016/06/21 8:30, Marc Grondin wrote:
> Hi everyone,
> 
> 
> I have a btrfs filesystem ontop of a 4x1tb mdraid raid5 array and I've
> been getting confusing output on metadata usage. Seems that even tho
> both data and metadata are in single profile metadata is reporting
> double the space(as if it was in dupe profile)
> 
> 
> 
> root@thebeach /h/marcg> uname -a
> Linux thebeach 4.6.2-gentoo-GMAN #1 SMP Sat Jun 11 22:32:27 ADT 2016
> x86_64 Intel(R) Core(TM) i5-2320 CPU @ 3.00GHz GenuineIntel GNU/Linux
> root@thebeach /h/marcg> btrfs --version
> btrfs-progs v4.5.3
> root@thebeach /h/marcg> btrfs fi us /media/Storage2
> Overall:
> Device size: 2.73TiB
> Device allocated: 1.71TiB
> Device unallocated: 1.02TiB
> Device missing: 0.00B
> Used: 1.38TiB
> Free (estimated): 1.34TiB (min: 1.34TiB)
> Data ratio: 1.00
> Metadata ratio: 1.00
> Global reserve: 512.00MiB (used: 0.00B)
> 
> 
> Data,single: Size:1.71TiB, Used:1.38TiB
> /dev/mapper/storage2 1.71TiB
> 
> 
> Metadata,single: Size:3.00GiB, Used:1.53GiB
> /dev/mapper/storage2 3.00GiB
> 
> 
> System,single: Size:32.00MiB, Used:208.00KiB
> /dev/mapper/storage2 32.00MiB
> 
> 
> Unallocated:
> /dev/mapper/storage2 1.02TiB
> root@thebeach /h/marcg> btrfs fi df /media/Storage2
> Data, single: total=1.71TiB, used=1.38TiB
> System, single: total=32.00MiB, used=208.00KiB
> Metadata, single: total=3.00GiB, used=1.53GiB
> GlobalReserve, single: total=512.00MiB, used=0.00B
> root@thebeach /h/marcg>
> 
> 
> I'm not sure if this is known and if it's btrfs-progs related or if it
> is actually allocating that space.

Could you tell me the location where you think
metadata is reporting double the space?

from fi us:
> Metadata,single: Size:3.00GiB, Used:1.53GiB
> /dev/mapper/storage2 3.00GiB

from fi df:
> Metadata,single: Size:3.00GiB, Used:1.53GiB
> /dev/mapper/storage2 3.00GiB

As far as I can see, Btrfs just allocates 3.00 GiB
from /dev/mapper/storage2, Metadata,single size is
the same as it (not double), and 1.53 GiB is used.


The following is in my case where data is single
and meta is dup.

from fi us:
Metadata,DUP: Size:384.00MiB, Used:221.36MiB
   /dev/vda3 768.00MiB

from fi df:
Metadata, DUP: total=384.00MiB, used=221.36MiB

Here Btrfs allocates 768.0MiB from /dev/vda3 and it's
twice as large as the size of Metadata,DUP(384.00MiB).
I guess it means that "metadata is reporting double the space"
as you said and your case it not the case.

CMIIW.

Thanks,
Satoru

> 
> 
> Thank you for reading.
> 
> 
> Marc
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Btrfs-progs: Initialize stripesize to the value of sectorsize

[Satoru Takeuchi]

On 2016/06/17 1:38, Chandan Rajendra wrote:
> stripesize should ideally be set to the value of sectorsize. However
> previous versions of btrfs-progs/mkfs.btrfs had set stripesize to a
> value of 4096. On machines with PAGE_SIZE other than 4096, This could
> lead to the following scenario,
> 
> - /dev/loop0, /dev/loop1 and /dev/loop2 are mounted as a single
>   filesystem. The filesystem was created by an older version of mkfs.btrfs
>   which set stripesize to 4k.
> - losetup -a
>/dev/loop0: [0030]:19477 (/root/disk-imgs/file-0.img)
>/dev/loop1: [0030]:16577 (/root/disk-imgs/file-1.img)
>/dev/loop2: [64770]:3423229 (/root/disk-imgs/file-2.img)
> - /etc/mtab lists only /dev/loop0
> - losetup /dev/loop4 /root/disk-imgs/file-1.img
>   The new mkfs.btrfs invoked as 'mkfs.btrfs -f /dev/loop4' succeeds even
>   though /dev/loop1 has already been mounted and has
>   /root/disk-imgs/file-1.img as its backing file.
> 
> The above behaviour occurs because check_super() function returns an
> error code (due to stripesize not being set to 4096) and hence
> check_mounted_where() function treats /dev/loop1 as a disk containing a
> filesystem other than Btrfs.
> 
> Hence as a workaround this commit allows 4096 as a valid stripesize.
> 
> Signed-off-by: Chandan Rajendra 
> ---
>  disk-io.c | 4 +++-
>  mkfs.c| 1 +
>  2 files changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/disk-io.c b/disk-io.c
> index 77eb0a6..1ac7631 100644
> --- a/disk-io.c
> +++ b/disk-io.c
> @@ -1476,7 +1476,9 @@ static int check_super(struct btrfs_super_block *sb)
>   error("invalid bytes_used %llu", btrfs_super_bytes_used(sb));
>   goto error_out;
>   }
> - if (btrfs_super_stripesize(sb) != 4096) {
> +
> +if ((btrfs_super_stripesize(sb) != 4096)

Just one trivial comment. Tab was converted to some spaces.

Thanks,
Satoru

> + && (btrfs_super_stripesize(sb) != btrfs_super_sectorsize(sb))) {
>   error("invalid stripesize %u", btrfs_super_stripesize(sb));
>   goto error_out;
>   }
> diff --git a/mkfs.c b/mkfs.c
> index a3a3c14..697bdc2 100644
> --- a/mkfs.c
> +++ b/mkfs.c
> @@ -1482,6 +1482,7 @@ int main(int argc, char **argv)
>   }
>  
>   sectorsize = max(sectorsize, (u32)sysconf(_SC_PAGESIZE));
> + stripesize = sectorsize;
>   saved_optind = optind;
>   dev_cnt = argc - optind;
>   if (dev_cnt == 0)
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] btrfs-progs: doc: correct the destination of btrfs-receive

[Satoru Takeuchi]

On 2016/06/14 18:16, Hugo Mills wrote:
> On Tue, Jun 14, 2016 at 10:51:33AM +0200, David Sterba wrote:
>> On Tue, Jun 14, 2016 at 02:50:19PM +0900, Satoru Takeuchi wrote:
>>> We can set not only btrfs mount point but also any path belong to
>>> btrfs mount point as btrfs-receive's destination.
>>>
>>> Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
>>
>> The patches from you have a consistent whitespace damage, I've fixed
>> the btrfs-crc but now that I see it again I suspect some error on your
>> side.

The problem is on my side. I'm sorry.

>>
>>> @@ -7,14 +7,14 @@ btrfs-receive - receive subvolumes from send stream
>>>
>>>   SYNOPSIS
>>>   
>>> -*btrfs receive* [options] 
>>> +*btrfs receive* [options] 
>>>
>>>   DESCRIPTION
>>>   ---
>>>
>>>   Receive a stream of changes and replicate one or more subvolumes that were
>>>   previously used with *btrfs send* The received subvolumes are stored to
>>> -'mount'.
>>> +'path'.
>>>
>>>   *btrfs receive* will fail int the following cases:
>>>
>>> @@ -37,7 +37,7 @@ by default, btrfs receive uses standard input to receive 
>>> the stream,
>>>   use this option to read from a file instead
>>>
>>>   -C|--chroot::
>>> -confine the process to 'mount' using `chroot`(1)
>>> +confine the process to 'path' using `chroot`(1)
>>>
>>>   -e::
>>>   terminate after receiving an 'end cmd' marker in the stream.
>>
>> ie. all the context lines start with two spaces instead of one. I'll
>> apply this patch manually but please have a look.
> 
>Looking at this, I suspect it's a consequence of sending it as
> "Content-Type: format=flowed; delsp=yes". I'm not sure which of those
> two options is the culprit. When I look at the message in my client
> (mutt), it looks absolutely fine. When I pipe it to hexdump, the
> double-spacing is apparent.

You're right. These are added to charset="iso-2022-jp" plain text
mail since thunderbird 45.

I disabled the setting that appends the above mentioned options.
So, probably the following sample patch doesn't have strange spaces.

===
We can set not only btrfs mount points but also any paths belong to
btrfs mount point as btrfs-receive's destination.

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
 Documentation/btrfs-receive.asciidoc | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/Documentation/btrfs-receive.asciidoc 
b/Documentation/btrfs-receive.asciidoc
index fbbded2..e246603 100644
--- a/Documentation/btrfs-receive.asciidoc
+++ b/Documentation/btrfs-receive.asciidoc
@@ -7,14 +7,14 @@ btrfs-receive - receive subvolumes from send stream

 SYNOPSIS
 
-*btrfs receive* [options] 
+*btrfs receive* [options] 

 DESCRIPTION
 ---

 Receive a stream of changes and replicate one or more subvolumes that were
 previously used with *btrfs send* The received subvolumes are stored to
-'mount'.
+'path'.

 *btrfs receive* will fail int the following cases:

@@ -37,7 +37,7 @@ by default, btrfs receive uses standard input to receive the 
stream,
 use this option to read from a file instead

 -C|--chroot::
-confine the process to 'mount' using `chroot`(1)
+confine the process to 'path' using `chroot`(1)

 -e::
 terminate after receiving an 'end cmd' marker in the stream.
-- 
2.5.5
===

Thanks,
Satoru

> 
>Hugo.
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] btrfs-progs: doc: correct the destination of btrfs-receive

[Satoru Takeuchi]

We can set not only btrfs mount point but also any path belong to
btrfs mount point as btrfs-receive's destination.

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
 Documentation/btrfs-receive.asciidoc | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/Documentation/btrfs-receive.asciidoc 
b/Documentation/btrfs-receive.asciidoc
index fbbded2..e246603 100644
--- a/Documentation/btrfs-receive.asciidoc
+++ b/Documentation/btrfs-receive.asciidoc
@@ -7,14 +7,14 @@ btrfs-receive - receive subvolumes from send stream

 SYNOPSIS
 
-*btrfs receive* [options] 
+*btrfs receive* [options] 

 DESCRIPTION
 ---

 Receive a stream of changes and replicate one or more subvolumes that were
 previously used with *btrfs send* The received subvolumes are stored to
-'mount'.
+'path'.

 *btrfs receive* will fail int the following cases:

@@ -37,7 +37,7 @@ by default, btrfs receive uses standard input to receive the 
stream,
 use this option to read from a file instead

 -C|--chroot::
-confine the process to 'mount' using `chroot`(1)
+confine the process to 'path' using `chroot`(1)

 -e::
 terminate after receiving an 'end cmd' marker in the stream.
--
2.5.5
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 5/5] btrfs-progs: btrfs-crc: make argc check more strict

[Satoru Takeuchi]

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
 btrfs-crc.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/btrfs-crc.c b/btrfs-crc.c
index c2b5f00..d433ff3 100644
--- a/btrfs-crc.c
+++ b/btrfs-crc.c
@@ -69,12 +69,14 @@ int main(int argc, char **argv)
str = argv[optind];

if (!loop) {
-   if (check_argc_min(argc - optind, 1))
+   if (check_argc_exact(argc - optind, 1))
print_usage(255);

printf("%12u - %s\n", crc32c(~1, str, strlen(str)), str);
return 0;
}
+   if (check_argc_exact(argc - optind, 0))
+   print_usage(255);

buf = malloc(length);
if (!buf)
--
2.5.5
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 4/5] btrfs-progs: btrfs-crc: improve usage message

[Satoru Takeuchi]

- If -c is set, filename argument is ignored.
- Describe about -h option

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
 btrfs-crc.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/btrfs-crc.c b/btrfs-crc.c
index 55a4c61..c2b5f00 100644
--- a/btrfs-crc.c
+++ b/btrfs-crc.c
@@ -26,10 +26,12 @@ void print_usage(int status)
 {
printf("usage: btrfs-crc filename\n");
printf("print out the btrfs crc for \"filename\"\n");
-   printf("usage: btrfs-crc filename -c crc [-s seed] [-l length]\n");
+   printf("usage: btrfs-crc -c crc [-s seed] [-l length]\n");
printf("brute force search for file names with the given crc\n");
printf("  -s seedthe random seed (default: random)\n");
printf("  -l length  the length of the file names (default: 10)\n");
+   printf("usage: btrfs-crc -h\n");
+   printf("print this message\n");
exit(status);
 }

--
2.5.5
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 3/5] btrfs-progs: btrfs-crc: print usage on receiving invalid arguments

[Satoru Takeuchi]

Usage is only printed if -h option is set. However it's nice to
do it when wrong option is set or the number of argument is wrong.

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
 btrfs-crc.c | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/btrfs-crc.c b/btrfs-crc.c
index 86dfe05..55a4c61 100644
--- a/btrfs-crc.c
+++ b/btrfs-crc.c
@@ -22,7 +22,7 @@
 #include "crc32c.h"
 #include "utils.h"

-void print_usage(void)
+void print_usage(int status)
 {
printf("usage: btrfs-crc filename\n");
printf("print out the btrfs crc for \"filename\"\n");
@@ -30,7 +30,7 @@ void print_usage(void)
printf("brute force search for file names with the given crc\n");
printf("  -s seedthe random seed (default: random)\n");
printf("  -l length  the length of the file names (default: 10)\n");
-   exit(1);
+   exit(status);
 }

 int main(int argc, char **argv)
@@ -57,9 +57,9 @@ int main(int argc, char **argv)
seed = atol(optarg);
break;
case 'h':
-   print_usage();
+   print_usage(1);
case '?':
-   return 255;
+   print_usage(255);
}
}

@@ -68,7 +68,7 @@ int main(int argc, char **argv)

if (!loop) {
if (check_argc_min(argc - optind, 1))
-   return 255;
+   print_usage(255);

printf("%12u - %s\n", crc32c(~1, str, strlen(str)), str);
return 0;
--
2.5.5
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/5] btrfs-progs: btrfs-crc should be ignored by git

[Satoru Takeuchi]

It's a binary built from btrfs-crc.c

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
 .gitignore | 1 +
 1 file changed, 1 insertion(+)

diff --git a/.gitignore b/.gitignore
index a27cb0d..aaf9702 100644
--- a/.gitignore
+++ b/.gitignore
@@ -33,6 +33,7 @@ btrfs-zero-log
 btrfs-corrupt-block
 btrfs-select-super
 btrfs-calc-size
+btrfs-crc
 btrfstune
 libbtrfs.a
 libbtrfs.so
--
2.5.5
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/5] btrfs-progs: btrfs-crc: fix build error

[Satoru Takeuchi]

Remove the following build error.

  
  $ make btrfs-crc
  [CC] btrfs-crc.o
  [LD] btrfs-crc
  btrfs-crc.o: In function `usage':
  /home/sat/src/btrfs-progs/btrfs-crc.c:26: multiple definition of `usage'
  help.o:/home/sat/src/btrfs-progs/help.c:125: first defined here
  collect2: error: ld returned 1 exit status
  Makefile:294: recipe for target 'btrfs-crc' failed
  make: *** [btrfs-crc] Error 1
  =

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
 btrfs-crc.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/btrfs-crc.c b/btrfs-crc.c
index 723e0b7..86dfe05 100644
--- a/btrfs-crc.c
+++ b/btrfs-crc.c
@@ -22,7 +22,7 @@
 #include "crc32c.h"
 #include "utils.h"

-void usage(void)
+void print_usage(void)
 {
printf("usage: btrfs-crc filename\n");
printf("print out the btrfs crc for \"filename\"\n");
@@ -57,7 +57,7 @@ int main(int argc, char **argv)
seed = atol(optarg);
break;
case 'h':
-   usage();
+   print_usage();
case '?':
return 255;
}
--
2.5.5
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Very quick embarrassing q: /var/lib permissions.

[Satoru Takeuchi]

On 2016/06/02 10:20, al wrote:

Satoru Takeuchi  jp.fujitsu.com> writes:


Thank you, sir!

I wonder if you would let me have the permissions (only) of any of the files
you have inside your equivalent directory.


Here it is.

===
# ls -l /var/lib/btrfs/
total 4
-rw---. 1 root root 413 Jun  1 08:19 scrub.status.
===



Effectively I'm asking (for evidence) if I'm using the command with
escalated permissions when this is unnecessary.


I wonder why you ask these question.
Probably it's enough to read the file permissions of
your system...

Thanks,
Satoru





--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Very quick embarrassing q: /var/lib permissions.

[Satoru Takeuchi]

On 2016/06/01 6:53, al wrote:

Please can someone run:

# ls -l /var/lib/ | grep btrfs

for me (and post to directly or via list as they think fit).



$ ls -l /var/lib | grep btrfs
drwxr-xr-x  1 root  root  196 May 18 10:34 btrfs


This directory will be created when you run "btrfs scrub start"
for the first time.

Thanks,
Satoru



Thank you.

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: About in-band dedupe for v4.7

[Satoru Takeuchi]

On 2016/05/11 10:40, Qu Wenruo wrote:



Chris Mason wrote on 2016/05/10 20:37 -0400:

On Tue, May 10, 2016 at 03:19:52PM +0800, Qu Wenruo wrote:

Hi, Chris, Josef and David,

As merge window for v4.7 is coming, it would be good to hear your ideas
about the inband dedupe.

We are addressing the ENOSPC problem which Josef pointed out, and we believe
the final fix patch would come out at the beginning of the merge
window.(Next week)


If it's fine, would you please consider to merge the in-memory backend
patchset for v4.7 as an experimental feature?


Most of the patch won't be changed from v10 patchset, only ENOSPC fix will
be updated, and ioctl patchset will introduce a new Kconfig option of "btrfs
experimental features" for inband dedupe.
(With explain about unstable ioctl/on-disk format for experimental features)


If you are all OK to merge inband dedupe in-memory backend, I'll prepare the
new v11 patchset for this merge.


We have to balance the part where we really want the features to come
in, and we want to lower the load on you to continue porting them.  But,
I really do agree that we need strong test suites included with every
major feature like this.

-chris



That's fine.

We're running all generic and btrfs test case with dedupe enabled,

> by modifying xfstest to call "btrfs dedeup enable"

just after mount, to ensure dedupe won't corrupt any existing test case.


How about writing the xfstests patch that we can run
the above tests without modifying xfstests by hand?
Then it becomes easy for everyone to test dedupe.

Thanks,
Satoru



And we are also enriching our dedupe test cases to reduce the stability concern 
on the new feature.


BTW, does it mean inband dedup won't catch v4.7 merge window and no need to 
submit a v11 patchset for this merge window?

Thanks,
Qu


--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 3/3] btrfs-progs: prop: remove conditions which never be satisfied

[Satoru Takeuchi]

parse_args() always set at least one parameter, 'object', for
{get,list} subcommands. In addition, it always set all three
parameters, 'object', 'name', and 'value' for set subcommand.
So the following conditions can be removed.

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>

---
 cmds-property.c | 12 
 1 file changed, 12 deletions(-)

diff --git a/cmds-property.c b/cmds-property.c
index 46be8f3..e59882b 100644
--- a/cmds-property.c
+++ b/cmds-property.c
@@ -385,10 +385,6 @@ static int cmd_property_get(int argc, char **argv)

parse_args(argc, argv, cmd_property_get_usage, , , ,
   NULL, 1);
-   if (!object) {
-   error("invalid arguments");
-   usage(cmd_property_get_usage);
-   }

if (name)
ret = setget_prop(types, object, name, NULL);
@@ -416,10 +412,6 @@ static int cmd_property_set(int argc, char **argv)

parse_args(argc, argv, cmd_property_set_usage, ,
   , , , 3);
-   if (!object || !name || !value) {
-   error("invalid arguments");
-   usage(cmd_property_set_usage);
-   }

ret = setget_prop(types, object, name, value);

@@ -442,10 +434,6 @@ static int cmd_property_list(int argc, char **argv)

parse_args(argc, argv, cmd_property_list_usage,
   , , NULL, NULL, 1);
-   if (!object) {
-   error("invalid arguments");
-   usage(cmd_property_list_usage);
-   }

ret = dump_props(types, object, 1);

-- 
2.5.5
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/3] btrfs-progs: prop: simplify parse_args()

[Satoru Takeuchi]

Since  parameter is mandatory for all subcommands,
'object' is always set by parse_args()'s callers.
In addition, on setting '*name' and '*value', if 'optind < argc'
is satisfied here, they are always set by parse_args()'s callers.

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>

---
 cmds-property.c | 13 +
 1 file changed, 5 insertions(+), 8 deletions(-)

diff --git a/cmds-property.c b/cmds-property.c
index 48a8945..46be8f3 100644
--- a/cmds-property.c
+++ b/cmds-property.c
@@ -298,7 +298,7 @@ static void parse_args(int argc, char **argv,
 {
int ret;
char *type_str = NULL;
-   int max_nonopt_args = 0;
+   int max_nonopt_args = 1;

optind = 1;
while (1) {
@@ -315,8 +315,6 @@ static void parse_args(int argc, char **argv,
}
}

-   if (object)
-   max_nonopt_args++;
if (name)
max_nonopt_args++;
if (value)
@@ -345,14 +343,13 @@ static void parse_args(int argc, char **argv,
}
}

-   if (object && optind < argc)
-   *object = argv[optind++];
-   if (name && optind < argc)
+   *object = argv[optind++];
+   if (optind < argc)
*name = argv[optind++];
-   if (value && optind < argc)
+   if (optind < argc)
*value = argv[optind++];

-   if (!*types && object && *object) {
+   if (!*types) {
ret = autodetect_object_types(*object, types);
if (ret < 0) {
error("failed to detect object type: %s",
-- 
2.5.5
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/3] btrfs-progs: prop: convert error messages to use error()

[Satoru Takeuchi]

props.c uses 'fprintf(stderr, "ERROR: ...")' as its error messages,
however we have generic error() function.

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>

---
 props.c | 21 +
 1 file changed, 9 insertions(+), 12 deletions(-)

diff --git a/props.c b/props.c
index 5b74932..5b4e26e 100644
--- a/props.c
+++ b/props.c
@@ -48,7 +48,7 @@ static int prop_read_only(enum prop_object_type type,
fd = open(object, O_RDONLY);
if (fd < 0) {
ret = -errno;
-   fprintf(stderr, "ERROR: open %s failed. %s\n",
+   error("open %s failed. %s",
object, strerror(-ret));
goto out;
}
@@ -56,7 +56,7 @@ static int prop_read_only(enum prop_object_type type,
ret = ioctl(fd, BTRFS_IOC_SUBVOL_GETFLAGS, );
if (ret < 0) {
ret = -errno;
-   fprintf(stderr, "ERROR: failed to get flags for %s. %s\n",
+   error("failed to get flags for %s. %s",
object, strerror(-ret));
goto out;
}
@@ -76,14 +76,14 @@ static int prop_read_only(enum prop_object_type type,
flags = flags & ~BTRFS_SUBVOL_RDONLY;
} else {
ret = -EINVAL;
-   fprintf(stderr, "ERROR: invalid value for property.\n");
+   error("invalid value for property.");
goto out;
}

ret = ioctl(fd, BTRFS_IOC_SUBVOL_SETFLAGS, );
if (ret < 0) {
ret = -errno;
-   fprintf(stderr, "ERROR: failed to set flags for %s. %s\n",
+   error("failed to set flags for %s. %s",
object, strerror(-ret));
goto out;
}
@@ -130,8 +130,7 @@ static int prop_compression(enum prop_object_type type,
fd = open_file_or_dir3(object, , open_flags);
if (fd == -1) {
ret = -errno;
-   fprintf(stderr, "ERROR: open %s failed. %s\n",
-   object, strerror(-ret));
+   error("open %s failed. %s", object, strerror(-ret));
goto out;
}

@@ -151,9 +150,8 @@ static int prop_compression(enum prop_object_type type,
if (sret < 0) {
ret = -errno;
if (ret != -ENOATTR)
-   fprintf(stderr,
-   "ERROR: failed to %s compression for %s. %s\n",
-   value ? "set" : "get", object, strerror(-ret));
+   error("failed to %s compression for %s. %s",
+ value ? "set" : "get", object, strerror(-ret));
else
ret = 0;
goto out;
@@ -169,9 +167,8 @@ static int prop_compression(enum prop_object_type type,
sret = fgetxattr(fd, xattr_name, buf, len);
if (sret < 0) {
ret = -errno;
-   fprintf(stderr,
-   "ERROR: failed to get compression for %s. %s\n",
-   object, strerror(-ret));
+   error("failed to get compression for %s. %s",
+ object, strerror(-ret));
goto out;
}
fprintf(stdout, "compression=%.*s\n", (int)len, buf);
-- 
2.5.5
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Question: raid1 behaviour on failure

[Satoru Takeuchi]

On 2016/04/23 16:07, Matthias Bodenbinder wrote:

Here is my newest test. The backports provide a 4.5 kernel:


kernel: 4.5.0-0.bpo.1-amd64
btrfs-tools: 4.4-1~bpo8+1


This time the raid1 is automatically unmounted after I unplug the device and it 
can not be mounted while the device is missing. See below.

Matthias


As I said before, I consider this problem is not
caused by Btrfs, but by hardware.

Please see the following comments.





1) turn on the Fantec case:

Apr 23 08:45:38 rakete kernel: usb 3-1: new SuperSpeed USB device number 2 
using xhci_hcd
Apr 23 08:45:38 rakete kernel: usb 3-1: New USB device found, idVendor=152d, 
idProduct=0567
Apr 23 08:45:38 rakete kernel: usb 3-1: New USB device strings: Mfr=10, 
Product=11, SerialNumber=5
Apr 23 08:45:38 rakete kernel: usb 3-1: Product: USB to ATA/ATAPI Bridge
Apr 23 08:45:38 rakete kernel: usb 3-1: Manufacturer: JMicron
Apr 23 08:45:38 rakete kernel: usb 3-1: SerialNumber: 152D00539000
Apr 23 08:45:38 rakete mtp-probe[3641]: checking bus 3, device 2: 
"/sys/devices/pci:00/:00:1c.5/:04:00.0/usb3/3-1"
Apr 23 08:45:38 rakete mtp-probe[3641]: bus: 3, device: 2 was not an MTP device
Apr 23 08:45:38 rakete kernel: usb-storage 3-1:1.0: USB Mass Storage device 
detected
Apr 23 08:45:38 rakete kernel: usb-storage 3-1:1.0: Quirks match for vid 152d 
pid 0567: 500
Apr 23 08:45:38 rakete kernel: scsi host8: usb-storage 3-1:1.0
Apr 23 08:45:38 rakete kernel: usbcore: registered new interface driver 
usb-storage
Apr 23 08:45:38 rakete kernel: usbcore: registered new interface driver uas
Apr 23 08:45:39 rakete kernel: scsi 8:0:0:0: Direct-Access WDC WD20 
02FAEX-007BA00125 PQ: 0 ANSI: 6
Apr 23 08:45:39 rakete kernel: scsi 8:0:0:1: Direct-Access WDC WD75 
00AACS-00C7B00125 PQ: 0 ANSI: 6
Apr 23 08:45:39 rakete kernel: scsi 8:0:0:2: Direct-Access WDC WD50 
01AALS-00L3B20125 PQ: 0 ANSI: 6
Apr 23 08:45:39 rakete kernel: scsi 8:0:0:3: Direct-Access SAMSUNG  SP2504C 
 0125 PQ: 0 ANSI: 6
Apr 23 08:45:39 rakete kernel: sd 8:0:0:0: Attached scsi generic sg6 type 0
Apr 23 08:45:39 rakete kernel: sd 8:0:0:0: [sdf] 3907029168 512-byte logical 
blocks: (2.00 TB/1.82 TiB)
Apr 23 08:45:39 rakete kernel: sd 8:0:0:1: Attached scsi generic sg7 type 0
Apr 23 08:45:39 rakete kernel: sd 8:0:0:0: [sdf] Write Protect is off
Apr 23 08:45:39 rakete kernel: sd 8:0:0:0: [sdf] Mode Sense: 67 00 10 08
Apr 23 08:45:39 rakete kernel: sd 8:0:0:2: Attached scsi generic sg8 type 0
Apr 23 08:45:39 rakete kernel: sd 8:0:0:1: [sdg] 1465149168 512-byte logical 
blocks: (750 GB/699 GiB)
Apr 23 08:45:39 rakete kernel: sd 8:0:0:1: [sdg] Write Protect is off
Apr 23 08:45:39 rakete kernel: sd 8:0:0:1: [sdg] Mode Sense: 67 00 10 08
Apr 23 08:45:39 rakete kernel: sd 8:0:0:2: [sdh] 976773168 512-byte logical 
blocks: (500 GB/466 GiB)
Apr 23 08:45:39 rakete kernel: sd 8:0:0:3: Attached scsi generic sg9 type 0
Apr 23 08:45:39 rakete kernel: sd 8:0:0:1: [sdg] No Caching mode page found
Apr 23 08:45:39 rakete kernel: sd 8:0:0:1: [sdg] Assuming drive cache: write 
through
Apr 23 08:45:39 rakete kernel: sd 8:0:0:2: [sdh] Write Protect is off
Apr 23 08:45:39 rakete kernel: sd 8:0:0:2: [sdh] Mode Sense: 67 00 10 08
Apr 23 08:45:39 rakete kernel: sd 8:0:0:3: [sdi] 488395055 512-byte logical 
blocks: (250 GB/233 GiB)
Apr 23 08:45:39 rakete kernel: sd 8:0:0:0: [sdf] No Caching mode page found
Apr 23 08:45:39 rakete kernel: sd 8:0:0:0: [sdf] Assuming drive cache: write 
through
Apr 23 08:45:39 rakete kernel: sd 8:0:0:3: [sdi] Write Protect is off
Apr 23 08:45:39 rakete kernel: sd 8:0:0:3: [sdi] Mode Sense: 67 00 10 08
Apr 23 08:45:39 rakete kernel: sd 8:0:0:2: [sdh] No Caching mode page found
Apr 23 08:45:39 rakete kernel: sd 8:0:0:2: [sdh] Assuming drive cache: write 
through
Apr 23 08:45:39 rakete kernel: sd 8:0:0:3: [sdi] No Caching mode page found
Apr 23 08:45:39 rakete kernel: sd 8:0:0:3: [sdi] Assuming drive cache: write 
through
Apr 23 08:45:39 rakete kernel:  sdf: sdf1
Apr 23 08:45:39 rakete kernel: sd 8:0:0:0: [sdf] Attached SCSI disk
Apr 23 08:45:39 rakete kernel: sd 8:0:0:1: [sdg] Attached SCSI disk
Apr 23 08:45:40 rakete kernel: sd 8:0:0:2: [sdh] Attached SCSI disk
Apr 23 08:45:40 rakete kernel: sd 8:0:0:3: [sdi] Attached SCSI disk


When you turned on Fantec case, four disks, WD20(sdf), WD75(sdg),
WD50(sgh), and SP2504C(sgi) were attached. It's a matter of course.


Apr 23 08:45:40 rakete kernel: BTRFS: device fsid 
16d5891f-5d52-4b29-8591-588ddf11e73d devid 1 transid 89 /dev/sdg
Apr 23 08:45:40 rakete kernel: BTRFS: device fsid 
16d5891f-5d52-4b29-8591-588ddf11e73d devid 2 transid 89 /dev/sdh
Apr 23 08:45:40 rakete kernel: BTRFS: device fsid 
16d5891f-5d52-4b29-8591-588ddf11e73d devid 3 transid 89 /dev/sdi
Apr 23 08:45:40 rakete kernel: EXT4-fs (sdf1): mounted filesystem with ordered 
data mode. Opts: (null)
Apr 23 08:45:40 rakete udisksd[2422]: Mounted /dev/sdf1 at 
/media/matthias/BACKUP on behalf of uid 1000



7# mount /mnt/raid1/

Re: Question: raid1 behaviour on failure

[Satoru Takeuchi]

On 2016/04/22 14:32, Qu Wenruo wrote:



Satoru Takeuchi wrote on 2016/04/22 11:21 +0900:

On 2016/04/21 20:58, Qu Wenruo wrote:



On 04/21/2016 03:45 PM, Satoru Takeuchi wrote:

On 2016/04/21 15:23, Satoru Takeuchi wrote:

On 2016/04/20 14:17, Matthias Bodenbinder wrote:

Am 18.04.2016 um 09:22 schrieb Qu Wenruo:

BTW, it would be better to post the dmesg for better debug.


So here we. I did the same test again. Here is a full log of what i
did. It seems to be mean like a bug in btrfs.
Sequenz of events:
1. mount the raid1 (2 disc with different size)
2. unplug the biggest drive (hotplug)
3. try to copy something to the degraded raid1
4. plugin the device again (hotplug)

This scenario does not work. The disc array is NOT redundant! I can
not work with it while a drive is missing and I can not reattach the
device so that everything works again.

The btrfs module crashes during the test.

I am using LMDE2 with backports:
btrfs-tools 4.4-1~bpo8+1
linux-image-4.4.0-0.bpo.1-amd64

Matthias


rakete - root - /root
1# mount /mnt/raid1/

Journal:

Apr 20 07:01:16 rakete kernel: BTRFS info (device sdi): enabling auto
defrag
Apr 20 07:01:16 rakete kernel: BTRFS info (device sdi): disk space
caching is enabled
Apr 20 07:01:16 rakete kernel: BTRFS: has skinny extents

rakete - root - /mnt/raid1
3# ll
insgesamt 0
drwxrwxr-x 1 root root   36 Nov 14  2014 AfterShot2(64-bit)
drwxrwxr-x 1 root root 5082 Apr 17 09:06 etc
drwxr-xr-x 1 root root  108 Mär 24 07:31 var

4# btrfs fi show
Label: none  uuid: 16d5891f-5d52-4b29-8591-588ddf11e73d
Total devices 3 FS bytes used 1.60GiB
devid1 size 698.64GiB used 3.03GiB path /dev/sdg
devid2 size 465.76GiB used 3.03GiB path /dev/sdh
devid3 size 232.88GiB used 0.00B path /dev/sdi


unplug device sdg:

Apr 20 07:03:05 rakete kernel: Buffer I/O error on dev sdf1, logical
block 243826688, lost sync page write
Apr 20 07:03:05 rakete kernel: JBD2: Error -5 detected when updating
journal superblock for sdf1-8.
Apr 20 07:03:05 rakete kernel: Aborting journal on device sdf1-8.
Apr 20 07:03:05 rakete kernel: Buffer I/O error on dev sdf1, logical
block 243826688, lost sync page write
Apr 20 07:03:05 rakete kernel: JBD2: Error -5 detected when updating
journal superblock for sdf1-8.
Apr 20 07:03:05 rakete umount[16405]: umount: /mnt/raid1: target is
busy
Apr 20 07:03:05 rakete umount[16405]: (In some cases useful info
about processes that
Apr 20 07:03:05 rakete umount[16405]: use the device is found by
lsof(8) or fuser(1).)
Apr 20 07:03:05 rakete systemd[1]: mnt-raid1.mount mount process
exited, code=exited status=32
Apr 20 07:03:05 rakete systemd[1]: Failed unmounting /mnt/raid1.
Apr 20 07:03:24 rakete kernel: usb 3-1: new SuperSpeed USB device
number 3 using xhci_hcd
Apr 20 07:03:24 rakete kernel: usb 3-1: New USB device found,
idVendor=152d, idProduct=0567
Apr 20 07:03:24 rakete kernel: usb 3-1: New USB device strings:
Mfr=10, Product=11, SerialNumber=5
Apr 20 07:03:24 rakete kernel: usb 3-1: Product: USB to ATA/ATAPI
Bridge
Apr 20 07:03:24 rakete kernel: usb 3-1: Manufacturer: JMicron
Apr 20 07:03:24 rakete kernel: usb 3-1: SerialNumber: 152D00539000
Apr 20 07:03:24 rakete kernel: usb-storage 3-1:1.0: USB Mass Storage
device detected
Apr 20 07:03:24 rakete kernel: usb-storage 3-1:1.0: Quirks match for
vid 152d pid 0567: 500
Apr 20 07:03:24 rakete kernel: scsi host9: usb-storage 3-1:1.0
Apr 20 07:03:24 rakete mtp-probe[16424]: checking bus 3, device 3:
"/sys/devices/pci:00/:00:1c.5/:04:00.0/usb3/3-1"
Apr 20 07:03:24 rakete mtp-probe[16424]: bus: 3, device: 3 was not an
MTP device
Apr 20 07:03:25 rakete kernel: scsi 9:0:0:0: Direct-Access WDC
WD20 02FAEX-007BA00125 PQ: 0 ANSI: 6
Apr 20 07:03:25 rakete kernel: scsi 9:0:0:1: Direct-Access WDC
WD50 01AALS-00L3B20125 PQ: 0 ANSI: 6
Apr 20 07:03:25 rakete kernel: scsi 9:0:0:2: Direct-Access
SAMSUNG  SP2504C  0125 PQ: 0 ANSI: 6
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: Attached scsi generic sg6
type 0
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: Attached scsi generic sg7
type 0
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] 3907029168 512-byte
logical blocks: (2.00 TB/1.82 TiB)
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] Write Protect is off
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] Mode Sense: 67 00
10 08
Apr 20 07:03:25 rakete kernel: sd 9:0:0:2: Attached scsi generic sg8
type 0
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] 976773168 512-byte
logical blocks: (500 GB/466 GiB)
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] No Caching mode page
found
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] Assuming drive
cache: write through
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] Write Protect is off
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] Mode Sense: 67 00
10 08
Apr 20 07:03:25 rakete kernel: sd 9:0:0:2: [sdk] 488395055 512-byte
logical blocks: (250 GB/233 GiB)
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] No Caching mode page
found

Re: Question: raid1 behaviour on failure

[Satoru Takeuchi]

On 2016/04/21 20:58, Qu Wenruo wrote:



On 04/21/2016 03:45 PM, Satoru Takeuchi wrote:

On 2016/04/21 15:23, Satoru Takeuchi wrote:

On 2016/04/20 14:17, Matthias Bodenbinder wrote:

Am 18.04.2016 um 09:22 schrieb Qu Wenruo:

BTW, it would be better to post the dmesg for better debug.


So here we. I did the same test again. Here is a full log of what i
did. It seems to be mean like a bug in btrfs.
Sequenz of events:
1. mount the raid1 (2 disc with different size)
2. unplug the biggest drive (hotplug)
3. try to copy something to the degraded raid1
4. plugin the device again (hotplug)

This scenario does not work. The disc array is NOT redundant! I can
not work with it while a drive is missing and I can not reattach the
device so that everything works again.

The btrfs module crashes during the test.

I am using LMDE2 with backports:
btrfs-tools 4.4-1~bpo8+1
linux-image-4.4.0-0.bpo.1-amd64

Matthias


rakete - root - /root
1# mount /mnt/raid1/

Journal:

Apr 20 07:01:16 rakete kernel: BTRFS info (device sdi): enabling auto
defrag
Apr 20 07:01:16 rakete kernel: BTRFS info (device sdi): disk space
caching is enabled
Apr 20 07:01:16 rakete kernel: BTRFS: has skinny extents

rakete - root - /mnt/raid1
3# ll
insgesamt 0
drwxrwxr-x 1 root root   36 Nov 14  2014 AfterShot2(64-bit)
drwxrwxr-x 1 root root 5082 Apr 17 09:06 etc
drwxr-xr-x 1 root root  108 Mär 24 07:31 var

4# btrfs fi show
Label: none  uuid: 16d5891f-5d52-4b29-8591-588ddf11e73d
Total devices 3 FS bytes used 1.60GiB
devid1 size 698.64GiB used 3.03GiB path /dev/sdg
devid2 size 465.76GiB used 3.03GiB path /dev/sdh
devid3 size 232.88GiB used 0.00B path /dev/sdi


unplug device sdg:

Apr 20 07:03:05 rakete kernel: Buffer I/O error on dev sdf1, logical
block 243826688, lost sync page write
Apr 20 07:03:05 rakete kernel: JBD2: Error -5 detected when updating
journal superblock for sdf1-8.
Apr 20 07:03:05 rakete kernel: Aborting journal on device sdf1-8.
Apr 20 07:03:05 rakete kernel: Buffer I/O error on dev sdf1, logical
block 243826688, lost sync page write
Apr 20 07:03:05 rakete kernel: JBD2: Error -5 detected when updating
journal superblock for sdf1-8.
Apr 20 07:03:05 rakete umount[16405]: umount: /mnt/raid1: target is busy
Apr 20 07:03:05 rakete umount[16405]: (In some cases useful info
about processes that
Apr 20 07:03:05 rakete umount[16405]: use the device is found by
lsof(8) or fuser(1).)
Apr 20 07:03:05 rakete systemd[1]: mnt-raid1.mount mount process
exited, code=exited status=32
Apr 20 07:03:05 rakete systemd[1]: Failed unmounting /mnt/raid1.
Apr 20 07:03:24 rakete kernel: usb 3-1: new SuperSpeed USB device
number 3 using xhci_hcd
Apr 20 07:03:24 rakete kernel: usb 3-1: New USB device found,
idVendor=152d, idProduct=0567
Apr 20 07:03:24 rakete kernel: usb 3-1: New USB device strings:
Mfr=10, Product=11, SerialNumber=5
Apr 20 07:03:24 rakete kernel: usb 3-1: Product: USB to ATA/ATAPI Bridge
Apr 20 07:03:24 rakete kernel: usb 3-1: Manufacturer: JMicron
Apr 20 07:03:24 rakete kernel: usb 3-1: SerialNumber: 152D00539000
Apr 20 07:03:24 rakete kernel: usb-storage 3-1:1.0: USB Mass Storage
device detected
Apr 20 07:03:24 rakete kernel: usb-storage 3-1:1.0: Quirks match for
vid 152d pid 0567: 500
Apr 20 07:03:24 rakete kernel: scsi host9: usb-storage 3-1:1.0
Apr 20 07:03:24 rakete mtp-probe[16424]: checking bus 3, device 3:
"/sys/devices/pci:00/:00:1c.5/:04:00.0/usb3/3-1"
Apr 20 07:03:24 rakete mtp-probe[16424]: bus: 3, device: 3 was not an
MTP device
Apr 20 07:03:25 rakete kernel: scsi 9:0:0:0: Direct-Access WDC
WD20 02FAEX-007BA00125 PQ: 0 ANSI: 6
Apr 20 07:03:25 rakete kernel: scsi 9:0:0:1: Direct-Access WDC
WD50 01AALS-00L3B20125 PQ: 0 ANSI: 6
Apr 20 07:03:25 rakete kernel: scsi 9:0:0:2: Direct-Access
SAMSUNG  SP2504C  0125 PQ: 0 ANSI: 6
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: Attached scsi generic sg6
type 0
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: Attached scsi generic sg7
type 0
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] 3907029168 512-byte
logical blocks: (2.00 TB/1.82 TiB)
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] Write Protect is off
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] Mode Sense: 67 00 10 08
Apr 20 07:03:25 rakete kernel: sd 9:0:0:2: Attached scsi generic sg8
type 0
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] 976773168 512-byte
logical blocks: (500 GB/466 GiB)
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] No Caching mode page
found
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] Assuming drive
cache: write through
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] Write Protect is off
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] Mode Sense: 67 00 10 08
Apr 20 07:03:25 rakete kernel: sd 9:0:0:2: [sdk] 488395055 512-byte
logical blocks: (250 GB/233 GiB)
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] No Caching mode page
found
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] Assuming drive
cache: write through
Apr 2

Re: Question: raid1 behaviour on failure

[Satoru Takeuchi]

On 2016/04/21 15:23, Satoru Takeuchi wrote:

On 2016/04/20 14:17, Matthias Bodenbinder wrote:

Am 18.04.2016 um 09:22 schrieb Qu Wenruo:

BTW, it would be better to post the dmesg for better debug.


So here we. I did the same test again. Here is a full log of what i did. It 
seems to be mean like a bug in btrfs.
Sequenz of events:
1. mount the raid1 (2 disc with different size)
2. unplug the biggest drive (hotplug)
3. try to copy something to the degraded raid1
4. plugin the device again (hotplug)

This scenario does not work. The disc array is NOT redundant! I can not work 
with it while a drive is missing and I can not reattach the device so that 
everything works again.

The btrfs module crashes during the test.

I am using LMDE2 with backports:
btrfs-tools 4.4-1~bpo8+1
linux-image-4.4.0-0.bpo.1-amd64

Matthias


rakete - root - /root
1# mount /mnt/raid1/

Journal:

Apr 20 07:01:16 rakete kernel: BTRFS info (device sdi): enabling auto defrag
Apr 20 07:01:16 rakete kernel: BTRFS info (device sdi): disk space caching is 
enabled
Apr 20 07:01:16 rakete kernel: BTRFS: has skinny extents

rakete - root - /mnt/raid1
3# ll
insgesamt 0
drwxrwxr-x 1 root root   36 Nov 14  2014 AfterShot2(64-bit)
drwxrwxr-x 1 root root 5082 Apr 17 09:06 etc
drwxr-xr-x 1 root root  108 Mär 24 07:31 var

4# btrfs fi show
Label: none  uuid: 16d5891f-5d52-4b29-8591-588ddf11e73d
Total devices 3 FS bytes used 1.60GiB
devid1 size 698.64GiB used 3.03GiB path /dev/sdg
devid2 size 465.76GiB used 3.03GiB path /dev/sdh
devid3 size 232.88GiB used 0.00B path /dev/sdi


unplug device sdg:

Apr 20 07:03:05 rakete kernel: Buffer I/O error on dev sdf1, logical block 
243826688, lost sync page write
Apr 20 07:03:05 rakete kernel: JBD2: Error -5 detected when updating journal 
superblock for sdf1-8.
Apr 20 07:03:05 rakete kernel: Aborting journal on device sdf1-8.
Apr 20 07:03:05 rakete kernel: Buffer I/O error on dev sdf1, logical block 
243826688, lost sync page write
Apr 20 07:03:05 rakete kernel: JBD2: Error -5 detected when updating journal 
superblock for sdf1-8.
Apr 20 07:03:05 rakete umount[16405]: umount: /mnt/raid1: target is busy
Apr 20 07:03:05 rakete umount[16405]: (In some cases useful info about 
processes that
Apr 20 07:03:05 rakete umount[16405]: use the device is found by lsof(8) or 
fuser(1).)
Apr 20 07:03:05 rakete systemd[1]: mnt-raid1.mount mount process exited, 
code=exited status=32
Apr 20 07:03:05 rakete systemd[1]: Failed unmounting /mnt/raid1.
Apr 20 07:03:24 rakete kernel: usb 3-1: new SuperSpeed USB device number 3 
using xhci_hcd
Apr 20 07:03:24 rakete kernel: usb 3-1: New USB device found, idVendor=152d, 
idProduct=0567
Apr 20 07:03:24 rakete kernel: usb 3-1: New USB device strings: Mfr=10, 
Product=11, SerialNumber=5
Apr 20 07:03:24 rakete kernel: usb 3-1: Product: USB to ATA/ATAPI Bridge
Apr 20 07:03:24 rakete kernel: usb 3-1: Manufacturer: JMicron
Apr 20 07:03:24 rakete kernel: usb 3-1: SerialNumber: 152D00539000
Apr 20 07:03:24 rakete kernel: usb-storage 3-1:1.0: USB Mass Storage device 
detected
Apr 20 07:03:24 rakete kernel: usb-storage 3-1:1.0: Quirks match for vid 152d 
pid 0567: 500
Apr 20 07:03:24 rakete kernel: scsi host9: usb-storage 3-1:1.0
Apr 20 07:03:24 rakete mtp-probe[16424]: checking bus 3, device 3: 
"/sys/devices/pci:00/:00:1c.5/:04:00.0/usb3/3-1"
Apr 20 07:03:24 rakete mtp-probe[16424]: bus: 3, device: 3 was not an MTP device
Apr 20 07:03:25 rakete kernel: scsi 9:0:0:0: Direct-Access WDC WD20 
02FAEX-007BA00125 PQ: 0 ANSI: 6
Apr 20 07:03:25 rakete kernel: scsi 9:0:0:1: Direct-Access WDC WD50 
01AALS-00L3B20125 PQ: 0 ANSI: 6
Apr 20 07:03:25 rakete kernel: scsi 9:0:0:2: Direct-Access SAMSUNG  SP2504C 
 0125 PQ: 0 ANSI: 6
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: Attached scsi generic sg6 type 0
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: Attached scsi generic sg7 type 0
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] 3907029168 512-byte logical 
blocks: (2.00 TB/1.82 TiB)
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] Write Protect is off
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] Mode Sense: 67 00 10 08
Apr 20 07:03:25 rakete kernel: sd 9:0:0:2: Attached scsi generic sg8 type 0
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] 976773168 512-byte logical 
blocks: (500 GB/466 GiB)
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] No Caching mode page found
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] Assuming drive cache: write 
through
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] Write Protect is off
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] Mode Sense: 67 00 10 08
Apr 20 07:03:25 rakete kernel: sd 9:0:0:2: [sdk] 488395055 512-byte logical 
blocks: (250 GB/233 GiB)
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] No Caching mode page found
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] Assuming drive cache: write 
through
Apr 20 07:03:25 rakete kernel: sd 9:0:0:2: [sdk] Write Protect is 

Re: Question: raid1 behaviour on failure

[Satoru Takeuchi]

On 2016/04/20 14:17, Matthias Bodenbinder wrote:

Am 18.04.2016 um 09:22 schrieb Qu Wenruo:

BTW, it would be better to post the dmesg for better debug.


So here we. I did the same test again. Here is a full log of what i did. It 
seems to be mean like a bug in btrfs.
Sequenz of events:
1. mount the raid1 (2 disc with different size)
2. unplug the biggest drive (hotplug)
3. try to copy something to the degraded raid1
4. plugin the device again (hotplug)

This scenario does not work. The disc array is NOT redundant! I can not work 
with it while a drive is missing and I can not reattach the device so that 
everything works again.

The btrfs module crashes during the test.

I am using LMDE2 with backports:
btrfs-tools 4.4-1~bpo8+1
linux-image-4.4.0-0.bpo.1-amd64

Matthias


rakete - root - /root
1# mount /mnt/raid1/

Journal:

Apr 20 07:01:16 rakete kernel: BTRFS info (device sdi): enabling auto defrag
Apr 20 07:01:16 rakete kernel: BTRFS info (device sdi): disk space caching is 
enabled
Apr 20 07:01:16 rakete kernel: BTRFS: has skinny extents

rakete - root - /mnt/raid1
3# ll
insgesamt 0
drwxrwxr-x 1 root root   36 Nov 14  2014 AfterShot2(64-bit)
drwxrwxr-x 1 root root 5082 Apr 17 09:06 etc
drwxr-xr-x 1 root root  108 Mär 24 07:31 var

4# btrfs fi show
Label: none  uuid: 16d5891f-5d52-4b29-8591-588ddf11e73d
Total devices 3 FS bytes used 1.60GiB
devid1 size 698.64GiB used 3.03GiB path /dev/sdg
devid2 size 465.76GiB used 3.03GiB path /dev/sdh
devid3 size 232.88GiB used 0.00B path /dev/sdi


unplug device sdg:

Apr 20 07:03:05 rakete kernel: Buffer I/O error on dev sdf1, logical block 
243826688, lost sync page write
Apr 20 07:03:05 rakete kernel: JBD2: Error -5 detected when updating journal 
superblock for sdf1-8.
Apr 20 07:03:05 rakete kernel: Aborting journal on device sdf1-8.
Apr 20 07:03:05 rakete kernel: Buffer I/O error on dev sdf1, logical block 
243826688, lost sync page write
Apr 20 07:03:05 rakete kernel: JBD2: Error -5 detected when updating journal 
superblock for sdf1-8.
Apr 20 07:03:05 rakete umount[16405]: umount: /mnt/raid1: target is busy
Apr 20 07:03:05 rakete umount[16405]: (In some cases useful info about 
processes that
Apr 20 07:03:05 rakete umount[16405]: use the device is found by lsof(8) or 
fuser(1).)
Apr 20 07:03:05 rakete systemd[1]: mnt-raid1.mount mount process exited, 
code=exited status=32
Apr 20 07:03:05 rakete systemd[1]: Failed unmounting /mnt/raid1.
Apr 20 07:03:24 rakete kernel: usb 3-1: new SuperSpeed USB device number 3 
using xhci_hcd
Apr 20 07:03:24 rakete kernel: usb 3-1: New USB device found, idVendor=152d, 
idProduct=0567
Apr 20 07:03:24 rakete kernel: usb 3-1: New USB device strings: Mfr=10, 
Product=11, SerialNumber=5
Apr 20 07:03:24 rakete kernel: usb 3-1: Product: USB to ATA/ATAPI Bridge
Apr 20 07:03:24 rakete kernel: usb 3-1: Manufacturer: JMicron
Apr 20 07:03:24 rakete kernel: usb 3-1: SerialNumber: 152D00539000
Apr 20 07:03:24 rakete kernel: usb-storage 3-1:1.0: USB Mass Storage device 
detected
Apr 20 07:03:24 rakete kernel: usb-storage 3-1:1.0: Quirks match for vid 152d 
pid 0567: 500
Apr 20 07:03:24 rakete kernel: scsi host9: usb-storage 3-1:1.0
Apr 20 07:03:24 rakete mtp-probe[16424]: checking bus 3, device 3: 
"/sys/devices/pci:00/:00:1c.5/:04:00.0/usb3/3-1"
Apr 20 07:03:24 rakete mtp-probe[16424]: bus: 3, device: 3 was not an MTP device
Apr 20 07:03:25 rakete kernel: scsi 9:0:0:0: Direct-Access WDC WD20 
02FAEX-007BA00125 PQ: 0 ANSI: 6
Apr 20 07:03:25 rakete kernel: scsi 9:0:0:1: Direct-Access WDC WD50 
01AALS-00L3B20125 PQ: 0 ANSI: 6
Apr 20 07:03:25 rakete kernel: scsi 9:0:0:2: Direct-Access SAMSUNG  SP2504C 
 0125 PQ: 0 ANSI: 6
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: Attached scsi generic sg6 type 0
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: Attached scsi generic sg7 type 0
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] 3907029168 512-byte logical 
blocks: (2.00 TB/1.82 TiB)
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] Write Protect is off
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] Mode Sense: 67 00 10 08
Apr 20 07:03:25 rakete kernel: sd 9:0:0:2: Attached scsi generic sg8 type 0
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] 976773168 512-byte logical 
blocks: (500 GB/466 GiB)
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] No Caching mode page found
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] Assuming drive cache: write 
through
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] Write Protect is off
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] Mode Sense: 67 00 10 08
Apr 20 07:03:25 rakete kernel: sd 9:0:0:2: [sdk] 488395055 512-byte logical 
blocks: (250 GB/233 GiB)
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] No Caching mode page found
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] Assuming drive cache: write 
through
Apr 20 07:03:25 rakete kernel: sd 9:0:0:2: [sdk] Write Protect is off
Apr 20 07:03:25 rakete kernel: sd 

[PATCH] btrfs-progs: prop: remove an unnecessary condition on parse_args

[Satoru Takeuchi]

>From commit c742debab11f ('btrfs-progs: fix a regression that
"property" with -t option doesn't work'), the number of arguments
is checked strictly. So the following condition never be
satisfied.

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
 cmds-property.c | 5 -
 1 file changed, 5 deletions(-)

diff --git a/cmds-property.c b/cmds-property.c
index eed5f4a..48a8945 100644
--- a/cmds-property.c
+++ b/cmds-property.c
@@ -352,11 +352,6 @@ static void parse_args(int argc, char **argv,
if (value && optind < argc)
*value = argv[optind++];

-   if (optind != argc) {
-   error("unexpected agruments found");
-   usage(usage_str);
-   }
-
if (!*types && object && *object) {
ret = autodetect_object_types(*object, types);
if (ret < 0) {
-- 
2.5.5
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] btrfs: Test that qgroup counts are valid after snapshot creation

[Satoru Takeuchi]

On 2016/04/20 7:25, Mark Fasheh wrote:

This has been broken since Linux v4.1. We may have worked out a solution on
the btrfs list but in the meantime sending a test to expose the issue seems
like a good idea.

Signed-off-by: Mark Fasheh 
---
  tests/btrfs/122   | 88 +++
  tests/btrfs/group |  1 +


You forgot to add tests/btrfs/122.out.


  2 files changed, 89 insertions(+)
  create mode 100755 tests/btrfs/122

diff --git a/tests/btrfs/122 b/tests/btrfs/122
new file mode 100755
index 000..b7e9e4b
--- /dev/null
+++ b/tests/btrfs/122
@@ -0,0 +1,88 @@
+#! /bin/bash
+# FS QA Test No. btrfs/122
+#
+# Test that qgroup counts are valid after snapshot creation. This has
+# been broken in btrfs since Linux v4.1
+#
+#---
+# Copyright (C) 2016 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#---
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1   # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+   cd /
+   rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+rm -f $seqres.full
+
+# Force a small leaf size to make it easier to blow out our root
+# subvolume tree
+_scratch_mkfs "--nodesize 16384"


nodesize 16384 is the default value. Do you
intend other value, for example 4096?


+_scratch_mount
+_run_btrfs_util_prog quota enable $SCRATCH_MNT
+
+mkdir "$SCRATCH_MNT/snaps"
+
+# First make some simple snapshots - the bug was initially reproduced like this
+_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT 
"$SCRATCH_MNT/snaps/empty1"
+_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT 
"$SCRATCH_MNT/snaps/empty2"
+
+# This forces the fs tree out past level 0, adding at least one tree
+# block which must be properly accounted for when we make our next
+# snapshots.
+mkdir "$SCRATCH_MNT/data"
+for i in `seq 0 640`; do
+$XFS_IO_PROG -f -c "pwrite 0 1M" "$SCRATCH_MNT/data/file$i" > /dev/null 
2>&1
+done;


";" after "done" is not necessary.

Thanks,
Satoru


+
+# Snapshot twice.
+_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT "$SCRATCH_MNT/snaps/snap1"
+_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT "$SCRATCH_MNT/snaps/snap2"
+
+_scratch_unmount
+
+# generate a qgroup report and look for inconsistent groups
+$BTRFS_UTIL_PROG check --qgroup-report $SCRATCH_DEV 2>&1 | \
+   grep -q -E "Counts for qgroup.*are different"
+if [ $? -ne 0 ]; then
+   status=0
+fi
+
+exit
diff --git a/tests/btrfs/group b/tests/btrfs/group
index 9403daa..f7e8cff 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -122,3 +122,4 @@
  119 auto quick snapshot metadata qgroup
  120 auto quick snapshot metadata
  121 auto quick snapshot qgroup
+122 auto quick snapshot qgroup


--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RFE: btrfs subvolume list -t, include marker for snapshots, and whether they are ro

[Satoru Takeuchi]

On 2016/04/15 10:55, Chris Murphy wrote:

Hi,

I'm realizing instead of doing 'btrfs subvolume -t' and then 'btrfs
subvolume -tr' and comparing, it would be better if -t just had a
column for whether a subvolume is ro. And maybe it's useful to know if
a subvolume is a snapshot or not (?). I'm not super picky about the
last one. But being able to see if it's ro or not is a nice to have.

In particular on openSUSE with snapper where there's a prolific amount
of snapshots, and the naming convention doesn't indicate whether
something is ro or not.



Although I'm not sure whether such changes is acceptable or not,
just FYI, you can list subvols with ro flags by the following script.


#!/bin/ruby

IO.popen("btrfs sub list -t " + ARGV[0]) {|p_list|
  puts p_list.gets.chomp + "ro"
  puts p_list.gets.chomp + "--"
  ro = ''
  p_list.each { |l|
l.chomp!
path = l.split[3]
IO.popen("btrfs property get #{ARGV[0]+path} | sed -nre 
's/^ro=(true|false)$/\\1/p'") { |p_prop|
  ro = p_prop.gets
  ro = "false" if ro.nil?
}

puts l + "\t" + ro
  }
}


# NOTE: It's not well tested.

sample output:

# ./sub-list-with-ro.rb /
ID  gen top level   pathro
--  --- -   --
257 48672   5   rootfalse
259 48623   257 var/lib/machinesfalse
452 36540   257 root/snaps/root-2016-03-18  true
457 45676   257 root/snaps/root-2016-04-08  true


Thanks,
Satoru
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] btrfs-progs: mkfs: fix an error when using DUP on multidev fs

[Satoru Takeuchi]

To accept DUP on multidev fs, in addition to the following
commit, we need to mark DUP as an allowed data/metadata
profile.

commit 42f1279bf8e9 ("btrfs-progs: mkfs: allow DUP on multidev fs, only warn")

* actual result

  =
  # ./mkfs.btrfs -f -m DUP -d DUP /dev/sdb1 /dev/sdb2
  btrfs-progs v4.5-24-ga35b7e6
  See http://btrfs.wiki.kernel.org for more information.

  WARNING: DUP is not recommended on filesystem with multiple devices
  ERROR: unable to create FS with metadata profile DUP (have 2 devices but 1 
devices are required)
  =

* expected result

  =
  # ./mkfs.btrfs -f -m dup -d dup /dev/sdb1 /dev/sdb2
  WARNING: DUP is not recommended on filesystem with multiple devices
  btrfs-progs v4.5-25-g1a10a3c
  See http://btrfs.wiki.kernel.org for more information.

  Label:  (null)
  UUID:   010d72ff-c87c-4516-8916-5e635719d110
  Node size:  16384
  Sector size:4096
  Filesystem size:28.87GiB
  Block group profiles:
Data: DUP   1.01GiB
Metadata: DUP   1.01GiB
System:   DUP  12.00MiB
  SSD detected:   no
  Incompat features:  extref, skinny-metadata
  Number of devices:  2
  Devices:
 IDSIZE  PATH
  1   953.00MiB  /dev/sdb1
  227.94GiB  /dev/sdb2

  ==

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
This patch can be applied to devel branch (commit: a35b7e6ee122)
---
 utils.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/utils.c b/utils.c
index 75ce6ea..7e45702 100644
--- a/utils.c
+++ b/utils.c
@@ -2455,7 +2455,6 @@ int test_num_disk_vs_raid(u64 metadata_profile, u64 
data_profile,
case 2:
allowed |= BTRFS_BLOCK_GROUP_RAID0 | BTRFS_BLOCK_GROUP_RAID1 |
BTRFS_BLOCK_GROUP_RAID5;
-   break;
case 1:
allowed |= BTRFS_BLOCK_GROUP_DUP;
}
-- 
2.5.0
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 3/5] btrfs-progs: "inspect-internal subvolid-resolve" doesn't work

[Satoru Takeuchi]

"inspect-internal subvolid-resolve" doesn't work from the following commit.

commit 176aeca9a148 ("btrfs-progs: add getopt stubs where needed")

It's because 1st argument, subvolid, is also used for the pathname of
filesystem. 2nd argument should be used for this purpose instead.

* actual result

  ==
  # ./btrfs inspect-internal subvolid-resolve 260 /btrfs
  ERROR: cannot access '260': No such file or directory
  ==

* expected result

  ==
  # btrfs inspect-internal subvolid-resolve 260 /btrfs
  snap
  ======

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
 cmds-inspect.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/cmds-inspect.c b/cmds-inspect.c
index 04e1ae8..6045985 100644
--- a/cmds-inspect.c
+++ b/cmds-inspect.c
@@ -273,7 +273,7 @@ static int cmd_inspect_subvolid_resolve(int argc, char 
**argv)
if (check_argc_exact(argc - optind, 2))
usage(cmd_inspect_subvolid_resolve_usage);

-   fd = btrfs_open_dir(argv[optind], , 1);
+   fd = btrfs_open_dir(argv[optind + 1], , 1);
if (fd < 0) {
ret = -ENOENT;
goto out;
--
2.7.0
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 4/5] btrfs-progs: "qgroup assign" can't handle options

[Satoru Takeuchi]

"qgroup assign" is considered as working without any options
from the following commit.

commit 176aeca9a148 ("btrfs-progs: add getopt stubs where needed")

However, we can pass options to this command.

* actual result

  ==
  # ./btrfs qgroup assign --rescan 0/260 1/261 /btrfs
  btrfs qgroup assign: unrecognized option '--rescan'
  usage: btrfs qgroup assign [options]   

  Assign SRC as the child qgroup of DST

  --rescan   schedule qutoa rescan if needed
  --no-rescan

  ==

* expected result

  ==
  # ./btrfs qgroup assign --rescan 0/260 1/261 /btrfs
  #
  ======

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
 cmds-qgroup.c | 62 ---
 1 file changed, 25 insertions(+), 37 deletions(-)

diff --git a/cmds-qgroup.c b/cmds-qgroup.c
index 7ae2253..ebd66ef 100644
--- a/cmds-qgroup.c
+++ b/cmds-qgroup.c
@@ -32,7 +32,8 @@ static const char * const qgroup_cmd_group_usage[] = {
NULL
 };

-static int _cmd_qgroup_assign(int assign, int argc, char **argv)
+static int _cmd_qgroup_assign(int assign, int argc, char **argv,
+ const char * const *usage_str)
 {
int ret = 0;
int fd;
@@ -41,28 +42,31 @@ static int _cmd_qgroup_assign(int assign, int argc, char 
**argv)
struct btrfs_ioctl_qgroup_assign_args args;
DIR *dirstream = NULL;

-   while (1) {
-   enum { GETOPT_VAL_RESCAN = 256 };
-   static const struct option long_options[] = {
-   { "rescan", no_argument, NULL, GETOPT_VAL_RESCAN },
-   { NULL, 0, NULL, 0 }
-   };
-   int c = getopt_long(argc, argv, "", long_options, NULL);
-
-   if (c < 0)
-   break;
-   switch (c) {
-   case GETOPT_VAL_RESCAN:
-   rescan = 1;
-   break;
-   default:
-   /* Usage printed by the caller */
-   return -1;
+   if (assign) {
+   while (1) {
+   enum { GETOPT_VAL_RESCAN = 256 };
+   static const struct option long_options[] = {
+   { "rescan", no_argument, NULL, 
GETOPT_VAL_RESCAN },
+   { NULL, 0, NULL, 0 }
+   };
+   int c = getopt_long(argc, argv, "", long_options, NULL);
+
+   if (c < 0)
+   break;
+   switch (c) {
+   case GETOPT_VAL_RESCAN:
+   rescan = 1;
+   break;
+   default:
+   usage(usage_str);
+   }
}
+   } else {
+   clean_args_no_options(argc, argv, usage_str);
}

if (check_argc_exact(argc - optind, 3))
-   return -1;
+   usage(usage_str);

memset(, 0, sizeof(args));
args.assign = assign;
@@ -208,15 +212,7 @@ static const char * const cmd_qgroup_assign_usage[] = {

 static int cmd_qgroup_assign(int argc, char **argv)
 {
-   int ret;
-
-   clean_args_no_options(argc, argv, cmd_qgroup_assign_usage);
-
-   ret = _cmd_qgroup_assign(1, argc, argv);
-
-   if (ret < 0)
-   usage(cmd_qgroup_assign_usage);
-   return ret;
+   return _cmd_qgroup_assign(1, argc, argv, cmd_qgroup_assign_usage);
 }

 static const char * const cmd_qgroup_remove_usage[] = {
@@ -227,15 +223,7 @@ static const char * const cmd_qgroup_remove_usage[] = {

 static int cmd_qgroup_remove(int argc, char **argv)
 {
-   int ret;
-
-   clean_args_no_options(argc, argv, cmd_qgroup_remove_usage);
-
-   ret = _cmd_qgroup_assign(0, argc, argv);
-
-   if (ret < 0)
-   usage(cmd_qgroup_remove_usage);
-   return ret;
+   return _cmd_qgroup_assign(0, argc, argv, cmd_qgroup_remove_usage);
 }

 static const char * const cmd_qgroup_create_usage[] = {
--
2.7.0
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/5] btrfs-progs: "sub get-default" doesn't work

[Satoru Takeuchi]

On 2016/03/17 3:29, David Sterba wrote:

Hi,

btrfs-progs 4.5-rc1 have been released. The ETA for final release is this
Friday, so please test and report if you find problems. Small fixes or
documentation updates are welcome.


Please apply this patchset. Especially [1/5]~[4/5] fix
the regressions caused by the following commit.

commit 176aeca9a148 ("btrfs-progs: add getopt stubs where needed")

I read whole this commit carefully and probably fixed
all problems in this commit by 1a521af, c742deb,
and this patchset.

Satoru

---
"sub get-default" does't work from the following commit.

commit 176aeca9a148 ("btrfs-progs: add getopt stubs where needed")

* actual result

  ==
  # ./btrfs sub get-default /btrfs
  btrfs subvolume get-default: too few arguments
  usage: btrfs subvolume get-default 

  Get the default subvolume of a filesystem
  ==

* expected result

  ==
  # btrfs sub get-default /btrfs
  ID 5 (FS_TREE)
  ======

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
This patch can be applied to devel branch (commit: 40dc7c504cf0)
---
 cmds-subvolume.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/cmds-subvolume.c b/cmds-subvolume.c
index 32caaa5..3953d7c 100644
--- a/cmds-subvolume.c
+++ b/cmds-subvolume.c
@@ -790,7 +790,7 @@ static int cmd_subvol_get_default(int argc, char **argv)

clean_args_no_options(argc, argv, cmd_subvol_get_default_usage);

-   if (check_argc_exact(argc - optind, 2))
+   if (check_argc_exact(argc - optind, 1))
usage(cmd_subvol_get_default_usage);

subvol = argv[1];
--
2.7.0
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 5/5] btrfs-progs: qgroup assign can't handle --no-rescan option

[Satoru Takeuchi]

* actual result

  ==
  # btrfs qgroup assign --no-rescan 0/260 1/261 /btrfs
  btrfs qgroup assign: unrecognized option '--no-rescan'
  usage: btrfs qgroup assign [options]   

  Assign SRC as the child qgroup of DST

  --rescan   schedule qutoa rescan if needed
  --no-rescan

  ==

* expected result

  ==
  # ./btrfs qgroup assign --no-rescan 0/260 1/261 /btrfs
  #
  ==

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
 cmds-qgroup.c | 11 +--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/cmds-qgroup.c b/cmds-qgroup.c
index ebd66ef..4b4149d 100644
--- a/cmds-qgroup.c
+++ b/cmds-qgroup.c
@@ -44,9 +44,13 @@ static int _cmd_qgroup_assign(int assign, int argc, char 
**argv,

if (assign) {
while (1) {
-   enum { GETOPT_VAL_RESCAN = 256 };
+   enum {
+   GETOPT_VAL_RESCAN = 256,
+   GETOPT_VAL_NO_RESCAN = 257,
+   };
static const struct option long_options[] = {
{ "rescan", no_argument, NULL, 
GETOPT_VAL_RESCAN },
+   { "no-rescan", no_argument, NULL, 
GETOPT_VAL_NO_RESCAN },
{ NULL, 0, NULL, 0 }
};
int c = getopt_long(argc, argv, "", long_options, NULL);
@@ -57,6 +61,9 @@ static int _cmd_qgroup_assign(int assign, int argc, char 
**argv,
case GETOPT_VAL_RESCAN:
rescan = 1;
break;
+   case GETOPT_VAL_NO_RESCAN:
+   rescan = 0;
+   break;
default:
usage(usage_str);
}
@@ -206,7 +213,7 @@ static const char * const cmd_qgroup_assign_usage[] = {
"Assign SRC as the child qgroup of DST",
"",
"--rescan   schedule qutoa rescan if needed",
-   "--no-rescan",
+   "--no-rescandon't schedule quota rescan",
NULL
 };

--
2.7.0
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/5] btrfs-progs: "qgroup create/destroy" don't work

[Satoru Takeuchi]

"qgroup create/destroy" don't work from the following commit.

commit 176aeca9a148 ("btrfs-progs: add getopt stubs where needed")

* actual result

  ==
  # ./btrfs qgroup create 1 /btrfs/sub
  btrfs qgroup create: too few arguments
  usage: btrfs qgroup create  

  Create a subvolume quota group.
  ==
  # btrfs qgroup create 1 /btrfs/sub
  # ./btrfs qgroup destroy 1 /btrfs/sub
  btrfs qgroup destroy: too few arguments
  usage: btrfs qgroup destroy  

  Destroy a quota group.

  ==

* expected result

  ==
  # btrfs qgroup create 1 /btrfs/sub
  # btrfs qgroup destroy 1 /btrfs/sub/
  ======

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
 cmds-qgroup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/cmds-qgroup.c b/cmds-qgroup.c
index c4a0c6b..7ae2253 100644
--- a/cmds-qgroup.c
+++ b/cmds-qgroup.c
@@ -124,7 +124,7 @@ static int _cmd_qgroup_create(int create, int argc, char 
**argv)
struct btrfs_ioctl_qgroup_create_args args;
DIR *dirstream = NULL;

-   if (check_argc_exact(argc - optind, 3))
+   if (check_argc_exact(argc - optind, 2))
return -1;

memset(, 0, sizeof(args));
--
2.7.0
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/2 v2] btrfs-progs: Fix a regression that "property" with -t option doesn't work

[Satoru Takeuchi]

Sorry, I forgot to add "btrfs-progs: " in the subject line.
---
"property" is considered as working without any options
from the following commit.

commit 176aeca9a148 ("btrfs-progs: add getopt stubs where needed")

However, we can pass -t option to this command.

* actual result

  ==
  $ ./btrfs prop list -t f /btrfs
  btrfs property list: invalid option -- 't'
  usage: btrfs property list [-t ] 

  Lists available properties with their descriptions for the given object.

  Please see the help of 'btrfs property get' for a description of
  objects and object types.

  ==

* expected result

  ==
  $ ./btrfs prop list -t f /btrfs
  label   Set/get label of device.
  ======

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com
---
v2: Reflect David's comment.
  - Check the number of non-option arguments
---
 cmds-property.c | 35 ---
 1 file changed, 16 insertions(+), 19 deletions(-)

diff --git a/cmds-property.c b/cmds-property.c
index 5b4da26..eed5f4a 100644
--- a/cmds-property.c
+++ b/cmds-property.c
@@ -294,10 +294,11 @@ out:
 static void parse_args(int argc, char **argv,
   const char * const *usage_str,
   int *types, char **object,
-  char **name, char **value)
+  char **name, char **value, int min_nonopt_args)
 {
int ret;
char *type_str = NULL;
+   int max_nonopt_args = 0;

optind = 1;
while (1) {
@@ -314,6 +315,17 @@ static void parse_args(int argc, char **argv,
}
}

+   if (object)
+   max_nonopt_args++;
+   if (name)
+   max_nonopt_args++;
+   if (value)
+   max_nonopt_args++;
+
+   if (check_argc_min(argc - optind, min_nonopt_args) ||
+   check_argc_max(argc - optind, max_nonopt_args))
+   usage(usage_str);
+
*types = 0;
if (type_str) {
if (!strcmp(type_str, "s") || !strcmp(type_str, "subvol")) {
@@ -379,13 +391,8 @@ static int cmd_property_get(int argc, char **argv)
char *name = NULL;
int types = 0;

-   clean_args_no_options(argc, argv, cmd_property_get_usage);
-
-   if (check_argc_min(argc, 2) || check_argc_max(argc, 5))
-   usage(cmd_property_get_usage);
-
parse_args(argc, argv, cmd_property_get_usage, , , ,
-   NULL);
+  NULL, 1);
if (!object) {
error("invalid arguments");
usage(cmd_property_get_usage);
@@ -415,13 +422,8 @@ static int cmd_property_set(int argc, char **argv)
char *value = NULL;
int types = 0;

-   clean_args_no_options(argc, argv, cmd_property_set_usage);
-
-   if (check_argc_min(argc, 4) || check_argc_max(argc, 6))
-   usage(cmd_property_set_usage);
-
parse_args(argc, argv, cmd_property_set_usage, ,
-   , , );
+  , , , 3);
if (!object || !name || !value) {
error("invalid arguments");
usage(cmd_property_set_usage);
@@ -446,13 +448,8 @@ static int cmd_property_list(int argc, char **argv)
char *object = NULL;
int types = 0;

-   clean_args_no_options(argc, argv, cmd_property_list_usage);
-
-   if (check_argc_min(argc, 2) || check_argc_max(argc, 4))
-   usage(cmd_property_list_usage);
-
parse_args(argc, argv, cmd_property_list_usage,
-   , , NULL, NULL);
+  , , NULL, NULL, 1);
if (!object) {
error("invalid arguments");
usage(cmd_property_list_usage);
-- 
2.5.0
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/2 v2] fix a regression that "property" with -t option doesn't work

[Satoru Takeuchi]

"property" is considered as working without any options
from the following commit.

commit 176aeca9a148 ("btrfs-progs: add getopt stubs where needed")

However, we can pass -t option to this command.

* actual result

  ==
  $ ./btrfs prop list -t f /btrfs
  btrfs property list: invalid option -- 't'
  usage: btrfs property list [-t ] 

  Lists available properties with their descriptions for the given object.

  Please see the help of 'btrfs property get' for a description of
  objects and object types.

  ==

* expected result

  ==
  $ ./btrfs prop list -t f /btrfs
  label   Set/get label of device.
  ======

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com
---
v2: Reflect David's comment.
  - Check the number of non-option arguments
---
 cmds-property.c | 35 ---
 1 file changed, 16 insertions(+), 19 deletions(-)

diff --git a/cmds-property.c b/cmds-property.c
index 5b4da26..eed5f4a 100644
--- a/cmds-property.c
+++ b/cmds-property.c
@@ -294,10 +294,11 @@ out:
 static void parse_args(int argc, char **argv,
   const char * const *usage_str,
   int *types, char **object,
-  char **name, char **value)
+  char **name, char **value, int min_nonopt_args)
 {
int ret;
char *type_str = NULL;
+   int max_nonopt_args = 0;

optind = 1;
while (1) {
@@ -314,6 +315,17 @@ static void parse_args(int argc, char **argv,
}
}

+   if (object)
+   max_nonopt_args++;
+   if (name)
+   max_nonopt_args++;
+   if (value)
+   max_nonopt_args++;
+
+   if (check_argc_min(argc - optind, min_nonopt_args) ||
+   check_argc_max(argc - optind, max_nonopt_args))
+   usage(usage_str);
+
*types = 0;
if (type_str) {
if (!strcmp(type_str, "s") || !strcmp(type_str, "subvol")) {
@@ -379,13 +391,8 @@ static int cmd_property_get(int argc, char **argv)
char *name = NULL;
int types = 0;

-   clean_args_no_options(argc, argv, cmd_property_get_usage);
-
-   if (check_argc_min(argc, 2) || check_argc_max(argc, 5))
-   usage(cmd_property_get_usage);
-
parse_args(argc, argv, cmd_property_get_usage, , , ,
-   NULL);
+  NULL, 1);
if (!object) {
error("invalid arguments");
usage(cmd_property_get_usage);
@@ -415,13 +422,8 @@ static int cmd_property_set(int argc, char **argv)
char *value = NULL;
int types = 0;

-   clean_args_no_options(argc, argv, cmd_property_set_usage);
-
-   if (check_argc_min(argc, 4) || check_argc_max(argc, 6))
-   usage(cmd_property_set_usage);
-
parse_args(argc, argv, cmd_property_set_usage, ,
-   , , );
+  , , , 3);
if (!object || !name || !value) {
error("invalid arguments");
usage(cmd_property_set_usage);
@@ -446,13 +448,8 @@ static int cmd_property_list(int argc, char **argv)
char *object = NULL;
int types = 0;

-   clean_args_no_options(argc, argv, cmd_property_list_usage);
-
-   if (check_argc_min(argc, 2) || check_argc_max(argc, 4))
-   usage(cmd_property_list_usage);
-
parse_args(argc, argv, cmd_property_list_usage,
-   , , NULL, NULL);
+  , , NULL, NULL, 1);
if (!object) {
error("invalid arguments");
usage(cmd_property_list_usage);
-- 
2.5.0
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/2] btrfs-progs: Describe optarg of -m option in the manpage of receive

[Satoru Takeuchi]

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
This patch can be applied to devel branch (commit: 4685a560811a)
---
 Documentation/btrfs-receive.asciidoc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Documentation/btrfs-receive.asciidoc 
b/Documentation/btrfs-receive.asciidoc
index 84b85c1..758eebe 100644
--- a/Documentation/btrfs-receive.asciidoc
+++ b/Documentation/btrfs-receive.asciidoc
@@ -43,7 +43,7 @@ or on EOF.
 --max-errors ::
 Terminate as soon as N errors happened while processing commands from the send
 stream. Default value is 1. A value of 0 means no limit.
--m::
+-m ::
 The root mount point of the destination fs.
 +
 By default the mountpoint is searched in /proc/self/mounts.
-- 
2.5.0
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/4] btrfs-progs: fix a reression that "property" with -t option doesn't work

[Satoru Takeuchi]

On 2016/03/14 21:23, David Sterba wrote:

On Mon, Mar 14, 2016 at 09:12:36AM +0900, Satoru Takeuchi wrote:

--- a/cmds-property.c
+++ b/cmds-property.c
@@ -379,9 +379,7 @@ static int cmd_property_get(int argc, char **argv)
char *name = NULL;
int types = 0;

-   clean_args_no_options(argc, argv, cmd_property_get_usage);
-
-   if (check_argc_min(argc, 2) || check_argc_max(argc, 5))
+   if (check_argc_min(argc, 2))
usage(cmd_property_get_usage);

parse_args(argc, argv, cmd_property_get_usage, , , ,


We still need to check the number of non-option arguments here, when the
optind is set from parse_args.


OK, I'll send a patch which checks the number after getopt.

Thanks,
Satoru




@@ -415,9 +413,7 @@ static int cmd_property_set(int argc, char **argv)
-   if (check_argc_min(argc, 4) || check_argc_max(argc, 6))
+   if (check_argc_min(argc, 4))
usage(cmd_property_set_usage);


...


parse_args(argc, argv, cmd_property_set_usage, ,
@@ -446,9 +442,7 @@ static int cmd_property_list(int argc, char **argv)
-   if (check_argc_min(argc, 2) || check_argc_max(argc, 4))
+   if (check_argc_min(argc, 2))


...


--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] btrfs: Simplify conditions about compress while mapping btrfs flags to inode flags

[Satoru Takeuchi]

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
This patch can be applied to 4.5
---
 fs/btrfs/ioctl.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
index 48aee98..b474e32 100644
--- a/fs/btrfs/ioctl.c
+++ b/fs/btrfs/ioctl.c
@@ -123,10 +123,10 @@ static unsigned int btrfs_flags_to_ioctl(unsigned int 
flags)
if (flags & BTRFS_INODE_NODATACOW)
iflags |= FS_NOCOW_FL;

-   if ((flags & BTRFS_INODE_COMPRESS) && !(flags & BTRFS_INODE_NOCOMPRESS))
-   iflags |= FS_COMPR_FL;
-   else if (flags & BTRFS_INODE_NOCOMPRESS)
+   if (flags & BTRFS_INODE_NOCOMPRESS)
iflags |= FS_NOCOMP_FL;
+   else if (flags & BTRFS_INODE_COMPRESS)
+   iflags |= FS_COMPR_FL;

return iflags;
 }
-- 
2.5.0
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 4/4] btrfs-progs: "device ready" accepts just one device

[Satoru Takeuchi]

* actual result

  ===
  # ./btrfs device ready /dev/sdb foo
  #
  ===

* expecting result

  ===
  # ./btrfs device ready /dev/sdb foo
  btrfs device ready: too many arguments
  usage: btrfs device ready 

  Check device to see if it has all of its devices in cache for mounting

  #
  ===

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
 cmds-device.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/cmds-device.c b/cmds-device.c
index 33da2ce..23656c3 100644
--- a/cmds-device.c
+++ b/cmds-device.c
@@ -326,7 +326,7 @@ static int cmd_device_ready(int argc, char **argv)

clean_args_no_options(argc, argv, cmd_device_ready_usage);

-   if (check_argc_min(argc - optind, 1))
+   if (check_argc_exact(argc - optind, 1))
usage(cmd_device_ready_usage);

fd = open("/dev/btrfs-control", O_RDWR);
-- 
2.7.0
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/4] btrfs-progs: fix a reression that "property" with -t option doesn't work

[Satoru Takeuchi]

"property" is considered as working without any options
from the following commit.

commit 176aeca9a148 ("btrfs-progs: add getopt stubs where needed")

However, we can pass -t option to this command.

* actual result

  ==
  # ./btrfs prop list -t f /btrfs
  btrfs property list: invalid option -- 't'
  usage: btrfs property list [-t ] 

  Lists available properties with their descriptions for the given object.

  Please see the help of 'btrfs property get' for a description of
  objects and object types.
  ==

* expected result

  ==
  # btrfs prop list -t f /btrfs
  label   Set/get label of device
  ======

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
 cmds-property.c | 12 +++-
 1 file changed, 3 insertions(+), 9 deletions(-)

diff --git a/cmds-property.c b/cmds-property.c
index 5b4da26..4c2beb7 100644
--- a/cmds-property.c
+++ b/cmds-property.c
@@ -379,9 +379,7 @@ static int cmd_property_get(int argc, char **argv)
char *name = NULL;
int types = 0;

-   clean_args_no_options(argc, argv, cmd_property_get_usage);
-
-   if (check_argc_min(argc, 2) || check_argc_max(argc, 5))
+   if (check_argc_min(argc, 2))
usage(cmd_property_get_usage);

parse_args(argc, argv, cmd_property_get_usage, , , ,
@@ -415,9 +413,7 @@ static int cmd_property_set(int argc, char **argv)
char *value = NULL;
int types = 0;

-   clean_args_no_options(argc, argv, cmd_property_set_usage);
-
-   if (check_argc_min(argc, 4) || check_argc_max(argc, 6))
+   if (check_argc_min(argc, 4))
usage(cmd_property_set_usage);

parse_args(argc, argv, cmd_property_set_usage, ,
@@ -446,9 +442,7 @@ static int cmd_property_list(int argc, char **argv)
char *object = NULL;
int types = 0;

-   clean_args_no_options(argc, argv, cmd_property_list_usage);
-
-   if (check_argc_min(argc, 2) || check_argc_max(argc, 4))
+   if (check_argc_min(argc, 2))
usage(cmd_property_list_usage);

parse_args(argc, argv, cmd_property_list_usage,
-- 
2.7.0
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/4] btrfs-progs: Avoid interpreting options after "--" when getting unit mode

[Satoru Takeuchi]

* actual result

  ==
  # ./btrfs device usage -- -m /btrfs

  /dev/sdf1, ID: 1
 Device size:  95367.41MiB
 Data,single:  2056.00MiB
 Metadata,DUP: 2048.00MiB
 System,DUP: 16.00MiB
 Unallocated:  91247.41MiB
  ==

* expected result

  ==
  # ./btrfs device usage -- -m /btrfs

  ERROR: can't access '-m': No such file or directory
  ==

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
This patch can be applied to devel branch (commit: 31266d0ef811).
---
 utils.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/utils.c b/utils.c
index d7ceaa8..77f0f68 100644
--- a/utils.c
+++ b/utils.c
@@ -3024,6 +3024,9 @@ unsigned int get_unit_mode_from_arg(int *argc, char 
*argv[], int df_mode)
int arg_end;

for (arg_i = 0; arg_i < *argc; arg_i++) {
+   if (!strcmp(argv[arg_i], "--"))
+   break;
+
if (!strcmp(argv[arg_i], "--raw")) {
unit_mode = UNITS_RAW;
argv[arg_i] = NULL;
-- 
2.7.0
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How to move a btrfs volume to a smaller disk

[Satoru Takeuchi]

On 2016/03/10 6:46, Hendrik Friedel wrote:

Hello,

I intend to move this subvolume to a new device.
btrfs fi show /mnt2/Data_Store/
Label: 'Data_Store'  uuid: 0ccc1e24-090d-42e2-9e61-d0a1b3101f93
 Total devices 1 FS bytes used 47.93GiB
 devid1 size 102.94GiB used 76.03GiB path /dev/sdb4

(fi usage at the bottom of this message)

The new device (sda4) is 8G smaller unfortunately.
sda   8:00 111.8G  0 disk
└─sda48:40 103.5G  0 part
sdb   8:16   0 119.2G  0 disk
└─sdb48:20   0   111G  0 part  /mnt2/Data_Store

Thus, btrfs replace does not work

What would you suggest now to move the FS (it does contain many subvolumes)?

I tried btrfs send /mnt2/Data_Store/read_only_snapshot/ | btrfs receive 
/mnt/sda4/
but this only created an empty subvolume /mnt/sda4/read_only_snapshot/


I suspect that you captured snapshot as

  ==
  btrfs sub snap -r /mnt2/Data_Store /mnt2/Data_Store/read_only_snapshot
  ==

and /mnt2/Data_Store only contains subvolumes
instead of having files/directries directly.
Then send/receive work as you said. Probably if you run
"ls -li /mnt2/Data_Store", all files have
inode 256 (it means subvolume). In addition,
all the directories under
/mnt2/Data_Store/read_only_snapshot are empty.

It's because "sub snap" only handles one subvolume
and doesn't handle subvolumes inside it. So, if you'd
like to copy all the data by send/receive, you should
capture and send/receive snapshot for each subvolume.



So, then
btrfs device add /dev/sda4 /mnt/Data_Store
btrfs balance start /mnt/Data_Store
btrfs device remove /dev/sdb4 /mnt/Data_Store
?


balance is not necessary. When you run
device remove /dev/sdb4, then all the contents of
/dev/sdb4 migrate to /dev/sda4 properly.

Thanks,
Satoru



Or is there a better option?

Regards,
Hendrik


  btrfs fi usage  /mnt2/Data_Store/
Overall:
 Device size: 102.94GiB
 Device allocated: 74.03GiB
 Device unallocated:   28.91GiB
 Device missing:  0.00B
 Used: 47.96GiB
 Free (estimated): 53.24GiB  (min: 53.24GiB)
 Data ratio:   1.00
 Metadata ratio:   1.00
 Global reserve:  512.00MiB  (used: 0.00B)

Data,single: Size:69.00GiB, Used:44.67GiB
/dev/sdb4  69.00GiB

Metadata,single: Size:5.00GiB, Used:3.29GiB
/dev/sdb4   5.00GiB

System,single: Size:32.00MiB, Used:16.00KiB
/dev/sdb4  32.00MiB

Unallocated:
/dev/sdb4  28.91GiB


---
Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft.
https://www.avast.com/antivirus

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] btrfs-progs: Fix device scan to interpret its argument properly

[Satoru Takeuchi]

Fix the following bug.

  
  # btrfs device scan -- /dev/sdb
  ERROR: not a block device: --
  

It should work as follow.

  
  # ./btrfs device scan -- /dev/sdb
  Scanning for Btrfs filesystems in '/dev/sdb'
  

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
This patch can be applied to devel (commit: 5d038a7ed212)
---
 cmds-device.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/cmds-device.c b/cmds-device.c
index 94ffdc5..e500638 100644
--- a/cmds-device.c
+++ b/cmds-device.c
@@ -246,7 +246,7 @@ static const char * const cmd_device_scan_usage[] = {
 static int cmd_device_scan(int argc, char **argv)
 {
int i;
-   int devstart = 1;
+   int devstart;
int all = 0;
int ret = 0;

@@ -269,6 +269,7 @@ static int cmd_device_scan(int argc, char **argv)
usage(cmd_device_scan_usage);
}
}
+   devstart = optind;

if (all && check_argc_max(argc - optind, 1))
usage(cmd_device_scan_usage);
-- 
2.5.0
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/2] btrfs-progs: Remove unnecessary variable devstart from cmd_device_scan()

[Satoru Takeuchi]

On 2016/03/10 22:27, David Sterba wrote:

On Thu, Mar 10, 2016 at 08:04:59AM +0900, Satoru Takeuchi wrote:

It's unnecessary since it's always 1.

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
This patch can be applied to devel (commit b2bdd0da8969).
---
  cmds-device.c | 3 +--
  1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/cmds-device.c b/cmds-device.c
index cb470af..78d6535 100644
--- a/cmds-device.c
+++ b/cmds-device.c
@@ -246,7 +246,6 @@ static const char * const cmd_device_scan_usage[] = {
  static int cmd_device_scan(int argc, char **argv)
  {
int i;
-   int devstart = 1;
int all = 0;
int ret = 0;

@@ -282,7 +281,7 @@ static int cmd_device_scan(int argc, char **argv)
goto out;
}

-   for( i = devstart ; i < argc ; i++ ){
+   for (i = 1; i < argc; i++) {


Actually it needs to be set to optind before the for cycle, otherwise:

   # btrfs device scan -- /dev/sda
   ERROR: not a block device: --


Thank you for your comment. I'll fix it.

Satoru
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/2] btrfs-progs: Describe device scan -d is a deprecated option in manpage

[Satoru Takeuchi]

It's already marked as deprecated in cmd_device_scan_usage().

commit 5444864e5605 ("btrfs-progs: remove BTRFS_SCAN_PROC scan method")

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
 Documentation/btrfs-device.asciidoc | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/Documentation/btrfs-device.asciidoc 
b/Documentation/btrfs-device.asciidoc
index bd878f4..bf16a94 100644
--- a/Documentation/btrfs-device.asciidoc
+++ b/Documentation/btrfs-device.asciidoc
@@ -89,8 +89,8 @@ Scan devices for a btrfs filesystem.
 If one or more devices are passed, these are scanned for a btrfs filesystem.
 If no devices are passed, btrfs uses block devices containing btrfs
 filesystem as listed by blkid.
-Finally, if '--all-devices' or '-d' is passed, all the devices under /dev are
-scanned.
+Finally, '--all-devices' or '-d' is the deprecated option. If it is passed,
+its behavior is the same as if no devices are passed.

 *stats* [-z] |::
 Read and print the device IO stats for all mounted devices of the filesystem
-- 
2.7.0
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/2] btrfs-progs: Remove unnecessary variable devstart from cmd_device_scan()

[Satoru Takeuchi]

It's unnecessary since it's always 1.

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
This patch can be applied to devel (commit b2bdd0da8969).
---
 cmds-device.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/cmds-device.c b/cmds-device.c
index cb470af..78d6535 100644
--- a/cmds-device.c
+++ b/cmds-device.c
@@ -246,7 +246,6 @@ static const char * const cmd_device_scan_usage[] = {
 static int cmd_device_scan(int argc, char **argv)
 {
int i;
-   int devstart = 1;
int all = 0;
int ret = 0;

@@ -282,7 +281,7 @@ static int cmd_device_scan(int argc, char **argv)
goto out;
}

-   for( i = devstart ; i < argc ; i++ ){
+   for (i = 1; i < argc; i++) {
char *path;

if (is_block_device(argv[i]) != 1) {
-- 
2.7.0

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Fix broken 'device scan' arguments parsing

[Satoru Takeuchi]

On 2016/03/09 10:19, Yauhen Kharuzhy wrote:
> commit 52179e4fea41e55f31c92cd033a0b53a5107b4f4 'btrfs-progs: unify argc
> min/max checking' brokes 'btrfs device scan' command when no argument
> was given. Fix this.
> 
> Signed-off-by: Yauhen Kharuzhy 
> ---
>   cmds-device.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/cmds-device.c b/cmds-device.c
> index cb470af..94ffdc5 100644
> --- a/cmds-device.c
> +++ b/cmds-device.c
> @@ -273,7 +273,7 @@ static int cmd_device_scan(int argc, char **argv)
>   if (all && check_argc_max(argc - optind, 1))

It's better to the above line as follows.


if (all && check_argc_exact(argc - optind, 0))


>   usage(cmd_device_scan_usage);
>   
> - if (all || argc - optind == 1) {
> + if (all || argc - optind == 0) {
>   printf("Scanning for Btrfs filesystems\n");
>   ret = btrfs_scan_lblkid();
>   error_on(ret, "error %d while scanning", ret);
> 

Here is the test result.

* v4.4.1

==
# ./btrfs device scan
Scanning for Btrfs filesystems
# ./btrfs device scan -d
Scanning for Btrfs filesystems
# ./btrfs device scan -d foo
btrfs device scan: too many arguments
usage: btrfs device scan [(-d|--all-devices)| [...]]

Scan devices for a btrfs filesystem

 -d|--all-devices (deprecated)

==

* devel branch with this patch.

==
# ./btrfs device scan
Scanning for Btrfs filesystems
# ./btrfs device scan -d
Scanning for Btrfs filesystems
# ./btrfs device scan -d foo
Scanning for Btrfs filesystems
===

* devel branch without this patch.

==
# ./btrfs device scan
# ./btrfs device scan -d
Scanning for Btrfs filesystems
# ./btrfs device scan -d foo
Scanning for Btrfs filesystems
===

* devel branch with this patch and my patch.

=
# ./btrfs device scan
Scanning for Btrfs filesystems
# ./btrfs device scan -d
Scanning for Btrfs filesystems
# ./btrfs device scan -d foo
btrfs device scan: too many arguments
usage: btrfs device scan [(-d|--all-devices)| [...]]

Scan devices for a btrfs filesystem

 -d|--all-devices (deprecated)

===

Thanks,
Satoru
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3] Btrfs: Show a warning message if one of objectid reaches its highest value

[Satoru Takeuchi]

It's better to show a warning message for the exceptional case
that one of objectid (in most case, inode number) reaches its
highest value. For example, if inode cache is off and this event
happens, we can't create any file even if there are not so many files.
This message ease detecting such problem.

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
This patch can be applied to 4.5-rc7

V3:
- Show this message every time when hitting this problem
  for simplicity (thanks to the comment from Goffredo).
- Reflect Naota's comment
  - Keep the error code as is.
V2:
 - Reflect Filipe's comment
   - Use btrfs_warn() instead of WARN_ONCE()
   - Print the id of the tree
---
 fs/btrfs/inode-map.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/fs/btrfs/inode-map.c b/fs/btrfs/inode-map.c
index e50316c..5d97ee7 100644
--- a/fs/btrfs/inode-map.c
+++ b/fs/btrfs/inode-map.c
@@ -556,6 +556,9 @@ int btrfs_find_free_objectid(struct btrfs_root *root, u64 
*objectid)
mutex_lock(>objectid_mutex);

if (unlikely(root->highest_objectid >= BTRFS_LAST_FREE_OBJECTID)) {
+   btrfs_warn(root->fs_info,
+  "The objectid of root %llu reaches its highest 
value.\n",
+  root->root_key.objectid);
ret = -ENOSPC;
goto out;
}
-- 
2.5.0
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] Btrfs: Show a warning message if one of objectid reaches its highest value

[Satoru Takeuchi]

On 2016/03/09 11:32, Naohiro Aota wrote:

2016-03-07 12:05 GMT+09:00 Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>:

- It's better to show a warning message for the exceptional case
   that one of objectid (in most case, inode number) reaches its
   highest value. Show this message only once to avoid filling
   dmesg with it.
- EOVERFLOW is more proper return value for this case.
   ENOSPC is for "No space left on device" case and objectid isn't
   related to any device.


I have concern about EOVERFLOW. The value returned here will
go through to the user space via btrfs_find_free_ino() and btrfs_create().
It means that "creat" and "mkdir" can now return EOVERFLOW when it
failed to assign new inode number. Such behavior would disagree with
other file systems, which result in user space programs to be
confused.

Also, I don't think EOVERFLOW described in "creat(2)" (or open(2))
suits for this case. As far as I read the following man page from
creat(2),
giving ENOSPC is better option here.


I consider, as I read man, ENOSPC is also doesn't explain
this case. It's not related to pathname.

man 2 creat:

ENOSPC pathname was to be created but the device
containing pathname has no room for the new file.


However, I agree with the ENOSPC is better than EOVERFLOW
because existing code has worked with the former value
in such case.

Next patch will keep the error code as is.

Thank you for your comment, Naota.

Satoru




ENOSPC: pathname was to be created but the device containing pathname has  no  
room  for  the  new file.
EOVERFLOW:
  pathname refers to a regular file that is too large to be opened.
(snip)



Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
This patch can be applied to 4.5-rc7
---
  fs/btrfs/inode-map.c | 10 +-
  1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/fs/btrfs/inode-map.c b/fs/btrfs/inode-map.c
index e50316c..f5e3228 100644
--- a/fs/btrfs/inode-map.c
+++ b/fs/btrfs/inode-map.c
@@ -556,7 +556,15 @@ int btrfs_find_free_objectid(struct btrfs_root *root, u64 
*objectid)
 mutex_lock(>objectid_mutex);

 if (unlikely(root->highest_objectid >= BTRFS_LAST_FREE_OBJECTID)) {
-   ret = -ENOSPC;
+   static bool __warned = false;
+
+   if (unlikely(!__warned)) {
+   btrfs_warn(root->fs_info,
+  "The objectid of root %llu reaches its highest 
value.\n",
+  root->root_key.objectid);
+   __warned = true;
+   }
+   ret = -EOVERFLOW;
 goto out;
 }

--
2.5.0
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] Btrfs: Show a warning message if one of objectid reaches its highest value

[Satoru Takeuchi]

On 2016/03/09 4:24, Goffredo Baroncelli wrote:
> On 2016-03-07 04:05, Satoru Takeuchi wrote:
>> - It's better to show a warning message for the exceptional case
>> that one of objectid (in most case, inode number) reaches its
>> highest value. Show this message only once to avoid filling
>> dmesg with it.
>> - EOVERFLOW is more proper return value for this case.
>> ENOSPC is for "No space left on device" case and objectid isn't
>> related to any device.
>>
>> Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
>> ---
>> This patch can be applied to 4.5-rc7
>> ---
>>fs/btrfs/inode-map.c | 10 +-
>>1 file changed, 9 insertions(+), 1 deletion(-)
>>
>> diff --git a/fs/btrfs/inode-map.c b/fs/btrfs/inode-map.c
>> index e50316c..f5e3228 100644
>> --- a/fs/btrfs/inode-map.c
>> +++ b/fs/btrfs/inode-map.c
>> @@ -556,7 +556,15 @@ int btrfs_find_free_objectid(struct btrfs_root *root, 
>> u64 *objectid)
>>  mutex_lock(>objectid_mutex);
>>
>>  if (unlikely(root->highest_objectid >= BTRFS_LAST_FREE_OBJECTID)) {
>> -ret = -ENOSPC;
>> +static bool __warned = false;
> 
> 
> Please, don't use a static GLOBAL variable. I suggest to move to a "per 
> filesystem" variables for two main reasons:
> 
> 1) if in the (very unlikely) case where two different filesystems reach 
> BTRFS_LAST_FREE_OBJECTID, the first error will hide the second one.
> 
> 2) if you umount and remount the filesystem the error is not shown anymore. A 
> module unload/load or a reboot is required.
> If something strange happens, one of the first thing that the user does, is 
> to umount/remount the filesystem. But the error is not show anymore. This 
> could complicate the diagnosis of the problem.

OK, I see.

I rethink what should this patch be since it's
overkill to prepare per-filesystem variable just
for this warning message.

I'll resend patch which shows this message
every time when hitting this condition instead of
does it just once.

Thanks,
Satoru

> 
> 
> 
> 
>> +
>> +if (unlikely(!__warned)) {
>> +btrfs_warn(root->fs_info,
>> +   "The objectid of root %llu reaches its 
>> highest value.\n",
>> +   root->root_key.objectid);
>> +__warned = true;
>> +}
>> +ret = -EOVERFLOW;
>>  goto out;
>>  }
>>
> 
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Documentation: btrfs: remove usage specific information

[Satoru Takeuchi]

On 2016/03/09 2:04, David Sterba wrote:
> The document in the kernel sources is yet another palce where the
> documentation would need to be updated, while it is not the primary
> source. We actively maintain the wiki pages.
> 
> Signed-off-by: David Sterba 
> ---
>   Documentation/filesystems/btrfs.txt | 261 
> ++--
>   1 file changed, 11 insertions(+), 250 deletions(-)
> 
> diff --git a/Documentation/filesystems/btrfs.txt 
> b/Documentation/filesystems/btrfs.txt
> index c772b47e7ef0..f9dad22d95ce 100644
> --- a/Documentation/filesystems/btrfs.txt
> +++ b/Documentation/filesystems/btrfs.txt
> @@ -1,20 +1,10 @@
> -
>   BTRFS
>   =
>   
...
> -* btrfs-convert: in-place conversion from ext2/3/4 filesystems
> +  https://btrfs.wiki.kernel.org
>   
> -* btrfs-image: dump filesystem metadata for debugging
> +that maintains information about administration tasks, frequently asked
> +questions, use cases, mount options, comprehensible changelogs, features,
> +manual pages, source code repositories, contacts etc.

About mount options, we also have "man 5 btrfs" and it's
newer than wiki page's one.

https://www.mail-archive.com/linux-btrfs%40vger.kernel.org/msg34545.html
commit 26341f734d6d ("btrfs-progs: add mount options to btrfs-mount.5")

Last update:
 - Btrfs wiki -> Mount options: Sep 19 2015
 - btrfs-man5.asciidoc: Jan 11 2016

So, how about refer to "man 5 btrfs" from
"Btrfs wiki -> Mount options -> List of options"
and/or this document?

Thanks,
Satoru
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Btrfsck memory usage reduce idea

[Satoru Takeuchi]

On 2016/03/08 17:46, Qu Wenruo wrote:


Satoru Takeuchi wrote on 2016/03/08 17:28 +0900:

Hi Qu,

On 2016/03/07 14:42, Qu Wenruo wrote:

Hi,

As many have already known, "btrfs check" is a memory eater.

The problem is, btrfsck checks extent tree in a very comprehensive
method.
1) Create extent_ref for each extent item with backref
2) Iterate all other trees to add extent ref
3) If one extent_ref with all ref/backref matches, it's deleted.

The method is good, can found any extent mismatch problem when
checking extent tree. (Although it has already iterated the whole fs)
For a large enough filesystem, it may have tegas of extents, and
memory is easy eaten up.


We hope to fix it in the following method:
1) Only check extent backref when iterating extent tree
Unlike current implement, we check one extent item and its backref
only.

If one backref can't be reached, then it's an error and output (or
try to fix).
After iterating all backref of an extent item, all related memory is
freed and we won't bother recording anything for later use.

That's to say, we only care backref mismatch case when checking
extent tree.
Case like missing EXTENT_ITEM for some extent is not checked here.

2) Check extent ref while iterating other trees
We only check forward-ref while iterating one tree.

In this step, we only check forward-ref, so we can find the remaining
problem like missing EXTENT_ITEM for given extent.

Any further advice/suggestion? Or is there anyone already doing such
work?


Thank you for your effort. I have basic questions.

1. Could you tell me what you'd like to do?

a) Provide completely the same function with current
   implementation by other, more efficient way.


Same function, but less efficient.
It may takes longer time, more IO, but less memory.


I see.



And some error message will be output at different time.
E.g, error message for missing backref may be output at fs tree checking time, 
instead of extent tree checking time.


It's OK if, finally, all error messages the same as the current
implementation are shown.




b) Replace the current implementation with the quicker
   one that provides the limited function.
c) Any other

2. Do you have the estimation that how long does the
new algorithm take compare with the current one?


Depends on the fs hierarchy. But in all case, IO will be more than original 
implement.

The most efficient case would be, one subvolume and no dedup file.
(which means one file extent refer to one extent on data, no in-band or
out-band dedup).

In that case, old implement will iterate the whole metadata twice,
and new implement will iterate the whole metadata twice + extra.

For worst case, like inband dedup with multiple almost identical snapshot,

> things will be much much slower, more IO, more tree search, maybe O(n^2)
> or more. But memory usage should not be much different though.


In short, use more IO to trade for memory.

Anyway, for a large fs, it won't be possible to take a short time for a 
comprehensive fsck.


Got it.

Thanks,
Satoru



Thanks,
Qu


# Of course, "currently not sure" is OK at this stage.

I'm interested in it because there is the trade-off
between speed and memory consumption in many case,
and btrfsck takes very long time with a large filesystem.

Thanks,
Satoru



Thanks,
Qu



--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html






--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Btrfsck memory usage reduce idea

[Satoru Takeuchi]

Hi Qu,

On 2016/03/07 14:42, Qu Wenruo wrote:

Hi,

As many have already known, "btrfs check" is a memory eater.

The problem is, btrfsck checks extent tree in a very comprehensive method.
1) Create extent_ref for each extent item with backref
2) Iterate all other trees to add extent ref
3) If one extent_ref with all ref/backref matches, it's deleted.

The method is good, can found any extent mismatch problem when checking extent 
tree. (Although it has already iterated the whole fs)
For a large enough filesystem, it may have tegas of extents, and memory is easy 
eaten up.


We hope to fix it in the following method:
1) Only check extent backref when iterating extent tree
Unlike current implement, we check one extent item and its backref
only.

If one backref can't be reached, then it's an error and output (or
try to fix).
After iterating all backref of an extent item, all related memory is
freed and we won't bother recording anything for later use.

That's to say, we only care backref mismatch case when checking
extent tree.
Case like missing EXTENT_ITEM for some extent is not checked here.

2) Check extent ref while iterating other trees
We only check forward-ref while iterating one tree.

In this step, we only check forward-ref, so we can find the remaining
problem like missing EXTENT_ITEM for given extent.

Any further advice/suggestion? Or is there anyone already doing such work?


Thank you for your effort. I have basic questions.

1. Could you tell me what you'd like to do?

   a) Provide completely the same function with current
  implementation by other, more efficient way.
   b) Replace the current implementation with the quicker
  one that provides the limited function.
   c) Any other

2. Do you have the estimation that how long does the
   new algorithm take compare with the current one?

   # Of course, "currently not sure" is OK at this stage.

   I'm interested in it because there is the trade-off
   between speed and memory consumption in many case,
   and btrfsck takes very long time with a large filesystem.

Thanks,
Satoru



Thanks,
Qu



--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] Btrfs: Show a warning message if one of objectid reaches its highest value

[Satoru Takeuchi]

On 2016/03/07 12:05, Satoru Takeuchi wrote:
> - It's better to show a warning message for the exceptional case
>that one of objectid (in most case, inode number) reaches its
>highest value. Show this message only once to avoid filling
>dmesg with it.
> - EOVERFLOW is more proper return value for this case.
>ENOSPC is for "No space left on device" case and objectid isn't
>related to any device.
> 
> Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
> ---
> This patch can be applied to 4.5-rc7

Sorry, I forgot to add changelog.

===
V2:
 - Reflect Filipe's comment
   - Use btrfs_warn() instead of WARN_ONCE()
   - Print the id of the tree
===

Thanks,
Satoru

> ---
>   fs/btrfs/inode-map.c | 10 +-
>   1 file changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/fs/btrfs/inode-map.c b/fs/btrfs/inode-map.c
> index e50316c..f5e3228 100644
> --- a/fs/btrfs/inode-map.c
> +++ b/fs/btrfs/inode-map.c
> @@ -556,7 +556,15 @@ int btrfs_find_free_objectid(struct btrfs_root *root, 
> u64 *objectid)
>   mutex_lock(>objectid_mutex);
> 
>   if (unlikely(root->highest_objectid >= BTRFS_LAST_FREE_OBJECTID)) {
> - ret = -ENOSPC;
> + static bool __warned = false;
> +
> + if (unlikely(!__warned)) {
> + btrfs_warn(root->fs_info,
> +"The objectid of root %llu reaches its 
> highest value.\n",
> +root->root_key.objectid);
> + __warned = true;
> + }
> + ret = -EOVERFLOW;
>   goto out;
>   }
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2] Btrfs: Show a warning message if one of objectid reaches its highest value

[Satoru Takeuchi]

- It's better to show a warning message for the exceptional case
  that one of objectid (in most case, inode number) reaches its
  highest value. Show this message only once to avoid filling
  dmesg with it.
- EOVERFLOW is more proper return value for this case.
  ENOSPC is for "No space left on device" case and objectid isn't
  related to any device.

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
This patch can be applied to 4.5-rc7
---
 fs/btrfs/inode-map.c | 10 +-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/fs/btrfs/inode-map.c b/fs/btrfs/inode-map.c
index e50316c..f5e3228 100644
--- a/fs/btrfs/inode-map.c
+++ b/fs/btrfs/inode-map.c
@@ -556,7 +556,15 @@ int btrfs_find_free_objectid(struct btrfs_root *root, u64 
*objectid)
mutex_lock(>objectid_mutex);

if (unlikely(root->highest_objectid >= BTRFS_LAST_FREE_OBJECTID)) {
-   ret = -ENOSPC;
+   static bool __warned = false;
+
+   if (unlikely(!__warned)) {
+   btrfs_warn(root->fs_info,
+  "The objectid of root %llu reaches its 
highest value.\n",
+  root->root_key.objectid);
+   __warned = true;
+   }
+   ret = -EOVERFLOW;
goto out;
}

-- 
2.5.0
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Show a warning message if one of highest objectid reaches its max value

[Satoru Takeuchi]

Hi Filipe,

On 2016/03/04 18:28, Filipe Manana wrote:

On Fri, Mar 4, 2016 at 2:37 AM, Satoru Takeuchi
<takeuchi_sat...@jp.fujitsu.com> wrote:

- It's better to show a warning message for the exceptional case
   that one of highest objectid (in most case, inode number)
   reaches its max value, BTRFS_LAST_FREE_OBJECTID. Show this
   message only once to avoid filling dmesg with it.
- EOVERFLOW is more proper return value for this case.
   ENOSPC is for "No space left on device" case and objectid isn't
   related to any device.

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
This patch can be applied to v4.5-rc6
---
  fs/btrfs/inode-map.c | 6 --
  1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/fs/btrfs/inode-map.c b/fs/btrfs/inode-map.c
index e50316c..a4860fd 100644
--- a/fs/btrfs/inode-map.c
+++ b/fs/btrfs/inode-map.c
@@ -555,8 +555,10 @@ int btrfs_find_free_objectid(struct btrfs_root *root, u64 
*objectid)
 int ret;
 mutex_lock(>objectid_mutex);

-   if (unlikely(root->highest_objectid >= BTRFS_LAST_FREE_OBJECTID)) {
-   ret = -ENOSPC;
+   if (WARN_ONCE(root->highest_objectid >= BTRFS_LAST_FREE_OBJECTID,
+ "BTRFS: The highest objectid reaches its max value 
%llu.\n",
+ BTRFS_LAST_FREE_OBJECTID)) {


Please, use btrfs_warn() to print messages... It avoids hardcoding
prefixes and having to copy the style of the
btrfs_[info|warn|error|...) functions.
Also include the id of the tree (root->root_key.objectid), it's much
more useful than printing root->highes_objectid

Also add the prefix "Btrfs: " to the subject.


Thank you for your comment. I'll fix it.



+   ret = -EOVERFLOW;
 goto out;
 }

--
2.5.0
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html





--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] Show a warning message if one of highest objectid reaches its max value

[Satoru Takeuchi]

- It's better to show a warning message for the exceptional case
  that one of highest objectid (in most case, inode number)
  reaches its max value, BTRFS_LAST_FREE_OBJECTID. Show this
  message only once to avoid filling dmesg with it.
- EOVERFLOW is more proper return value for this case.
  ENOSPC is for "No space left on device" case and objectid isn't
  related to any device.

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
This patch can be applied to v4.5-rc6
---
 fs/btrfs/inode-map.c | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/fs/btrfs/inode-map.c b/fs/btrfs/inode-map.c
index e50316c..a4860fd 100644
--- a/fs/btrfs/inode-map.c
+++ b/fs/btrfs/inode-map.c
@@ -555,8 +555,10 @@ int btrfs_find_free_objectid(struct btrfs_root *root, u64 
*objectid)
int ret;
mutex_lock(>objectid_mutex);

-   if (unlikely(root->highest_objectid >= BTRFS_LAST_FREE_OBJECTID)) {
-   ret = -ENOSPC;
+   if (WARN_ONCE(root->highest_objectid >= BTRFS_LAST_FREE_OBJECTID,
+ "BTRFS: The highest objectid reaches its max value 
%llu.\n",
+ BTRFS_LAST_FREE_OBJECTID)) {
+   ret = -EOVERFLOW;
goto out;
}

-- 
2.5.0
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Btrfs: fix listxattrs not listing all xattrs packed in the same item

[Satoru Takeuchi]

On 2016/02/23 2:52, fdman...@kernel.org wrote:

From: Filipe Manana <fdman...@suse.com>

In the listxattrs handler, we were not listing all the xattrs that are
packed in the same btree item, which happens when multiple xattrs have
a name that when crc32c hashed produce the same checksum value.

Fix this by processing them all.

The following test case for xfstests reproduces the issue:

   seq=`basename $0`
   seqres=$RESULT_DIR/$seq
   echo "QA output created by $seq"
   tmp=/tmp/$$
   status=1 # failure is the default!
   trap "_cleanup; exit \$status" 0 1 2 3 15

   _cleanup()
   {
   cd /
   rm -f $tmp.*
   }

   # get standard environment, filters and checks
   . ./common/rc
   . ./common/filter
   . ./common/attr

   # real QA test starts here
   _supported_fs generic
   _supported_os Linux
   _require_scratch
   _require_attrs

   rm -f $seqres.full

   _scratch_mkfs >>$seqres.full 2>&1
   _scratch_mount

   # Create our test file with a few xattrs. The first 3 xattrs have a name
   # that when given as input to a crc32c function result in the same checksum.
   # This made btrfs list only one of the xattrs through listxattrs system call
   # (because it packs xattrs with the same name checksum into the same btree
   # item).
   touch $SCRATCH_MNT/testfile
   $SETFATTR_PROG -n user.foobar -v 123 $SCRATCH_MNT/testfile
   $SETFATTR_PROG -n user.WvG1c1Td -v qwerty $SCRATCH_MNT/testfile
   $SETFATTR_PROG -n user.J3__T_Km3dVsW_ -v hello $SCRATCH_MNT/testfile
   $SETFATTR_PROG -n user.something -v pizza $SCRATCH_MNT/testfile
   $SETFATTR_PROG -n user.ping -v pong $SCRATCH_MNT/testfile

   # Now call getfattr with --dump, which calls the listxattrs system call.
   # It should list all the xattrs we have set before.
   $GETFATTR_PROG --absolute-names --dump $SCRATCH_MNT/testfile | 
_filter_scratch

   status=0
   exit


Tested-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>

* chris/integration-4.6(HEAD is 790dd8b)

===
# ./check generic/337
FSTYP -- btrfs
PLATFORM  -- Linux/x86_64 fedora23 4.2.7-300.fc23.x86_64
MKFS_OPTIONS  -- /dev/vdc
MOUNT_OPTIONS -- -o context=system_u:object_r:nfs_t:s0 /dev/vdc /scratch_mnt

generic/337  - output mismatch (see 
/root/xfstests/results//generic/337.out.bad)
--- tests/generic/337.out   2016-02-24 07:26:33.0 +0900
+++ /root/xfstests/results//generic/337.out.bad 2016-02-24 
07:43:02.47100 +0900
@@ -1,7 +1,5 @@
 QA output created by 337
 # file: SCRATCH_MNT/testfile
-user.J3__T_Km3dVsW_="hello"
-user.WvG1c1Td="qwerty"
 user.foobar="123"
 user.ping="pong"
 user.something="pizza"
...
(Run 'diff -u tests/generic/337.out 
/root/xfstests/results//generic/337.out.bad'  to see the entire diff)
Ran: generic/337
Failures: generic/337
Failed 1 of 1 tests


* the above source + your patch


# ./check generic/337
FSTYP -- btrfs
PLATFORM  -- Linux/x86_64 fedora23 4.5.0-rc3-ktest+
MKFS_OPTIONS  -- /dev/vdc
MOUNT_OPTIONS -- /dev/vdc /scratch_mnt

generic/337  0s
Ran: generic/337
Passed all 1 tests


Thanks,
Satoru



Signed-off-by: Filipe Manana <fdman...@suse.com>
---
  fs/btrfs/xattr.c | 62 +++-
  1 file changed, 39 insertions(+), 23 deletions(-)

diff --git a/fs/btrfs/xattr.c b/fs/btrfs/xattr.c
index f2a20d5..caf643d 100644
--- a/fs/btrfs/xattr.c
+++ b/fs/btrfs/xattr.c
@@ -260,16 +260,12 @@ out:

  ssize_t btrfs_listxattr(struct dentry *dentry, char *buffer, size_t size)
  {
-   struct btrfs_key key, found_key;
+   struct btrfs_key key;
struct inode *inode = d_inode(dentry);
struct btrfs_root *root = BTRFS_I(inode)->root;
struct btrfs_path *path;
-   struct extent_buffer *leaf;
-   struct btrfs_dir_item *di;
-   int ret = 0, slot;
+   int ret = 0;
size_t total_size = 0, size_left = size;
-   unsigned long name_ptr;
-   size_t name_len;

/*
 * ok we want all objects associated with this id.
@@ -291,6 +287,13 @@ ssize_t btrfs_listxattr(struct dentry *dentry, char 
*buffer, size_t size)
goto err;

while (1) {
+   struct extent_buffer *leaf;
+   int slot;
+   struct btrfs_dir_item *di;
+   struct btrfs_key found_key;
+   u32 item_size;
+   u32 cur;
+
leaf = path->nodes[0];
slot = path->slots[0];

@@ -319,28 +322,41 @@ ssize_t btrfs_listxattr(struct dentry *dentry, char 
*buffer, size_t size)
goto next;

di = btrfs_item_ptr(leaf, slot, 

Re: [PATCH] btrfs: Avoid BUG_ON()s because of ENOMEM caused by kmalloc() failure

[Satoru Takeuchi]

On 2016/02/18 0:11, David Sterba wrote:

On Wed, Feb 17, 2016 at 02:54:23PM +0900, Satoru Takeuchi wrote:

On 2016/02/16 2:53, David Sterba wrote:

On Mon, Feb 15, 2016 at 02:38:09PM +0900, Satoru Takeuchi wrote:

There are some BUG_ON()'s after kmalloc() as follows.

=
foo = kmalloc();
BUG_ON(!foo);   /* -ENOMEM case */
=

A Docker + memory cgroup user hit these BUG_ON()s.

https://bugzilla.kernel.org/show_bug.cgi?id=112101

Since it's very hard to handle these ENOMEMs properly,
preventing these kmalloc() failures to avoid these
BUG_ON()s for now, are a bit better than the current
implementation anyway.


Beware that the NOFAIL semantics is can cause deadlocks if it's on the
critical writeback path or and can be reentered from itself through the
reclaim. Unless you're sure that this is not the case, please do not add
them just because it would seemingly fix the allocation failures.


About the all cases I changed, kmalloc()s can block
since gfp_flags_allow_blocking() are true. Then no locks
are acquired here and deadlocks don't happen.

Am I missing something?


Waiting as in GFP_WAIT is not the same as GFP_NOFAIL that can wait
indefinetelly. The locking of NOFAIL is implied. The kmalloc callsite
will block until there's memory available. If another thread of btrfs
waits for this code to progress, it will block as well.


In the docker example, the memory is limited by cgroups so the NOFAIL
mode can exhaust all reserves and just loop endlessly waiting for the
OOM killer to get some memory or just waiting without any chance to
progress.


I consider triggering OOM killer and killing processes
in a cgroup are better than killing whole system.


The action of OOM killer is not a problem. The cgroup memory limit can
be low or all the memory is unreclaimable. At this point btrfs code will
block.



About the possibility of endless loop, there are many
such problems in the whole kernel. Of course it can be
said to Btrfs.


Many? Examples? In this context we're talking about endless loops caused
by non-failing allocations.


==
$ grep -rnH __GFP_NOFAIL fs/btrfs/
fs/btrfs/extent-tree.c:5970: GFP_NOFS | __GFP_NOFAIL);
fs/btrfs/extent-tree.c:6043: bytenr + num_bytes - 1, GFP_NOFS | __GFP_NOFAIL);
fs/btrfs/extent_io.c:4643: eb = kmem_cache_zalloc(extent_buffer_cache, 
GFP_NOFS|__GFP_NOFAIL);
fs/btrfs/extent_io.c:4909: p = find_or_create_page(mapping, index, 
GFP_NOFS|__GFP_NOFAIL);
==


I'm aware of the existing NOFAIL allocations. There are two at
extent_buffer allocation time, added recently and provoked by the
intended changes to memory allocator that would fail the GFP_NOFS
allocations.

The other two are from year 2010, set_extent_bit, IMHO added so the
error handling does not get complicated and expressing "we don't want to
fail here". But there are many other calls to set_extent_bit that could
fail. This is inconsistent and should be unified. In a way that we're
sure that we're not introducing potential hangs.


I understand fixing these problems cooperate with
memory cgroup guys is the best in the long run.


It's not about cgroups, btrfs needs to ideally handle all memory
allocation failures in a way that uses some sort of fallbacks but still
can lead to a transaction abort if there's simply no memory left.


However, I consider bypassing this problem for now
is better than the current implementation.


It's more like replacing one problem with another. With every new
NOFAIL, one has to think about the runtime interactions with the others.
I'd rather see this fixed with a particular call path in mind or class,
eg. the extent_map bit settings, than throwing NOFAIL at places that
somebody accidentally hit.

As a temporary fix we can add __GFP_HIGH to the interesting sites so we
can get access to the emergency reserves, and this is on my list of
things to do after the NOFS -> KERNEL updates are done.


OK, got it. I'll reconsider how to fix there problem.

Thanks,
Satoru


--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] btrfs: Avoid BUG_ON()s because of ENOMEM caused by kmalloc() failure

[Satoru Takeuchi]

On 2016/02/16 2:53, David Sterba wrote:

On Mon, Feb 15, 2016 at 02:38:09PM +0900, Satoru Takeuchi wrote:

There are some BUG_ON()'s after kmalloc() as follows.

=
foo = kmalloc();
BUG_ON(!foo);   /* -ENOMEM case */
=

A Docker + memory cgroup user hit these BUG_ON()s.

https://bugzilla.kernel.org/show_bug.cgi?id=112101

Since it's very hard to handle these ENOMEMs properly,
preventing these kmalloc() failures to avoid these
BUG_ON()s for now, are a bit better than the current
implementation anyway.


Beware that the NOFAIL semantics is can cause deadlocks if it's on the
critical writeback path or and can be reentered from itself through the
reclaim. Unless you're sure that this is not the case, please do not add
them just because it would seemingly fix the allocation failures.


About the all cases I changed, kmalloc()s can block
since gfp_flags_allow_blocking() are true. Then no locks
are acquired here and deadlocks don't happen.

Am I missing something?



In the docker example, the memory is limited by cgroups so the NOFAIL
mode can exhaust all reserves and just loop endlessly waiting for the
OOM killer to get some memory or just waiting without any chance to
progress.


I consider triggering OOM killer and killing processes
in a cgroup are better than killing whole system.

About the possibility of endless loop, there are many
such problems in the whole kernel. Of course it can be
said to Btrfs.

==
$ grep -rnH __GFP_NOFAIL fs/btrfs/
fs/btrfs/extent-tree.c:5970: GFP_NOFS | __GFP_NOFAIL);
fs/btrfs/extent-tree.c:6043: bytenr + num_bytes - 1, GFP_NOFS | __GFP_NOFAIL);
fs/btrfs/extent_io.c:4643: eb = kmem_cache_zalloc(extent_buffer_cache, 
GFP_NOFS|__GFP_NOFAIL);
fs/btrfs/extent_io.c:4909: p = find_or_create_page(mapping, index, 
GFP_NOFS|__GFP_NOFAIL);
==

I understand fixing these problems cooperate with
memory cgroup guys is the best in the long run.
However, I consider bypassing this problem for now
is better than the current implementation.

Thanks,
Satoru


--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] btrfs: Avoid BUG_ON()s because of ENOMEM caused by kmalloc() failure

[Satoru Takeuchi]

There are some BUG_ON()'s after kmalloc() as follows.

=
foo = kmalloc();
BUG_ON(!foo);   /* -ENOMEM case */
=

A Docker + memory cgroup user hit these BUG_ON()s.

https://bugzilla.kernel.org/show_bug.cgi?id=112101

Since it's very hard to handle these ENOMEMs properly,
preventing these kmalloc() failures to avoid these
BUG_ON()s for now, are a bit better than the current
implementation anyway.

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
 fs/btrfs/extent_io.c  | 6 ++
 fs/btrfs/inode.c  | 6 ++
 fs/btrfs/relocation.c | 3 +--
 3 files changed, 5 insertions(+), 10 deletions(-)

diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
index 2e7c97a..5f92450 100644
--- a/fs/btrfs/extent_io.c
+++ b/fs/btrfs/extent_io.c
@@ -874,10 +874,8 @@ __set_extent_bit(struct extent_io_tree *tree, u64 start, 
u64 end,

bits |= EXTENT_FIRST_DELALLOC;
 again:
-   if (!prealloc && gfpflags_allow_blocking(mask)) {
-   prealloc = alloc_extent_state(mask);
-   BUG_ON(!prealloc);
-   }
+   if (!prealloc && gfpflags_allow_blocking(mask))
+   prealloc = alloc_extent_state(mask|__GFP_NOFAIL);

spin_lock(>lock);
if (cached_state && *cached_state) {
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 85afe66..d20e5c5 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -357,8 +357,7 @@ static noinline int add_async_extent(struct async_cow *cow,
 {
struct async_extent *async_extent;

-   async_extent = kmalloc(sizeof(*async_extent), GFP_NOFS);
-   BUG_ON(!async_extent); /* -ENOMEM */
+   async_extent = kmalloc(sizeof(*async_extent), GFP_NOFS|__GFP_NOFAIL);
async_extent->start = start;
async_extent->ram_size = ram_size;
async_extent->compressed_size = compressed_size;
@@ -1143,8 +1142,7 @@ static int cow_file_range_async(struct inode *inode, 
struct page *locked_page,
clear_extent_bit(_I(inode)->io_tree, start, end, EXTENT_LOCKED,
 1, 0, NULL, GFP_NOFS);
while (start < end) {
-   async_cow = kmalloc(sizeof(*async_cow), GFP_NOFS);
-   BUG_ON(!async_cow); /* -ENOMEM */
+   async_cow = kmalloc(sizeof(*async_cow), GFP_NOFS|__GFP_NOFAIL);
async_cow->inode = igrab(inode);
async_cow->root = root;
async_cow->locked_page = locked_page;
diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
index ef6d8fc..6b9f718 100644
--- a/fs/btrfs/relocation.c
+++ b/fs/btrfs/relocation.c
@@ -1373,8 +1373,7 @@ static struct btrfs_root *create_reloc_root(struct 
btrfs_trans_handle *trans,
u64 last_snap = 0;
int ret;

-   root_item = kmalloc(sizeof(*root_item), GFP_NOFS);
-   BUG_ON(!root_item);
+   root_item = kmalloc(sizeof(*root_item), GFP_NOFS|__GFP_NOFAIL);

root_key.objectid = BTRFS_TREE_RELOC_OBJECTID;
root_key.type = BTRFS_ROOT_ITEM_KEY;
-- 
2.5.0
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] btrfs-progs: write down the meaning of BTRFS_ARG_BLKDEV

[Satoru Takeuchi]

Although BTRFS_ARG_BLKDEV can be returned from check_arg_type(),
it's not explained the meaning.

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
 utils.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/utils.c b/utils.c
index 3df8b42..eabc36d 100644
--- a/utils.c
+++ b/utils.c
@@ -1007,6 +1007,7 @@ static int is_reg_file(const char *path)
  * return BTRFS_ARG_UUID:  given input is uuid
  * return BTRFS_ARG_MNTPOINT:  given input is path
  * return BTRFS_ARG_REG:   given input is regular file
+ * return BTRFS_ARG_BLKDEV:given input is block device
  */
 int check_arg_type(const char *input)
 {
-- 
2.5.0


--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] btrfs-progs: Fix self-reference of man btrfs-subvolume

[Satoru Takeuchi]

btrfs-subvolume(8) is mentioned at "SEE ALSO" section of itself.

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
 Documentation/btrfs-subvolume.asciidoc | 1 -
 1 file changed, 1 deletion(-)

diff --git a/Documentation/btrfs-subvolume.asciidoc 
b/Documentation/btrfs-subvolume.asciidoc
index c187fd8..96cfe4a 100644
--- a/Documentation/btrfs-subvolume.asciidoc
+++ b/Documentation/btrfs-subvolume.asciidoc
@@ -178,6 +178,5 @@ further details.
 SEE ALSO
 
 `mkfs.btrfs`(8),
-`btrfs-subvolume`(8),
 `btrfs-quota`(8),
 `btrfs-qgroup`(8),
-- 
2.7.0.rc3


--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] btrfs-progs: describe btrfs-send requires read-only subvolume

[Satoru Takeuchi]

Both man btrfs-send(8) and usage message don't describe
btrfs-send needs read-only snapshot as its argument.

Signed-off-by: Satoru Takeuchi <takeuchi_sat...@jp.fujitsu.com>
---
 Documentation/btrfs-send.asciidoc | 1 +
 cmds-send.c   | 1 +
 2 files changed, 2 insertions(+)

diff --git a/Documentation/btrfs-send.asciidoc 
b/Documentation/btrfs-send.asciidoc
index 1dba8a3..e05342f 100644
--- a/Documentation/btrfs-send.asciidoc
+++ b/Documentation/btrfs-send.asciidoc
@@ -12,6 +12,7 @@ SYNOPSIS
 DESCRIPTION
 ---
 Sends the subvolume(s) specified by  to stdout.
+ should be read-only here.
 
 By default, this will send the whole subvolume. To do an incremental
 send, use '-p '.
diff --git a/cmds-send.c b/cmds-send.c
index 478ace1..3e34d75 100644
--- a/cmds-send.c
+++ b/cmds-send.c
@@ -711,6 +711,7 @@ const char * const cmd_send_usage[] = {
"btrfs send [-ve] [-p ] [-c ] [-f ] 
 [...]",
"Send the subvolume(s) to stdout.",
"Sends the subvolume(s) specified by  to stdout.",
+   " should be read-only here.",
"By default, this will send the whole subvolume. To do an incremental",
"send, use '-p '. If you want to allow btrfs to clone from",
"any additional local snapshots, use '-c ' (multiple times",
-- 
2.5.0

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Kernel bug in 3.19-rc4

[Satoru Takeuchi]

Hi,

On 2015/01/16 10:05, Tomasz Chmielewski wrote:

I just started some btrfs stress testing on latest linux kernel 3.19-rc4:
A few hours later, filesystem stopped working - the kernel bug report
can be found below.


Hi,

your kernel BUG at fs/btrfs/inode.c:3142! from 3.19-rc4 corresponds to 
http://marc.info/?l=linux-btrfsm=141903172106342w=2 - it was kernel BUG at 
/home/apw/COD/linux/fs/btrfs/inode.c:3123! in 3.18.1, and is exactly the same code in both cases:


 /* grab metadata reservation from transaction handle */
 if (reserve) {
 ret = btrfs_orphan_reserve_metadata(trans, inode);
 BUG_ON(ret); /* -ENOSPC in reservation; Logic error? JDM */
 }


Year, it's the same.

BTW, I've tried to reproduce this problem by using the way
you told me. However, it hasn't reproduced yet.

Thanks,
Satoru

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Kernel bug in 3.19-rc4

[Satoru Takeuchi]

Hi Marcel,

On 2015/01/16 4:46, Marcel Ritter wrote:

Hi,

I just started some btrfs stress testing on latest linux kernel 3.19-rc4:
A few hours later, filesystem stopped working - the kernel bug report
can be found below.

The test consists of one massive IO thread (writing 100GB files with dd),
and 2 tar instances extracting kernel sources and deleting them afterwards
(I can provide the simple bash script doing this, if needed).


Could you give me this script?

Thanks,
Satoru



System information (Ubuntu 14.04.1, latest kernel):

root@thunder # uname -a
Linux thunder 3.19.0-rc4-custom #1 SMP Mon Jan 12 16:13:44 CET 2015
x86_64 x86_64 x86_64 GNU/Linux

root@thunder # /root/btrfs-progs/btrfs --version
Btrfs v3.18-36-g0173148

Tests are done on 14 SCSI disks, using raid6 for data and metadata:

root@thunder # /root/btrfs-progs/btrfs fi show
Label: 'raid6'  uuid: cbe34d2b-5f75-46cf-9263-9813028ebc19
 Total devices 14 FS bytes used 674.62GiB
 devid1 size 279.39GiB used 59.24GiB path /dev/cciss/c1d0
 devid2 size 279.39GiB used 59.22GiB path /dev/cciss/c1d1
 devid3 size 279.39GiB used 59.22GiB path /dev/cciss/c1d10
 devid4 size 279.39GiB used 59.22GiB path /dev/cciss/c1d11
 devid5 size 279.39GiB used 59.22GiB path /dev/cciss/c1d12
 devid6 size 279.39GiB used 59.22GiB path /dev/cciss/c1d13
 devid7 size 279.39GiB used 59.22GiB path /dev/cciss/c1d2
 devid8 size 279.39GiB used 59.22GiB path /dev/cciss/c1d3
 devid9 size 279.39GiB used 59.22GiB path /dev/cciss/c1d4
 devid   10 size 279.39GiB used 59.22GiB path /dev/cciss/c1d5
 devid   11 size 279.39GiB used 59.22GiB path /dev/cciss/c1d6
 devid   12 size 279.39GiB used 59.22GiB path /dev/cciss/c1d7
 devid   13 size 279.39GiB used 59.22GiB path /dev/cciss/c1d8
 devid   14 size 279.39GiB used 59.22GiB path /dev/cciss/c1d9

Btrfs v3.18-36-g0173148

# This is provided for completeness only, and is taken
# somewhen *before* the kernel crash occured, so basic
# setup is the same, but allocated/free sizes won't match
root@thunder # /root/btrfs-progs/btrfs fi df /tmp/m
Data, single: total=8.00MiB, used=0.00B
Data, RAID6: total=727.45GiB, used=697.84GiB
System, single: total=4.00MiB, used=0.00B
System, RAID6: total=13.50MiB, used=64.00KiB
Metadata, single: total=8.00MiB, used=0.00B
Metadata, RAID6: total=3.43GiB, used=805.91MiB
GlobalReserve, single: total=272.00MiB, used=0.00B


Here's what happens after some hours of stress testing:

[85162.472989] [ cut here ]
[85162.473071] kernel BUG at fs/btrfs/inode.c:3142!
[85162.473139] invalid opcode:  [#1] SMP
[85162.473212] Modules linked in: btrfs(E) xor(E) raid6_pq(E)
radeon(E) ttm(E) drm_kms_helper(E) drm(E) hpwdt(E) amd64_edac_mod(E)
kvm(E) edac_core(E) shpchp(E) k8temp(E) serio_raw(E) hpilo(E)
edac_mce_amd(E) mac_hid(E) i2c_algo_bit(E) ipmi_si(E) nfsd(E)
auth_rpcgss(E) nfs_acl(E) nfs(E) lockd(E) grace(E) sunrpc(E) lp(E)
fscache(E) parport(E) hid_generic(E) usbhid(E) hid(E) hpsa(E)
psmouse(E) bnx2(E) cciss(E) pata_acpi(E) pata_amd(E)
[85162.473911] CPU: 4 PID: 3039 Comm: btrfs-cleaner Tainted: G
E  3.19.0-rc4-custom #1
[85162.474028] Hardware name: HP ProLiant DL585 G2   , BIOS A07 05/02/2011
[85162.474122] task: 88085b054aa0 ti: 88205ad4c000 task.ti:
88205ad4c000
[85162.474230] RIP: 0010:[a06a8182]  [a06a8182]
btrfs_orphan_add+0x1d2/0x1e0 [btrfs]
[85162.474422] RSP: 0018:88205ad4fc48  EFLAGS: 00010286
[85162.474497] RAX: ffe4 RBX: 8810a35d42f8 RCX: 88185b896000
[85162.474595] RDX: 6a54 RSI: 0004 RDI: 88185b896138
[85162.474694] RBP: 88205ad4fc88 R08: 0001e670 R09: 88016194b240
[85162.474793] R10: a06bd797 R11: ea0004f71800 R12: 88185baa2000
[85162.474892] R13: 88085f6d7630 R14: 88185baa2458 R15: 0001
[85162.474992] FS:  7fb3f27fb740() GS:88085fd0()
knlGS:
[85162.475105] CS:  0010 DS:  ES:  CR0: 8005003b
[85162.475184] CR2: 7f896c02c220 CR3: 00085b328000 CR4: 07e0
[85162.475286] Stack:
[85162.475318]  88205ad4fc88 a06e6a14 88185b896b04
88105b03e800
[85162.475442]  88016194b240 8810a35d42f8 881e8ffe9a00
88133dc48ea0
[85162.475561]  88205ad4fd18 a0691a57 88016194b244
88016194b240
[85162.475680] Call Trace:
[85162.475738]  [a06e6a14] ?
lookup_free_space_inode+0x44/0x100 [btrfs]
[85162.475849]  [a0691a57]
btrfs_remove_block_group+0x137/0x740 [btrfs]
[85162.475964]  [a06ca8d2] btrfs_remove_chunk+0x672/0x780 [btrfs]
[85162.476065]  [a06922bf] btrfs_delete_unused_bgs+0x25f/0x280 [btrfs]
[85162.476172]  [a0699e0c] cleaner_kthread+0x12c/0x190 [btrfs]
[85162.476269]  [a0699ce0] ? check_leaf+0x350/0x350 [btrfs]
[85162.476355]  [8108f8d2] 

Re: [PATCH] btrfs: get the accurate value of used_bytes in btrfs_get_block_group_info().

[Satoru Takeuchi]

Hi Dongsheng,

On 2015/01/05 20:19, Dongsheng Yang wrote:


Ping.

IOCTL of BTRFS_IOC_SPACE_INFO currently does not report
the data used but not synced to user.  Then btrfs fi df will
give user a wrong numbers before sync. This patch solve
this problem.

On 10/27/2014 08:38 PM, Dongsheng Yang wrote:

Reproducer:
# mkfs.btrfs -f -b 20G /dev/sdb
# mount /dev/sdb /mnt/test
# fallocate  -l 17G /mnt/test/largefile
# btrfs fi df /mnt/test
Data, single: total=17.49GiB, used=6.00GiB - only 6G, but actually it 
should be 17G.
System, DUP: total=8.00MiB, used=16.00KiB
System, single: total=4.00MiB, used=0.00B
Metadata, DUP: total=1.00GiB, used=112.00KiB
Metadata, single: total=8.00MiB, used=0.00B
GlobalReserve, single: total=16.00MiB, used=0.00B


I tried to reproduce your problem with 3.19-rc1.
However, this problem doesn't happen. Could you
also try to reproduce with the upstream kernel?

* Detail

test script (named yang-test.sh here):
===
#!/bin/bash -x

PART1=/dev/vdb
MNT_PNT=./mnt
mkfs.btrfs -f -b 20G ${PART1}
mount ${PART1} ${MNT_PNT}
fallocate -l 17G ${MNT_PNT}/largefile
btrfs fi df ${MNT_PNT}
sync
btrfs fi df ${MNT_PNT}
umount ${MNT_PNT}
===

Result:
===
# ./yang-test.sh
+ PART1=/dev/vdb
+ MNT_PNT=./mnt
+ mkfs.btrfs -f -b 20G /dev/vdb
Btrfs v3.17
See http://btrfs.wiki.kernel.org for more information.

Turning ON incompat feature 'extref': increased hardlink limit per file to 65536
fs created label (null) on /dev/vdb
nodesize 16384 leafsize 16384 sectorsize 4096 size 20.00GiB
+ mount /dev/vdb ./mnt
+ fallocate -l 17G ./mnt/largefile
+ btrfs fi df ./mnt
Data, single: total=17.01GiB, used=17.00GiB   # Used 17GiB properly
System, DUP: total=8.00MiB, used=16.00KiB
System, single: total=4.00MiB, used=0.00B
Metadata, DUP: total=1.00GiB, used=112.00KiB
Metadata, single: total=8.00MiB, used=0.00B
GlobalReserve, single: total=16.00MiB, used=0.00B
+ sync
+ btrfs fi df ./mnt
Data, single: total=17.01GiB, used=17.00GiB# (of course) used 17GiB too
System, DUP: total=8.00MiB, used=16.00KiB
System, single: total=4.00MiB, used=0.00B
Metadata, DUP: total=1.00GiB, used=112.00KiB
Metadata, single: total=8.00MiB, used=0.00B
GlobalReserve, single: total=16.00MiB, used=0.00B
+ umount ./mnt
===

Although I ran this test five times, the results are the same.

Thanks,
Satoru


# sync
# btrfs fi df /mnt/test
Data, single: total=17.49GiB, used=17.00GiB - After sync, it is expected.
System, DUP: total=8.00MiB, used=16.00KiB
System, single: total=4.00MiB, used=0.00B
Metadata, DUP: total=1.00GiB, used=112.00KiB
Metadata, single: total=8.00MiB, used=0.00B
GlobalReserve, single: total=16.00MiB, used=0.00B

The value of 6.00GiB is actually calculated in btrfs_get_block_group_info()
by adding the @block_group-item-used for each group together. In this way,
it did not consider the bytes in cache.

This patch adds the value of @pinned, @reserved and @bytes_super in
struct btrfs_block_group_cache to make sure we can get the accurate @used_bytes.

Reported-by: Qu Wenruo quwen...@cn.fujitsu.com
Signed-off-by: Dongsheng Yang yangds.f...@cn.fujitsu.com
---
  fs/btrfs/ioctl.c | 4 
  1 file changed, 4 insertions(+)

diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
index 33c80f5..bc2aaeb 100644
--- a/fs/btrfs/ioctl.c
+++ b/fs/btrfs/ioctl.c
@@ -3892,6 +3892,10 @@ void btrfs_get_block_group_info(struct list_head 
*groups_list,
  space-total_bytes += block_group-key.offset;
  space-used_bytes +=
  btrfs_block_group_used(block_group-item);
+/* Add bytes-info in cache */
+space-used_bytes += block_group-pinned;
+space-used_bytes += block_group-reserved;
+space-used_bytes += block_group-bytes_super;
  }
  }


--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] btrfs-progs: Fix a copy-n-paste bug in btrfs_read_fs_root().

[Satoru Takeuchi]

On 2015/01/07 18:23, Qu Wenruo wrote:
 Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com

Reviewed-by: Satoru Takeuchi takeuchi_sat...@jp.fujitsu.com

 ---
   disk-io.c | 2 +-
   1 file changed, 1 insertion(+), 1 deletion(-)
 
 diff --git a/disk-io.c b/disk-io.c
 index 2bf8586..b853f66 100644
 --- a/disk-io.c
 +++ b/disk-io.c
 @@ -693,7 +693,7 @@ struct btrfs_root *btrfs_read_fs_root(struct 
 btrfs_fs_info *fs_info,
   if (location-objectid == BTRFS_CSUM_TREE_OBJECTID)
   return fs_info-csum_root;
   if (location-objectid == BTRFS_QUOTA_TREE_OBJECTID)
 - return fs_info-csum_root;
 + return fs_info-quota_root;
   
   BUG_ON(location-objectid == BTRFS_TREE_RELOC_OBJECTID ||
  location-offset != (u64)-1);
 

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 0/3] Btrfs: Enhancment for qgroup.

[Satoru Takeuchi]

Hi Yang,

On 2015/01/05 15:16, Dongsheng Yang wrote:
 Hi Josef and others,
 
 This patch set is about enhancing qgroup.
 
 [1/3]: fix a bug about qgroup leak when we exceed quota limit,
   It is reviewd by Josef.
 [2/3]: introduce a new accounter in qgroup to close a window where
   user will exceed the limit by qgroup. It looks good to Josef.
 [3/3]: a new patch to fix a bug reported by Satoru.

I tested your the patchset v3. Although it's far better
than the patchset v2, there is still one problem in this patchset.
When I wrote 1.5GiB to a subvolume with 1.0 GiB limit,
1.0GiB - 139 block (in this case, 1KiB/block) was written.

I consider user should be able to write just 1.0GiB in this case.

* Test result

===
+ mkfs.btrfs -f /dev/vdb
Btrfs v3.17
See http://btrfs.wiki.kernel.org for more information.

Turning ON incompat feature 'extref': increased hardlink limit per file to 65536
fs created label (null) on /dev/vdb
nodesize 16384 leafsize 16384 sectorsize 4096 size 30.00GiB
+ mount /dev/vdb /root/btrfs-auto-test/
+ ret=0
+ btrfs quota enable /root/btrfs-auto-test/
+ btrfs subvolume create /root/btrfs-auto-test//sub
Create subvolume '/root/btrfs-auto-test/sub'
+ btrfs qgroup limit 1G /root/btrfs-auto-test//sub
+ dd if=/dev/zero of=/root/btrfs-auto-test//sub/file bs=1024 count=150
dd: error writing '/root/btrfs-auto-test//sub/file': Disk quota exceeded
1048438+0 records in# Tried to write 1GiB - 138 KiB
1048437+0 records out   # Succeeded to write 1GiB - 139 KiB
1073599488 bytes (1.1 GB) copied, 19.0247 s, 56.4 MB/s
===

* note

I tried to run the reproducer five times and the result is
a bit different for each time.

=
#   Written
-
1   1GiB - 139 KiB
2   1GiB - 139 KiB
3   1GiB - 145 KiB
4   1GiB - 135 KiB
5   1GiB - 135 KiB
==

So I consider it's a problem comes from timing.

If I changed the block size from 1KiB to 1 MiB,
the difference in bytes got larger.


#   Written

1   1GiB - 1 MiB
2   1GiB - 1 MiB
3   1GiB - 1 MiB
4   1GiB - 1 MiB
5   1GiB - 1 MiB


Thanks,
Satoru

 
 BTW, I have some other plan about qgroup in my TODO list:
 
 Kernel:
 a). adjust the accounters in parent qgroup when we move
 the child qgroup.
   Currently, when we move a qgroup, the parent qgroup
 will not updated at the same time. This will cause some wrong
 numbers in qgroup.
 
 b). add a ioctl to show the qgroup info.
   Command btrfs qgroup show is showing the qgroup info
 read from qgroup tree. But there is some information in memory
 which is not synced into device. Then it will show some outdate
 number.
 
 c). limit and account size in 3 modes, data, metadata and both.
   qgroup is accounting the size both of data and metadata
 togather, but to a user, the data size is the most useful to them.
 
 d). remove a subvolume related qgroup when subvolume is deleted and
 there is no other reference to it.
 
 user-tool:
 a). Add the unit of B/K/M/G to btrfs qgroup show.
 b). get the information via ioctl rather than reading it from
 btree. Will keep the old way as a fallback for compatiblity.
 
 Any comment and sugguestion is welcome. :)
 
 Yang
 
 Dongsheng Yang (3):
Btrfs: qgroup: free reserved in exceeding quota.
Btrfs: qgroup: Introduce a may_use to account
  space_info-bytes_may_use.
Btrfs: qgroup, Account data space in more proper timings.
 
   fs/btrfs/extent-tree.c | 41 +++---
   fs/btrfs/file.c|  9 ---
   fs/btrfs/inode.c   | 18 -
   fs/btrfs/qgroup.c  | 68 
 +++---
   fs/btrfs/qgroup.h  |  4 +++
   5 files changed, 117 insertions(+), 23 deletions(-)
 

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] btrfs: clear bio reference after submit_one_bio()

[Satoru Takeuchi]

Hi Naota,

On 2015/01/06 1:01, Naohiro Aota wrote:
 After submit_one_bio(), `bio' can go away. However submit_extent_page()
 leave `bio' referable if submit_one_bio() failed (e.g. -ENOMEM on OOM).
 It will cause invalid paging request when submit_extent_page() is called
 next time.
 
 I reproduced ENOMEM case with the following script (need
 CONFIG_FAIL_PAGE_ALLOC, and CONFIG_FAULT_INJECTION_DEBUG_FS).

I confirmed that this problem reproduce with 3.19-rc3 and
not reproduce with 3.19-rc3 with your patch.

Tested-by: Satoru Takeuchi takeuchi_sat...@jp.fujitsu.com

Thank you for reporting this problem with the reproducer
and fixing it too.

  NOTE:
  I used v3.19-rc3's tools/testing/fault-injection/failcmd.sh
  for the following ./failcmd.sh.

  ./failcmd.sh -p $percent -t $times -i $interval \
  --ignore-gfp-highmem=N --ignore-gfp-wait=N --min-order=0 
\
  -- \
  cat $directory/file  /dev/null

* 3.19-rc1 + your patch

===
# ./run
512+0 records in
512+0 records out
# 
===

* 3.19-rc3

===
# ./run
512+0 records in
512+0 records out
[  188.433726] run (776): drop_caches: 1
[  188.682372] FAULT_INJECTION: forcing a failure.
name fail_page_alloc, interval 100, probability 111000, space 0, times 3
[  188.689986] CPU: 0 PID: 954 Comm: cat Not tainted 3.19.0-rc3-ktest #1
[  188.693834] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 
Bochs 01/01/2011
[  188.698466]  0064 88007b343618 816e5563 
88007fc0fc78
[  188.702730]  81c655c0 88007b343638 813851b5 
0010
[  188.707043]  0002 88007b343768 81188126 
88007b3435a8
[  188.711283] Call Trace:
[  188.712620]  [816e5563] dump_stack+0x45/0x57
[  188.715330]  [813851b5] should_fail+0x135/0x140
[  188.718218]  [81188126] __alloc_pages_nodemask+0xd6/0xb30
[  188.721567]  [81339075] ? blk_rq_map_sg+0x35/0x170
[  188.724558]  [a0010705] ? virtio_queue_rq+0x145/0x2b0 [virtio_blk]
[  188.728191]  [a01bd00f] ? btrfs_submit_compressed_read+0xcf/0x4d0 
[btrfs]
[  188.732079]  [811d99fb] ? kmem_cache_alloc+0x1cb/0x230
[  188.735153]  [81181265] ? mempool_alloc_slab+0x15/0x20
[  188.738188]  [811cee1a] alloc_pages_current+0x9a/0x120
[  188.741153]  [a01bd0e9] btrfs_submit_compressed_read+0x1a9/0x4d0 
[btrfs]
[  188.744835]  [a0178621] btrfs_submit_bio_hook+0x1c1/0x1d0 [btrfs]
[  188.748225]  [a018b7b3] ? lookup_extent_mapping+0x13/0x20 [btrfs]
[  188.751547]  [a0179c08] ? btrfs_get_extent+0x98/0xad0 [btrfs]
[  188.754656]  [a01901d7] submit_one_bio+0x67/0xa0 [btrfs]
[  188.757554]  [a0193f27] submit_extent_page.isra.35+0xd7/0x1c0 
[btrfs]
[  188.760981]  [a019509d] __do_readpage+0x31d/0x7b0 [btrfs]
[  188.763920]  [a0195f10] ? btrfs_create_repair_bio+0x110/0x110 
[btrfs]
[  188.767382]  [a0179b70] ? btrfs_submit_direct+0x7b0/0x7b0 [btrfs]
[  188.770671]  [a018f88d] ? btrfs_lookup_ordered_range+0x13d/0x180 
[btrfs]
[  188.774366]  [a01958ca] 
__extent_readpages.constprop.42+0x2ba/0x2d0 [btrfs]
[  188.778031]  [a0179b70] ? btrfs_submit_direct+0x7b0/0x7b0 [btrfs]
[  188.781241]  [a01969b9] extent_readpages+0x169/0x1b0 [btrfs]
[  188.784322]  [a0179b70] ? btrfs_submit_direct+0x7b0/0x7b0 [btrfs]
[  188.789014]  [a0176b0f] btrfs_readpages+0x1f/0x30 [btrfs]
[  188.792028]  [8118bf5c] __do_page_cache_readahead+0x18c/0x1f0
[  188.795078]  [8118c09f] ondemand_readahead+0xdf/0x260
[  188.797702]  [a016c5df] ? btrfs_congested_fn+0x5f/0xa0 [btrfs]
[  188.800718]  [8118c291] page_cache_async_readahead+0x71/0xa0
[  188.803650]  [8118017f] generic_file_read_iter+0x40f/0x5e0
[  188.806480]  [811f43be] new_sync_read+0x7e/0xb0
[  188.808832]  [811f55d8] __vfs_read+0x18/0x50
[  188.811068]  [811f569a] vfs_read+0x8a/0x140
[  188.813298]  [811f5796] SyS_read+0x46/0xb0
[  188.815486]  [81125806] ? __audit_syscall_exit+0x1f6/0x2a0
[  188.818293]  [816eb8e9] system_call_fastpath+0x12/0x17
[  188.821005] BUG: unable to handle kernel paging request at 0001000c
[  188.821984] IP: [a01901b3] submit_one_bio+0x43/0xa0 [btrfs]
[  188.821984] PGD 7bad3067 PUD 0 
[  188.821984] Oops:  [#1] SMP 
[  188.821984] Modules linked in: ip6table_filter ip6_tables ebtable_nat 
ebtables bnep bluetooth rfkill btrfs xor raid6_pq microcode 8139too serio_raw 
virtio_balloon 8139cp mii nfsd auth_rpcgss nfs_acl lockd grace sunrpc 
virtio_blk ata_generic pata_acpi
[  188.821984] CPU: 1 PID: 954 Comm: cat Not tainted 3.19.0-rc3-ktest #1

Re: [PATCH] btrfs: qgroup: move WARN_ON() to the correct location.

[Satoru Takeuchi]

On 2015/01/06 21:54, Dongsheng Yang wrote:
 In function qgroup_excl_accounting(), we need to WARN when
 qg-excl is less than what we want to free, same to child
 and parents. But currently, for parent qgroup, the WARN_ON()
 is located after freeing qg-excl. It will WARN out even we
 free it normally.
 
 This patch move this WARN_ON() before freeing qg-excl.
 
 Signed-off-by: Dongsheng Yang yangds.f...@cn.fujitsu.com

Reviewed-by: Satoru Takeuchi takeuchi_sat...@jp.fujitsu.com

 ---
   fs/btrfs/qgroup.c | 3 +--
   1 file changed, 1 insertion(+), 2 deletions(-)
 
 diff --git a/fs/btrfs/qgroup.c b/fs/btrfs/qgroup.c
 index 48b60db..97159a8 100644
 --- a/fs/btrfs/qgroup.c
 +++ b/fs/btrfs/qgroup.c
 @@ -1431,9 +1431,8 @@ static int qgroup_excl_accounting(struct btrfs_fs_info 
 *fs_info,
   qgroup = u64_to_ptr(unode-aux);
   qgroup-rfer += sign * oper-num_bytes;
   qgroup-rfer_cmpr += sign * oper-num_bytes;
 + WARN_ON(sign  0  qgroup-excl  oper-num_bytes);
   qgroup-excl += sign * oper-num_bytes;
 - if (sign  0)
 - WARN_ON(qgroup-excl  oper-num_bytes);
   qgroup-excl_cmpr += sign * oper-num_bytes;
   qgroup_dirty(fs_info, qgroup);
   
 

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: kernel BUG at /home/apw/COD/linux/fs/btrfs/inode.c:3123!

[Satoru Takeuchi]

Hi Tomasz,

On 2014/12/20 8:28, Tomasz Chmielewski wrote:

Get this BUG with 3.18.1 (pasted at the bottom of the email).
Below all actions from creating the fs to BUG. I did not attempt to reproduce.


I tried to reproduce this problem and have some questions.



# mkfs.btrfs /dev/vdb
Btrfs v3.17.3
See http://btrfs.wiki.kernel.org for more information.

Turning ON incompat feature 'extref': increased hardlink limit per file to 65536
fs created label (null) on /dev/vdb
 nodesize 16384 leafsize 16384 sectorsize 4096 size 256.00GiB

# mount -o noatime /dev/vdb /mnt/test/
# cd /mnt/test
# btrfs sub cre subvolume
Create subvolume './subvolume'
# dd if=/dev/urandom of=bigfile.img bs=64k


Does it really this command? I consider it will fill up
whole /dev/vdb. And is it not subvolume/bigfile.img
but bigfile.img?


^C91758+0 records in
91757+0 records out
6013386752 bytes (6.0 GB) copied, 374.777 s, 16.0 MB/s
# btrfs sub list /mnt/test/
ID 257 gen 16 top level 5 path subvolume

# btrfs quota enable /mnt/test

# btrfs qgroup show /mnt/test
qgroupid rfer   excl
    
0/5  16384  16384
0/2576013403136 6013403136

# dd if=/dev/urandom of=bigfile2.img bs=64k
^C47721+0 records in
47720+0 records out
3127377920 bytes (3.1 GB) copied, 194.641 s, 16.1 MB/s


If bigfile.img is just under /mnt/test, I can't understand
why this command succeeded to write more 3 GiB.



# btrfs qgroup show /mnt/test
qgroupid rfer   excl
    
0/5  16384  16384
0/2578704049152 8704049152
root@srv2:/mnt/test/subvolume# sync
root@srv2:/mnt/test/subvolume# btrfs qgroup show /mnt/test
qgroupid rfer   excl
    
0/5  16384  16384
0/2579140781056 9140781056

# dd if=/dev/urandom of=bigfile3.img bs=64k
^C3617580+0 records in
3617579+0 records out
237081657344 bytes (237 GB) copied, 14796 s, 16.0 MB/s


It's too.

Thanks,
Satoru



# df -h
Filesystem  Size  Used Avail Use% Mounted on
(...)
/dev/vdb256G  230G   25G  91% /mnt/test


# btrfs qgroup show /mnt/test
qgroupid rfer excl
  
0/5  1638416384
0/257245960245248 245960245248

# ls -l
total 240451584
-rw-r--r-- 1 root root   3127377920 Dec 19 20:06 bigfile2.img
-rw-r--r-- 1 root root 237081657344 Dec 20 00:15 bigfile3.img
-rw-r--r-- 1 root root   6013386752 Dec 19 20:02 bigfile.img

# rm bigfile3.img

# sync

# dmesg
(...)
[   95.055420] BTRFS: device fsid 97f98279-21e7-4822-89be-3aed9dc05f2c devid 1 
transid 3 /dev/vdb
[  118.446509] BTRFS info (device vdb): disk space caching is enabled
[  118.446518] BTRFS: flagging fs with big metadata feature
[  118.452176] BTRFS: creating UUID tree
[  575.189412] BTRFS info (device vdb): qgroup scan completed
[15948.234826] [ cut here ]
[15948.234883] kernel BUG at /home/apw/COD/linux/fs/btrfs/inode.c:3123!
[15948.234906] invalid opcode:  [#1] SMP
[15948.234925] Modules linked in: nf_log_ipv6 ip6t_REJECT nf_reject_ipv6 
nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables nf_log_ipv4 
nf_log_common xt_LOG ipt_REJECT nf_reject_ipv4 xt_tcpudp nf_conntrack_ipv4 
nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables 
dm_crypt btrfs xor crct10dif_pclmul crc32_pclmul ghash_clmulni_intel 
aesni_intel ppdev aes_x86_64 lrw raid6_pq gf128mul glue_helper ablk_helper 
cryptd serio_raw mac_hid pvpanic 8250_fintek parport_pc i2c_piix4 lp parport 
psmouse qxl ttm floppy drm_kms_helper drm
[15948.235172] CPU: 0 PID: 3274 Comm: btrfs-cleaner Not tainted 
3.18.1-031801-generic #201412170637
[15948.235193] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 
rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014
[15948.235222] task: 880036708a00 ti: 88007b97c000 task.ti: 
88007b97c000
[15948.235240] RIP: 0010:[c0458ec9]  [c0458ec9] 
btrfs_orphan_add+0x1a9/0x1c0 [btrfs]
[15948.235305] RSP: 0018:88007b97fc98  EFLAGS: 00010286
[15948.235318] RAX: ffe4 RBX: 88007b80a800 RCX: 
[15948.235333] RDX: 219e RSI: 0004 RDI: 880079418138
[15948.235349] RBP: 88007b97fcd8 R08: 88007fc1cae0 R09: 88007ad272d0
[15948.235366] R10:  R11: 0010 R12: 88007a2d9500
[15948.235381] R13: 8800027d60e0 R14: 88007b80ac58 R15: 0001
[15948.235401] FS:  () GS:88007fc0() 
knlGS:
[15948.235418] CS:  0010 DS:  ES:  CR0: 80050033
[15948.235432] CR2: 7f0489ff CR3: 7a5e CR4: 001407f0
[15948.235464] Stack:
[15948.235473]  88007b97fcd8 c0497acf 88007b809800 
88003c207400
[15948.235498]  88007b809800 88007ad272d0 88007a2d9500 
0001
[15948.235521]  88007b97fd58 c04412e0 880079418000 
0004c0427fea
[15948.235551] Call Trace:
[15948.235601]  [c0497acf] ? 

Re: [PATCH] btrfs: suppress a build warning on building 32bit kernel

[Satoru Takeuchi]

Hi David,

On 2014/12/30 0:09, David Sterba wrote:

On Thu, Dec 25, 2014 at 06:21:41PM +0900, Satoru Takeuchi wrote:

From: Satoru Takeuchi takeuchi_sat...@jp.fujitsu.com
--- a/fs/btrfs/extent_io.c
+++ b/fs/btrfs/extent_io.c
@@ -2190,7 +2190,7 @@ void btrfs_free_io_failure_record(struct inode *inode, 
u64 start, u64 end)

next = next_state(state);

-   failrec = (struct io_failure_record *)state-private;
+   failrec = (struct io_failure_record *)(unsigned 
long)state-private;


We're always using the 'private' data to store a pointer to
'struct io_failure_record *', please change the defintion in
'struct extent_state' instead of the typecasting.


Current definition is as follow.

===
struct extent_state {
...
/* for use by the FS */
u64 private;
};
===

It it OK to changing u64 private to struct io_failure_record *failrec
and change {set,get}_state_private() to {set,get}_state_failrec()?
Or is it better to keep the name private as is and just change its type
to unsigned long or (void *)?

Thanks,
Satoru




free_extent_state(state);
kfree(failrec);


--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] btrfs-progs: Fix a extent buffer leak in count_csum_range().

[Satoru Takeuchi]

On 2015/01/05 16:56, Qu Wenruo wrote:
 The commit f495a2ac6611 (btrfs-progs: fsck: remove unfriendly BUG_ON()
 for searching tree failure) is causing tons of extent buffer leak if some
 csum mismatches in btrfsck.
 
 This is caused by a misplaced btrfs_release_path(), fix it.
 
 Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com

Reviewed-by: Satoru Takeuchi takeuchi_sat...@jp.fujitsu.com

 ---
   cmds-check.c | 2 +-
   1 file changed, 1 insertion(+), 1 deletion(-)
 
 diff --git a/cmds-check.c b/cmds-check.c
 index d2d218a..5b644cf 100644
 --- a/cmds-check.c
 +++ b/cmds-check.c
 @@ -1186,9 +1186,9 @@ static int count_csum_range(struct btrfs_root *root, 
 u64 start,
   path.slots[0]++;
   }
   out:
 + btrfs_release_path(path);
   if (ret  0)
   return ret;
 - btrfs_release_path(path);
   return 0;
   }
   
 

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Debian/Jessie 3.16.7-ckt2-1 kernel error

[Satoru Takeuchi]

Hi Petr,

On 2014/12/28 0:36, Petr Janecek wrote:

Hello Satoru and all,

   that Oct. report was the only time I've experienced the error, so I
don't have much to add. I can try to answer your questions:


Here are my questions.

1. Is your system btrfs scrub clean?


   yes,


2. Is this message shown every boot time?


   no, I have seen them only during one boot


3. Is this message shown only in boot?


   As in my Oct. email
http://permalink.gmane.org/gmane.comp.file-systems.btrfs/39721
I've seen a similar one after creating a subvolume on a new fs.
But it was during the same boot.


4. When this message is started to be shown?
5. Do you have any trouble, change your operation or configuration
just before the answer of Q4 ?


   a disk was added to the fs and balance has been run. The balance
crashed, as in https://bugzilla.kernel.org/show_bug.cgi?id=64961
(probably unrelated).  After reboot, I've seen the messages.


Additional questions.
Q5. Could you give me your kernel configuration?
 At least, could you tell me whether your kernel
 enabled CONFIG_PREEMPT or not?


   CONFIG_PREEMPT_VOLUNTARY=y


Q6. If the answer of Q1 is correct, please give me the
 file system image which can be captured by the following command.


   Sorry, the fs's are long gone. I continued to run similar workloads on
that test box, but these errors never appeared again.


Thank you for giving me information.

So, further investigation of this problem seems to be hard.
Please give us the above-mentioned information if this problem
happens again.

Thanks,
Satoru




Regards,

Petr
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html



--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html