Re: [PATCHSET v2] cgroup, writeback, btrfs: make sure btrfs issues metadata IOs from the root cgroup

[Jan Kara]

On Wed 29-11-17 13:38:26, Chris Mason wrote:
> On 11/29/2017 12:05 PM, Tejun Heo wrote:
> >On Wed, Nov 29, 2017 at 09:03:30AM -0800, Tejun Heo wrote:
> >>Hello,
> >>
> >>On Wed, Nov 29, 2017 at 05:56:08PM +0100, Jan Kara wrote:
> >>>What has happened with this patch set?
> >>
> >>No idea.  cc'ing Chris directly.  Chris, if the patchset looks good,
> >>can you please route them through the btrfs tree?
> >
> >lol looking at the patchset again, I'm not sure that's obviously the
> >right tree.  It can either be cgroup, block or btrfs.  If no one
> >objects, I'll just route them through cgroup.
> 
> We'll have to coordinate a bit during the next merge window but I don't have
> a problem with these going in through cgroup.  Dave does this sound good to
> you?

Also I was wondering about another thing: How does this play with Josef's
series for metadata writeback (Metadata specific accouting and dirty
writeout)? Would the per-inode selection of cgroup writeback still be
needed when Josef's series is applied since metadata writeback then won't
be associated with any particular mapping anymore?

Honza
-- 
Jan Kara 
SUSE Labs, CR
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCHSET v2] cgroup, writeback, btrfs: make sure btrfs issues metadata IOs from the root cgroup

[Chris Mason]

On 11/30/2017 12:23 PM, David Sterba wrote:

On Wed, Nov 29, 2017 at 01:38:26PM -0500, Chris Mason wrote:

On 11/29/2017 12:05 PM, Tejun Heo wrote:

On Wed, Nov 29, 2017 at 09:03:30AM -0800, Tejun Heo wrote:

Hello,

On Wed, Nov 29, 2017 at 05:56:08PM +0100, Jan Kara wrote:

What has happened with this patch set?


No idea.  cc'ing Chris directly.  Chris, if the patchset looks good,
can you please route them through the btrfs tree?


lol looking at the patchset again, I'm not sure that's obviously the
right tree.  It can either be cgroup, block or btrfs.  If no one
objects, I'll just route them through cgroup.


We'll have to coordinate a bit during the next merge window but I don't
have a problem with these going in through cgroup.  Dave does this sound
good to you?


There are only minor changes to btrfs code so cgroup tree would be
better.


I'd like to include my patch to do all crcs inline (instead of handing
off to helper threads) when io controls are in place.  By the merge
window we should have some good data on how much it's all helping.


Are there any problems in sight if the inline crc and cgroup chnanges go
separately? I assume there's a runtime dependency, not a code
dependency, so it could be sorted by the right merge order.



The feature is just more useful with the inline crcs.  Without them we 
end up with kworkers doing both high and low prio submissions and it all 
boils down to the speed of the lowest priority.


-chris

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCHSET v2] cgroup, writeback, btrfs: make sure btrfs issues metadata IOs from the root cgroup

[David Sterba]

On Wed, Nov 29, 2017 at 01:38:26PM -0500, Chris Mason wrote:
> On 11/29/2017 12:05 PM, Tejun Heo wrote:
> > On Wed, Nov 29, 2017 at 09:03:30AM -0800, Tejun Heo wrote:
> >> Hello,
> >>
> >> On Wed, Nov 29, 2017 at 05:56:08PM +0100, Jan Kara wrote:
> >>> What has happened with this patch set?
> >>
> >> No idea.  cc'ing Chris directly.  Chris, if the patchset looks good,
> >> can you please route them through the btrfs tree?
> > 
> > lol looking at the patchset again, I'm not sure that's obviously the
> > right tree.  It can either be cgroup, block or btrfs.  If no one
> > objects, I'll just route them through cgroup.
> 
> We'll have to coordinate a bit during the next merge window but I don't 
> have a problem with these going in through cgroup.  Dave does this sound 
> good to you?

There are only minor changes to btrfs code so cgroup tree would be
better.

> I'd like to include my patch to do all crcs inline (instead of handing 
> off to helper threads) when io controls are in place.  By the merge 
> window we should have some good data on how much it's all helping.

Are there any problems in sight if the inline crc and cgroup chnanges go
separately? I assume there's a runtime dependency, not a code
dependency, so it could be sorted by the right merge order.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCHSET v2] cgroup, writeback, btrfs: make sure btrfs issues metadata IOs from the root cgroup

[Chris Mason]

On 11/29/2017 12:05 PM, Tejun Heo wrote:

On Wed, Nov 29, 2017 at 09:03:30AM -0800, Tejun Heo wrote:

Hello,

On Wed, Nov 29, 2017 at 05:56:08PM +0100, Jan Kara wrote:

What has happened with this patch set?


No idea.  cc'ing Chris directly.  Chris, if the patchset looks good,
can you please route them through the btrfs tree?


lol looking at the patchset again, I'm not sure that's obviously the
right tree.  It can either be cgroup, block or btrfs.  If no one
objects, I'll just route them through cgroup.


We'll have to coordinate a bit during the next merge window but I don't 
have a problem with these going in through cgroup.  Dave does this sound 
good to you?


I'd like to include my patch to do all crcs inline (instead of handing 
off to helper threads) when io controls are in place.  By the merge 
window we should have some good data on how much it's all helping.


-chris

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCHSET v2] cgroup, writeback, btrfs: make sure btrfs issues metadata IOs from the root cgroup

[Tejun Heo]

On Wed, Nov 29, 2017 at 09:03:30AM -0800, Tejun Heo wrote:
> Hello,
> 
> On Wed, Nov 29, 2017 at 05:56:08PM +0100, Jan Kara wrote:
> > What has happened with this patch set?
> 
> No idea.  cc'ing Chris directly.  Chris, if the patchset looks good,
> can you please route them through the btrfs tree?

lol looking at the patchset again, I'm not sure that's obviously the
right tree.  It can either be cgroup, block or btrfs.  If no one
objects, I'll just route them through cgroup.

Thanks.

-- 
tejun
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCHSET v2] cgroup, writeback, btrfs: make sure btrfs issues metadata IOs from the root cgroup

[Tejun Heo]

Hello,

On Wed, Nov 29, 2017 at 05:56:08PM +0100, Jan Kara wrote:
> What has happened with this patch set?

No idea.  cc'ing Chris directly.  Chris, if the patchset looks good,
can you please route them through the btrfs tree?

Thanks.

-- 
tejun
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCHSET v2] cgroup, writeback, btrfs: make sure btrfs issues metadata IOs from the root cgroup

[Jan Kara]

Hi Tejun,

What has happened with this patch set?

Honza

On Tue 10-10-17 08:54:36, Tejun Heo wrote:
> Changes from the last version are
> 
> * blkcg_root_css exported to fix build breakage on modular btrfs.
> 
> * Use ext4_should_journal_data() test instead of
>   EXT4_MOUNT_JOURNAL_DATA.
> 
> * Separated out create_bh_bio() and used it to implement
>   submit_bh_blkcg_css() as suggested by Jan.
> 
> btrfs has different ways to issue metadata IOs and may end up issuing
> metadata or otherwise shared IOs from a non-root cgroup, which can
> lead to priority inversion and ineffective IO isolation.
> 
> This patchset makes sure that btrfs issues all metadata and shared IOs
> from the root cgroup by exempting btree_inodes from cgroup writeback
> and explicitly associating shared IOs with the root cgroup.
> 
> This patchset containst he following three patches
> 
>  [PATCH 1/5] blkcg: export blkcg_root_css
>  [PATCH 2/5] cgroup, writeback: replace SB_I_CGROUPWB with per-inode
>  [PATCH 3/5] buffer_head: separate out create_bh_bio() from
>  [PATCH 4/5] cgroup, buffer_head: implement submit_bh_blkcg_css()
>  [PATCH 5/5] btrfs: ensure that metadata and flush are issued from the
> 
> and is also available in the following git branch
> 
>  git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git 
> review-cgroup-btrfs-metadata-v2
> 
> diffstat follows.  Thanks.
> 
>  block/blk-cgroup.c  |1 +
>  fs/block_dev.c  |3 +--
>  fs/btrfs/check-integrity.c  |2 +-
>  fs/btrfs/disk-io.c  |4 
>  fs/btrfs/ioctl.c|6 +-
>  fs/btrfs/super.c|1 -
>  fs/buffer.c |   42 ++
>  fs/ext2/inode.c |3 ++-
>  fs/ext2/super.c |1 -
>  fs/ext4/inode.c |5 -
>  fs/ext4/super.c |2 --
>  include/linux/backing-dev.h |2 +-
>  include/linux/buffer_head.h |3 +++
>  include/linux/fs.h  |3 ++-
>  14 files changed, 58 insertions(+), 20 deletions(-)
> 
> --
> tejun
-- 
Jan Kara <j...@suse.com>
SUSE Labs, CR
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCHSET v2] cgroup, writeback, btrfs: make sure btrfs issues metadata IOs from the root cgroup

[Tejun Heo]

Hello,

Changes from the last version are

* blkcg_root_css exported to fix build breakage on modular btrfs.

* Use ext4_should_journal_data() test instead of
  EXT4_MOUNT_JOURNAL_DATA.

* Separated out create_bh_bio() and used it to implement
  submit_bh_blkcg_css() as suggested by Jan.

btrfs has different ways to issue metadata IOs and may end up issuing
metadata or otherwise shared IOs from a non-root cgroup, which can
lead to priority inversion and ineffective IO isolation.

This patchset makes sure that btrfs issues all metadata and shared IOs
from the root cgroup by exempting btree_inodes from cgroup writeback
and explicitly associating shared IOs with the root cgroup.

This patchset containst he following three patches

 [PATCH 1/5] blkcg: export blkcg_root_css
 [PATCH 2/5] cgroup, writeback: replace SB_I_CGROUPWB with per-inode
 [PATCH 3/5] buffer_head: separate out create_bh_bio() from
 [PATCH 4/5] cgroup, buffer_head: implement submit_bh_blkcg_css()
 [PATCH 5/5] btrfs: ensure that metadata and flush are issued from the

and is also available in the following git branch

 git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git 
review-cgroup-btrfs-metadata-v2

diffstat follows.  Thanks.

 block/blk-cgroup.c  |1 +
 fs/block_dev.c  |3 +--
 fs/btrfs/check-integrity.c  |2 +-
 fs/btrfs/disk-io.c  |4 
 fs/btrfs/ioctl.c|6 +-
 fs/btrfs/super.c|1 -
 fs/buffer.c |   42 ++
 fs/ext2/inode.c |3 ++-
 fs/ext2/super.c |1 -
 fs/ext4/inode.c |5 -
 fs/ext4/super.c |2 --
 include/linux/backing-dev.h |2 +-
 include/linux/buffer_head.h |3 +++
 include/linux/fs.h  |3 ++-
 14 files changed, 58 insertions(+), 20 deletions(-)

--
tejun
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCHSET] cgroup, writeback, btrfs: make sure btrfs issues metadata IOs from the root cgroup

[Tejun Heo]

Hello,

btrfs has different ways to issue metadata IOs and may end up issuing
metadata or otherwise shared IOs from a non-root cgroup, which can
lead to priority inversion and ineffective IO isolation.

This patchset makes sure that btrfs issues all metadata and shared IOs
from the root cgroup by exempting btree_inodes from cgroup writeback
and explicitly associating shared IOs with the root cgroup.

This patchset containst he following three patches

 [PATCH 1/3] cgroup, writeback: replace SB_I_CGROUPWB with per-inode
 [PATCH 2/3] cgroup, writeback: implement submit_bh_blkcg_css()
 [PATCH 3/3] btrfs: ensure that metadata and flush are issued from the

and is also available in the following git branch

 git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git 
review-cgroup-btrfs-metadata

diffstat follows.  Thanks.

 fs/block_dev.c  |3 +--
 fs/btrfs/check-integrity.c  |2 +-
 fs/btrfs/disk-io.c  |4 
 fs/btrfs/ioctl.c|6 +-
 fs/btrfs/super.c|1 -
 fs/buffer.c |   12 
 fs/ext2/inode.c |3 ++-
 fs/ext2/super.c |1 -
 fs/ext4/inode.c |5 -
 fs/ext4/super.c |2 --
 fs/fs-writeback.c   |1 +
 include/linux/backing-dev.h |2 +-
 include/linux/buffer_head.h |   11 +++
 include/linux/fs.h  |3 ++-
 include/linux/writeback.h   |6 --
 15 files changed, 48 insertions(+), 14 deletions(-)

--
tejun
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Btrfs Issues

[Ruoxin Jiang]

Hi Satoru,

I'm so sorry for the late reply! I rewrote the test programs to invoke
syscalls in the conventional way and added a few comments. Please find
attached the revised report.

As to your questions on "mmap", you are totally right that the first
mmap in the original 1/min.cpp is irrelevant. The original program was
automatically generated from a fuzzer I used and I hope the revised
ones to be much more understandable.

Thanks,
Amy

On Sun, Sep 17, 2017 at 11:10 PM, Satoru Takeuchi
 wrote:
> At Wed, 13 Sep 2017 11:53:35 -0400,
> Ruoxin Jiang wrote:
>>
>> [1  ]
>> Hello,
>>
>> We are researchers from Columbia University, New York. As part of our
>> current research we have found some semantic discrepancies between
>> btrfs and other popular filesystems.
>>
>> We have attached two cases. The first one involves an invalid O_DIRECT
>> write() that fails back to buffered write instead of failing with
>> error EINVAL. In directory 2, we discovered that btrfs calculates
>> write_bytes in __btrfs_buffered_write differently from that in
>> generic_perform_writes in fs/mmap.c. This can cause inconsistent
>> behavior between btrfs and other filesystems when program invokes the
>> same writev/write() syscall.
>>
>> In each directory, you will find a Readme.md describing the issue and
>> pointing to the code that may cause the problem. For your convenience,
>> we also included test programs (min.cpp) and instructions in Readme to
>> help reproduce the issues.
>>
>> We would appreciate very much if you could confirm the two cases at
>> your conveniences.
>
> I took a look at your test programs, btrfs_issues/{1,2}/min.cpp. It looks
> very hard to read since you call syscalls in odd ways and all flags are
> hardcoded as literal hexadecimal numbers. Could rewrite these program
> to improve readability?
>
> In addition, I have two questions about btrfs_issues/1/min.cpp.
>
> 1. Why you set 'filename' as the 1st argument of mmap()?
> 2. What's the purpose of mmap() call? I guess mmap() is not related to issue 
> 1.
>
> Thanks,
> Satoru
>
>>
>> Thanks,
>> Amy
>> [2 btrfs_issues.tar.gz ]
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Ruoxin(Amy) Jiang
Columbia University, M.S. Computer Science
347-277-5455


btrfs_issues_revised.tar.gz
Description: GNU Zip compressed data

Re: Btrfs Issues

[Satoru Takeuchi]

At Wed, 13 Sep 2017 11:53:35 -0400,
Ruoxin Jiang wrote:
> 
> [1  ]
> Hello,
> 
> We are researchers from Columbia University, New York. As part of our
> current research we have found some semantic discrepancies between
> btrfs and other popular filesystems.
> 
> We have attached two cases. The first one involves an invalid O_DIRECT
> write() that fails back to buffered write instead of failing with
> error EINVAL. In directory 2, we discovered that btrfs calculates
> write_bytes in __btrfs_buffered_write differently from that in
> generic_perform_writes in fs/mmap.c. This can cause inconsistent
> behavior between btrfs and other filesystems when program invokes the
> same writev/write() syscall.
> 
> In each directory, you will find a Readme.md describing the issue and
> pointing to the code that may cause the problem. For your convenience,
> we also included test programs (min.cpp) and instructions in Readme to
> help reproduce the issues.
> 
> We would appreciate very much if you could confirm the two cases at
> your conveniences.

I took a look at your test programs, btrfs_issues/{1,2}/min.cpp. It looks
very hard to read since you call syscalls in odd ways and all flags are
hardcoded as literal hexadecimal numbers. Could rewrite these program
to improve readability?

In addition, I have two questions about btrfs_issues/1/min.cpp.

1. Why you set 'filename' as the 1st argument of mmap()?
2. What's the purpose of mmap() call? I guess mmap() is not related to issue 1.

Thanks,
Satoru

> 
> Thanks,
> Amy
> [2 btrfs_issues.tar.gz ]
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Btrfs Issues

[Ruoxin Jiang]

Hello,

We are researchers from Columbia University, New York. As part of our
current research we have found some semantic discrepancies between
btrfs and other popular filesystems.

We have attached two cases. The first one involves an invalid O_DIRECT
write() that fails back to buffered write instead of failing with
error EINVAL. In directory 2, we discovered that btrfs calculates
write_bytes in __btrfs_buffered_write differently from that in
generic_perform_writes in fs/mmap.c. This can cause inconsistent
behavior between btrfs and other filesystems when program invokes the
same writev/write() syscall.

In each directory, you will find a Readme.md describing the issue and
pointing to the code that may cause the problem. For your convenience,
we also included test programs (min.cpp) and instructions in Readme to
help reproduce the issues.

We would appreciate very much if you could confirm the two cases at
your conveniences.

Thanks,
Amy


btrfs_issues.tar.gz
Description: GNU Zip compressed data

Re: btrfs issues in 3.14

[Liu Bo]

On Thu, May 08, 2014 at 10:51:03AM -0300, Kenny MacDermid wrote:
 On Wed, May 7, 2014 at 11:48 PM, Liu Bo bo.li@oracle.com wrote:
 
  On Wed, May 07, 2014 at 09:35:06AM -0300, Kenny MacDermid wrote:
   On Tue, May 6, 2014 at 11:22 PM, Liu Bo bo.li@oracle.com wrote:
   
What does sysrq+w say when the hang happens?
  
   The whole system isn't hung, I may have explained that wrong. The
   system will hang if I try to shutdown, and the process will hang if I
   try to kill -9 it.
  
   It looks like the browser is in this state currently so I did an 'echo
   w /proc/sysrq-trigger' and have attached the full dmesg with the
   browser issues and the output.
 
  Those stacks show the blocked tasks are waiting for a page's writeback, but
  they don't show what blocks the endio process of that page.
 
  I'd recommand you to try the lastest 3.15.0-rc4 or btrfs-next, as many fixes
  are merged during this period.
 
 
 Thank you, I upgraded to the Arch package for 3.15.0-1-mainline (it's
 rc4) and I'll let you know if the errors reoccur.

FYI, this patch seems to address your problem.

Btrfs: fix hang on error (such as ENOSPC) when writing extent pages
https://patchwork.kernel.org/patch/4139971/

-liubo

 
 Should the filesystem be rebuilt again?
 
 A 'btrfs check' of it returned:
 
 checking extents
 checking free space cache
 checking fs roots
 checking csums
 checking root refs
 Checking filesystem on /dev/mapper/home
 UUID: 9a60a25f-eeb4-494c-b1af-ebd8e4f79b6b
 free space inode generation (0) did not match free space cache generation 
 (6409)
 free space inode generation (0) did not match free space cache generation 
 (6397)
 found 41686685877 bytes used err is 0
 total csum bytes: 74074632
 total tree bytes: 907673600
 total fs tree bytes: 807567360
 total extent tree bytes: 18251776
 btree space waste bytes: 116552179
 file data blocks allocated: 112191107072
  referenced 75535110144
 Btrfs v3.14.1
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: btrfs issues in 3.14

[Kenny MacDermid]

On Wed, May 7, 2014 at 11:48 PM, Liu Bo bo.li@oracle.com wrote:

 On Wed, May 07, 2014 at 09:35:06AM -0300, Kenny MacDermid wrote:
  On Tue, May 6, 2014 at 11:22 PM, Liu Bo bo.li@oracle.com wrote:
  
   What does sysrq+w say when the hang happens?
 
  The whole system isn't hung, I may have explained that wrong. The
  system will hang if I try to shutdown, and the process will hang if I
  try to kill -9 it.
 
  It looks like the browser is in this state currently so I did an 'echo
  w /proc/sysrq-trigger' and have attached the full dmesg with the
  browser issues and the output.

 Those stacks show the blocked tasks are waiting for a page's writeback, but
 they don't show what blocks the endio process of that page.

 I'd recommand you to try the lastest 3.15.0-rc4 or btrfs-next, as many fixes
 are merged during this period.


Thank you, I upgraded to the Arch package for 3.15.0-1-mainline (it's
rc4) and I'll let you know if the errors reoccur.

Should the filesystem be rebuilt again?

A 'btrfs check' of it returned:

checking extents
checking free space cache
checking fs roots
checking csums
checking root refs
Checking filesystem on /dev/mapper/home
UUID: 9a60a25f-eeb4-494c-b1af-ebd8e4f79b6b
free space inode generation (0) did not match free space cache generation (6409)
free space inode generation (0) did not match free space cache generation (6397)
found 41686685877 bytes used err is 0
total csum bytes: 74074632
total tree bytes: 907673600
total fs tree bytes: 807567360
total extent tree bytes: 18251776
btree space waste bytes: 116552179
file data blocks allocated: 112191107072
 referenced 75535110144
Btrfs v3.14.1
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: btrfs issues in 3.14

[Kenny MacDermid]

On Wed, May 7, 2014 at 9:35 AM, Kenny MacDermid
kenny.macder...@gmail.com wrote:
 On Tue, May 6, 2014 at 11:22 PM, Liu Bo bo.li@oracle.com wrote:

 What does sysrq+w say when the hang happens?

 The whole system isn't hung, I may have explained that wrong. The
 system will hang if I try to shutdown, and the process will hang if I
 try to kill -9 it.

 It looks like the browser is in this state currently so I did an 'echo
 w /proc/sysrq-trigger' and have attached the full dmesg with the
 browser issues and the output.

I had to hard reboot to clear that issue, and I decided to do another
'btrfs check' while /home was unmounted. It generated the following
output:

checking extents
checking free space cache
Wanted bytes 45056, found 32768 for off 63805808640
Wanted bytes 90016, found 32768 for off 63805808640
cache appears valid but isnt 62843256832
Checking filesystem on //dev/mapper/home
UUID: 9a60a25f-eeb4-494c-b1af-ebd8e4f79b6b
found 13672418478 bytes used err is -22
total csum bytes: 72089212
total tree bytes: 906100736
total fs tree bytes: 808370176
total extent tree bytes: 18153472
btree space waste bytes: 116247440
file data blocks allocated: 101046853632
 referenced 73680674816
Btrfs v3.14.1

This is on the new filesystem. I redid the dmcrypt and the lvm lv when
I recreated the filesystem as well, so it's less than a week old.
Before rebuilding the old was was telling me:

Checking filesystem on /dev/mapper/home
UUID: 4f5d7a10-d003-48a7-a901-bf22d534888f
free space inode generation (0) did not match free space cache
generation (115200)
found 29963117667 bytes used err is 1
total csum bytes: 63740440
total tree bytes: 745504768
total fs tree bytes: 624951296
total extent tree bytes: 36749312
btree space waste bytes: 119018687
file data blocks allocated: 181026942976
 referenced 73759866880
Btrfs v0.20-rc1-358-g194aa4a-dirty

and

checking extents
checking free space cache
checking fs roots
root 257 inode 29647 errors 200, dir isize wrong
root 257 inode 391917 errors 200, dir isize wrong
root 257 inode 497392 errors 410, odd dir item, nbytes wrong
Checking filesystem on /dev/mapper/home
UUID: 4f5d7a10-d003-48a7-a901-bf22d534888f
free space inode generation (0) did not match free space cache
generation (115200)
found 31310902624 bytes used err is 1
total csum bytes: 63579480
total tree bytes: 743342080
total fs tree bytes: 623198208
total extent tree bytes: 36601856
btree space waste bytes: 118906643
file data blocks allocated: 180831965184
 referenced 73631731712
Btrfs v3.14
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: btrfs issues in 3.14

[Liu Bo]

On Wed, May 07, 2014 at 09:35:06AM -0300, Kenny MacDermid wrote:
 On Tue, May 6, 2014 at 11:22 PM, Liu Bo bo.li@oracle.com wrote:
 
  What does sysrq+w say when the hang happens?
 
 The whole system isn't hung, I may have explained that wrong. The
 system will hang if I try to shutdown, and the process will hang if I
 try to kill -9 it.
 
 It looks like the browser is in this state currently so I did an 'echo
 w /proc/sysrq-trigger' and have attached the full dmesg with the
 browser issues and the output.

Those stacks show the blocked tasks are waiting for a page's writeback, but
they don't show what blocks the endio process of that page.

I'd recommand you to try the lastest 3.15.0-rc4 or btrfs-next, as many fixes
are merged during this period.

thanks,
-liubo
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

btrfs issues in 3.14

[Kenny MacDermid]

Hello,

I've been having a number of issues with processes hanging due to
btrfs using 3.14 kernels. This seems pretty new as it has been working
fine before. I also rebuilt the filesystem and am still receiving
hangs.

The filesystem is running on dmcrypt which is running on lvm2 which is
running on an SSD (SAMSUNG MZMTD256HAGM-000L1).

When the issue occurs the process is unable to be killed and the
system will not fully shutdown.

$ uname -a
Linux orange 3.14.2-1-ARCH #1 SMP PREEMPT Sun Apr 27 11:28:44 CEST
2014 x86_64 GNU/Linux

$ btrfs --version
Btrfs v3.14.1

$ btrfs fi show
Btrfs v3.14.1

$ btrfs fi df /home
Data, single: total=71.01GiB, used=68.72GiB
System, DUP: total=8.00MiB, used=16.00KiB
System, single: total=4.00MiB, used=0.00
Metadata, DUP: total=1.50GiB, used=863.33MiB
Metadata, single: total=8.00MiB, used=0.00

I opened bugs 75181 and 75191 and I'll include the relevant journalctl
entries. The kernel was upgraded from 3.14.1-1 to 3.14.2-1 during this
time, and the filesystem was rebuilt after the orphan issue.

I'm not on this list so please CC me on replies.

Thanks,

Kenny


journal.txt.gz
Description: GNU Zip compressed data

Re: btrfs issues in 3.14

[Liu Bo]

On Tue, May 06, 2014 at 08:49:04PM -0300, Kenny MacDermid wrote:
 Hello,
 
 I've been having a number of issues with processes hanging due to
 btrfs using 3.14 kernels. This seems pretty new as it has been working
 fine before. I also rebuilt the filesystem and am still receiving
 hangs.
 
 The filesystem is running on dmcrypt which is running on lvm2 which is
 running on an SSD (SAMSUNG MZMTD256HAGM-000L1).
 
 When the issue occurs the process is unable to be killed and the
 system will not fully shutdown.
 
 $ uname -a
 Linux orange 3.14.2-1-ARCH #1 SMP PREEMPT Sun Apr 27 11:28:44 CEST
 2014 x86_64 GNU/Linux
 
 $ btrfs --version
 Btrfs v3.14.1
 
 $ btrfs fi show
 Btrfs v3.14.1
 
 $ btrfs fi df /home
 Data, single: total=71.01GiB, used=68.72GiB
 System, DUP: total=8.00MiB, used=16.00KiB
 System, single: total=4.00MiB, used=0.00
 Metadata, DUP: total=1.50GiB, used=863.33MiB
 Metadata, single: total=8.00MiB, used=0.00
 
 I opened bugs 75181 and 75191 and I'll include the relevant journalctl
 entries. The kernel was upgraded from 3.14.1-1 to 3.14.2-1 during this
 time, and the filesystem was rebuilt after the orphan issue.
 
 I'm not on this list so please CC me on replies.

What does sysrq+w say when the hang happens?

-liubo
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: btrfs: issues with SUBVOL_SETFLAGS

[Li Zefan]

Dan Rosenberg wrote:
 Commit 0caa102da82799efaba88e234484786a9591c797 introduced the
 SUBVOL_SETFLAGS ioctl, which contains the following check:
 
   if (flags  ~BTRFS_SUBVOL_CREATE_ASYNC)

Oops, should be:

if (flags  BTRFS_SUBVOL_CREATE_ASYNC)

   return -EINVAL;
 
   if (flags  ~BTRFS_SUBVOL_RDONLY)
   return -EOPNOTSUPP;
 
 Is it intentional that 0 is the only acceptable flags value?  In
 addition, there should probably be an inode ownership check before
 allowing setting subvolume flags.
 

Thanks for pointing it out! Fix will be sent out soon.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html