QGroups Semantics

I have some questions about the *intended* qgroups semantics, and why we allow certain operations: 1) Why can you create a level 0 qgroup for a non-existent subvolume? It looks like it just gets reset when you create the subvol anyway? Should we prevent this? 2) Why do we allow deleting a level

Re: btrfs metadata reclaim behavior/performance characteristics

> From: Liu Bo > > Subject: [PATCH] Btrfs: skip commit transaction if we don't have enough > pinned bytes > > We commit transaction in order to reclaim space from pinned bytes because it > could process delayed refs, and in may_commit_transaction(), we check first > if

Re: btrfs list corruption and soft lockups while testing writeback error handling

On Thu, 2017-05-18 at 21:57 -0700, Liu Bo wrote: > On Fri, May 12, 2017 at 10:22:47AM -0400, Jeff Layton wrote: > > On Fri, 2017-05-12 at 08:12 -0400, Jeff Layton wrote: > > > On Thu, 2017-05-11 at 15:56 -0400, Chris Mason wrote: > > > > On 05/11/2017 03:52 PM, Jeff Layton wrote: > > > > > On Thu,

Re: [PATCH v2 0/2] btrfs: allow mechanism to override quota

On Fri, May 12, 2017 at 7:48 AM, David Sterba wrote: > On Thu, May 11, 2017 at 09:17:01PM +, Sargun Dhillon wrote: >> This patchset makes it so that on a per-filesystem basis one can disable >> quota enforcement for users with cap_sys_resource. This patchset can >> likely

Re: RAID 6 corrupted

On Thu, May 18, 2017 at 06:12:22AM +, Duncan wrote: > Roman Mamedov posted on Thu, 18 May 2017 10:17:19 +0500 as excerpted: > > > On Thu, 18 May 2017 04:09:38 +0200 ??ukasz Wróblewski wrote: > > > >> I will try when stable 4.12 comes out. > >> Unfortunately I do not have a

Re: dedicated error codes for the block layer

On Thu, May 18, 2017 at 04:55:08PM +0200, David Sterba wrote: > JFYI, the patches 13 and 15 are missing, not in linux-btrfs mailbox and > patchwork web. Does not look like a delay, maybe vger refused them > completely. They still haven't made it to linux-block and linux-btrfs, but they did make

Re: [PATCH 12/15] block: merge blk_types.h into bio.h

On Thu, May 18, 2017 at 02:25:52PM +, Bart Van Assche wrote: > On Thu, 2017-05-18 at 15:18 +0200, Christoph Hellwig wrote: > > We've cleaned up our headers sufficiently that we don't need this split > > anymore. > > Hello Christoph, > > Request-based drivers need the structure definitions

Re: RAID 6 corrupted

On Fri, 19 May 2017 11:55:27 +0300 Pasi Kärkkäinen wrote: > > > Try saving your data with "btrfs restore" first > > > > First post, he tried that. No luck. Tho that was with 4.4 userspace. > > It might be worth trying with the 4.11-rc or soon to be released 4.11 > >

Re: 4.11.0: kernel BUG at fs/btrfs/ctree.h:1779!

On Sat, May 20, 2017 at 12:37:47AM +, Hugo Mills wrote: > > Can I make another plea for just removing all those BUG/BUG_ON? > > They really have no place in production code, there is no excuse for a > > filesystem to bring down the entire and in the process not even tell you > > which of your

Re: 4.11.0: kernel BUG at fs/btrfs/ctree.h:1779!

On Sat, May 20, 2017 at 12:57:09AM +, Hugo Mills wrote: >I think from the POV of removing these BUG_ONs, it doesn't matter > which FS causes them. "All" you need to know is where the error > happened. From there, you can (in theory) work out what was wrong and > handle it more elagantly

Re: 4.11.0: kernel BUG at fs/btrfs/ctree.h:1779!

On Fri, May 19, 2017 at 06:25:22PM -0700, Marc MERLIN wrote: > On Sat, May 20, 2017 at 12:57:09AM +, Hugo Mills wrote: > >I think from the POV of removing these BUG_ONs, it doesn't matter > > which FS causes them. "All" you need to know is where the error > > happened. From there, you can

Re: RAID 6 corrupted

Pasi Kärkkäinen posted on Fri, 19 May 2017 11:55:27 +0300 as excerpted: > On Thu, May 18, 2017 at 06:12:22AM +, Duncan wrote: >> Roman Mamedov posted on Thu, 18 May 2017 10:17:19 +0500 as excerpted: >> >> > On Thu, 18 May 2017 04:09:38 +0200 ??ukasz Wróblewski >> > wrote: >> >

Re: 4.11.0: kernel BUG at fs/btrfs/ctree.h:1779!

On Fri, May 19, 2017 at 12:03:58PM -0700, Liu Bo wrote: > Hi Marc, > > On Thu, May 18, 2017 at 09:16:38PM -0700, Marc MERLIN wrote: > > Looks like all the unhelpful BUG() aren't gone yet :-/ > > This one is really not helpful, I don't even know which one of my > > filesystems caused the crash :(

Re: 4.11.0: kernel BUG at fs/btrfs/ctree.h:1779!

On Fri, May 19, 2017 at 05:11:34PM -0700, Marc MERLIN wrote: > On Fri, May 19, 2017 at 12:03:58PM -0700, Liu Bo wrote: > > Hi Marc, > > > > On Thu, May 18, 2017 at 09:16:38PM -0700, Marc MERLIN wrote: > > > Looks like all the unhelpful BUG() aren't gone yet :-/ > > > This one is really not

Re: 4.11.0: kernel BUG at fs/btrfs/ctree.h:1779!

On Fri, May 19, 2017 at 05:47:48PM -0700, Marc MERLIN wrote: > On Sat, May 20, 2017 at 12:37:47AM +, Hugo Mills wrote: > > > Can I make another plea for just removing all those BUG/BUG_ON? > > > They really have no place in production code, there is no excuse for a > > > filesystem to bring

[PATCH 2/3] Btrfs: lzo compression must free at least PAGE_SIZE

If data compression didn't free at least one PAGE_SIZE, it useless to store that compressed extent Signed-off-by: Timofey Titovets --- fs/btrfs/lzo.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/lzo.c b/fs/btrfs/lzo.c index

[PATCH 3/3] Btrfs: zlib compression must free at least PAGE_SIZE

If data compression didn't free at least one PAGE_SIZE, it useless to store that compressed extent Signed-off-by: Timofey Titovets --- fs/btrfs/zlib.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/zlib.c b/fs/btrfs/zlib.c index

[PATCH 1/3] Btrfs: lzo.c pr_debug() deflate->lzo

Signed-off-by: Timofey Titovets --- fs/btrfs/lzo.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/lzo.c b/fs/btrfs/lzo.c index f48c8c14..bd0b0938 100644 --- a/fs/btrfs/lzo.c +++ b/fs/btrfs/lzo.c @@ -141,7 +141,7 @@ static int

[PATCH 0/3] Btrfs: compression fixes

First patch fix copy paste typo in debug message in lzo.c, lzo is not deflate Second and third patches force btrfs not compress data if compression will not free at least one PAGE_SIZE Because it's useless in term of storage and read data from disk, as a result productivity suffers Timofey

enospc_debug seems to need more info, or btrfs file usage bug?

grep btrfs /proc/mounts /dev/sdb /mirror btrfs rw,noatime,compress=zlib,space_cache=v2,enospc_debug,subvolid=5,subvol=/ 0 0 Filesystem on 4.9.18 went read-only w/ ENOSPC. Now it is close to full, but can't really tell why it actually filled up at that time. Here's the dmesg: [1491582.099306]

Re: [PATCH RFC] vfs: add mount umount logs

On Fri, May 19, 2017 at 08:17:55AM +0800, Anand Jain wrote: > > XFS already logs its own unmounts. > > Nice. as far as I know its only in XFS. Ext4 logs mounts, but not unmounts. > > I prefer to let each filesystem log > > its own unmount, because then the mount/unmount messages also have the

Re: [PATCH 2/3] Btrfs: lzo compression must free at least PAGE_SIZE

Hi, Le 19/05/2017 à 15:38, Timofey Titovets a écrit : > If data compression didn't free at least one PAGE_SIZE, it useless to store > that compressed extent > > Signed-off-by: Timofey Titovets > --- > fs/btrfs/lzo.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-)

Re: [PATCH v2 0/2] btrfs: allow mechanism to override quota

On Fri, May 19, 2017 at 01:39:49AM -0700, Sargun Dhillon wrote: > On Fri, May 12, 2017 at 7:48 AM, David Sterba wrote: > > On Thu, May 11, 2017 at 09:17:01PM +, Sargun Dhillon wrote: > >> This patchset makes it so that on a per-filesystem basis one can disable > >> quota

Re: [PATCH RFC] vfs: add mount umount logs

On Thu, May 18, 2017, at 06:08 AM, Anand Jain wrote: > By looking at the logs we should be able to know when was the FS > mounted and unmounted and the options used, so to help forensic > investigations. Worth noting here that systemd already tracks mounts (via monitoring /proc/self/mountinfo)

Recurring free space warnings for same blocks

Hello, looking at the journals of three different machines running btrfs on crypt-devices on SSDs for root (dm-0 in the logs below) and other volumes, I'm seeing space cache warnings recurring for the same blocks over and over again. The machines are usually (re)booted once a day. See below

Re: [PATCH] Btrfs: work around maybe-uninitialized warning

On Thu, May 18, 2017 at 03:33:29PM +0200, Arnd Bergmann wrote: > A rewrite of btrfs_submit_direct_hook appears to have introduced a warning: > > fs/btrfs/inode.c: In function 'btrfs_submit_direct_hook': > fs/btrfs/inode.c:8467:14: error: 'bio' may be used uninitialized in this > function

Re: [PATCH] Btrfs: add statx support

On Fri, May 12, 2017 at 03:07:43PM -0700, Omar Sandoval wrote: > From: Yonghong Song > > Return enhanced file attributes from the btrfs, including: > (1). inode creation time as stx_btime, and > (2). Certain BTRFS_INODE_xxx flags are mapped to stx_attributes flags. > > Example

Re: btrfs metadata reclaim behavior/performance characteristics

On Fri, May 19, 2017 at 12:54:59PM +0300, Nikolay Borisov wrote: > > From: Liu Bo > > > > Subject: [PATCH] Btrfs: skip commit transaction if we don't have enough > > pinned bytes > > > > We commit transaction in order to reclaim space from pinned bytes because > > it

Re: [RFC PATCH] btrfs: Clean up btrfs_leaf_data

On Fri, May 19, 2017 at 10:49:10AM +0300, Nikolay Borisov wrote: > Commit 5f39d397dfbe ("Btrfs: Create extent_buffer interface for large > blocksizes") > refactored btrfs_leaf_data function to take extent_buffer rather than > struct btrfs_leaf. However, as it turns out the parameter being passed

[PATCH] Btrfs: skip commit transaction if we don't have enough pinned bytes

We commit transaction in order to reclaim space from pinned bytes because it could process delayed refs, and in may_commit_transaction(), we check first if pinned bytes are enough for the required space, we then check if that plus bytes reserved for delayed insert are enough for the required

Re: [PATCH 0/8 v1] utilize bio_clone_fast to clean up

On Wed, May 17, 2017 at 04:22:44PM -0600, Liu Bo wrote: > v1: - Drop the RFC tag. > - Update to use bio_segments accordingly as __bio_segments is removed. > - Remove if (!bio) since bio_clone_fast with bioset and GFP_NOFS will > never fail. > > This attempts to use bio_clone_fast()

Re: [PATCH v3 1/2] btrfs: Separate space_info create/update

Only minor nits On Fri, May 19, 2017 at 10:21:06AM +0300, Nikolay Borisov wrote: > Currently the struct space_info creation code is intermixed in the > udpate_space_info function. There are well-defined points at which the we ^ > actually want to create brand-new space_info

Re: [PATCH 1/3] btrfs: fix race with relocation recovery and fs_root setup

On Thu, May 18, 2017 at 04:58:19PM -0700, Liu Bo wrote: > On Wed, May 17, 2017 at 11:38:34AM -0400, je...@suse.com wrote: > > From: Jeff Mahoney > > > > If we have to recover relocation during mount, we'll ultimately have to > > evict the orphan inode. That goes through the

Re: [PATCH 2/3] Btrfs: lzo compression must free at least PAGE_SIZE

2017-05-19 23:19 GMT+03:00 Lionel Bouton : > I was too focused on other problems and having a fresh look at what I > wrote I'm embarrassed by what I read. > Used pages for a given amount of data should be (amount / PAGE_SIZE) + > ((amount % PAGE_SIZE) == 0 ? 0 : 1)

Re: [PATCH] Btrfs: work around maybe-uninitialized warning

On Fri, May 19, 2017 at 8:10 PM, Liu Bo wrote: > On Thu, May 18, 2017 at 03:33:29PM +0200, Arnd Bergmann wrote: >> A rewrite of btrfs_submit_direct_hook appears to have introduced a warning: >> >> fs/btrfs/inode.c: In function 'btrfs_submit_direct_hook': >>

Re: btrfs hang with 4.11.1 kernel

On Wed, May 17, 2017 at 05:44:02PM +0900, Tomasz Chmielewski wrote: > After upgrading to 4.11.1 and running for ~12 hours, btrfs filesystem hanged > - most processes on a server accessing the filesystem went into "D" state > and blocked. > > The server would not reboot and had to be power cycled.

Re: [PATCH 2/3] Btrfs: lzo compression must free at least PAGE_SIZE

Le 19/05/2017 à 16:17, Lionel Bouton a écrit : > Hi, > > Le 19/05/2017 à 15:38, Timofey Titovets a écrit : >> If data compression didn't free at least one PAGE_SIZE, it useless to store >> that compressed extent >> >> Signed-off-by: Timofey Titovets >> --- >>

Re: [PATCH] Btrfs: clear EXTENT_DEFRAG bits in finish_ordered_io

On Tue, May 09, 2017 at 05:02:15PM -0600, Liu Bo wrote: > Before this, we use 'filled' mode here, ie. if all range has been filled > with EXTENT_DEFRAG bits, get to clear it, but if the defrag range joins > the adjacent delalloc range, then we'll leave EXTENT_DEFRAG bits until > evicting inode. >

Re: [PATCH] Btrfs: clear EXTENT_DEFRAG bits in finish_ordered_io

On Fri, May 19, 2017 at 09:06:42PM +0200, David Sterba wrote: > On Tue, May 09, 2017 at 05:02:15PM -0600, Liu Bo wrote: > > Before this, we use 'filled' mode here, ie. if all range has been filled > > with EXTENT_DEFRAG bits, get to clear it, but if the defrag range joins > > the adjacent delalloc

Re: 4.11.0: kernel BUG at fs/btrfs/ctree.h:1779!

Hi Marc, On Thu, May 18, 2017 at 09:16:38PM -0700, Marc MERLIN wrote: > Looks like all the unhelpful BUG() aren't gone yet :-/ > This one is really not helpful, I don't even know which one of my filesystems > caused the crash :( > > Why is this not remounting the filesystem read only? > Really,

Re: [LKP] [btrfs] "fio: pid=2214, got signal=7" error showed in fio test for btrfs

On 03/29, Ye Xiaolong wrote: >Attach kmsg and reproduce script. > >Note: kernel cmdline contained "memmap=104G!4G memmap=104G!132G" Hi, Any feedback about this issue? Thanks, Xiaolong > >Thanks, >Xiaolong > >On 03/29, kernel test robot wrote: >>Hi, >> >>We detected below error messages in fio

Re: [PATCH 2/2] btrfs-progs: test for restoring multiple devices fs into a single device

Looks fine, just couple of minor feedback. > +run_check $SUDO_HELPER $TOP/mkfs.btrfs -f $loop1 $loop2 Please add quotation around variable in the script, as suggested in tests/README.md 'Coding style, best practices' section. This section is added recently, we are updating older scripts. > + >

[PATCH v3 1/2] btrfs: Separate space_info create/update

Currently the struct space_info creation code is intermixed in the udpate_space_info function. There are well-defined points at which the we actually want to create brand-new space_info structs (e.g. during mount of the filesystem as well as sometimes when adding/initialising new chunks). In such

[PATCH v3 2/2] btrfs: Refactor update_space_info

Following the factoring out of the creation code udpate_space_info can only be called for already-existing space_info structs. As such it cannot fail. Remove superfulous error handling and make the function return void. Reviewed-by: Jeff Mahoney Reviewed-by: Liu Bo

[RFC PATCH] btrfs: Clean up btrfs_leaf_data

Commit 5f39d397dfbe ("Btrfs: Create extent_buffer interface for large blocksizes") refactored btrfs_leaf_data function to take extent_buffer rather than struct btrfs_leaf. However, as it turns out the parameter being passed is never used. Help the brave souls which are going to read this code in

Re: [PATCH 06/10] btrfs: scrub: use bool for flush_all_writes

On Thu, May 18, 2017 at 05:42:24PM -0700, Liu Bo wrote: > On Tue, May 16, 2017 at 07:10:38PM +0200, David Sterba wrote: > > flush_all_writes is an atomic but does not use the semantics at all, > > it's just on/off indicator, we can use bool. > > > > It might use atomic to avoid reordering, but