Re: [PATCH v3 07/14] btrfs-progs: Doc/mkfs: Add extra condition for rootdir option

On 2017年09月30日 00:57, Goffredo Baroncelli wrote: On 09/28/2017 02:00 AM, Qu Wenruo wrote: On 2017年09月28日 00:20, David Sterba wrote: On Mon, Sep 25, 2017 at 07:15:30AM -0400, Austin S. Hemmelgarn wrote: On 2017-09-24 10:08, Goffredo Baroncelli wrote: On 09/24/2017 12:10 PM, Anand Jain

Re: [PATCH 1/3] Btrfs: remove nr_async_bios

On Wed, Sep 27, 2017 at 01:30:13PM +0200, David Sterba wrote: > On Thu, Sep 07, 2017 at 11:22:20AM -0600, Liu Bo wrote: > > This was intended to congest higher layers to not send bios, but as > > > > 1) the congested bit has been taken by writeback > > Can you please be more specific here? >

Re: Can't remove device -> I/O error

>> Did you removed the disk before mounting (physically or doing echo 1 >/sys/block/xxx/device/delete)? Which steps you performed ? - removed drive physically - mounted degraded mode - btrfs dev del -> same i/o error -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs"

Re: Can't remove device -> I/O error

On 09/29/2017 11:09 PM, DocMAX wrote: > Thanks for the reply. > > I don't want to replace the drive. I want to remove. > > Also tried in degraded mode. I get the exact same error. Did you removed the disk before mounting (physically or doing echo 1 >/sys/block/xxx/device/delete)? Which steps

Re: Can't remove device -> I/O error

Thanks for the reply. I don't want to replace the drive. I want to remove. Also tried in degraded mode. I get the exact same error. I'm not sure but i think i formated the drive on Kernel 4.11. I am on Kernel 4.13 now. I have the bad feeling that i will never get rid of that small drive

Re: Can't remove device -> I/O error

On 09/29/2017 10:00 PM, Dirk Diggler wrote: > Hi, > > is there any chance to get my device removed? I simulated a device removing in KVM with echo 1 >/sys/block/sdj/device/delete then btrfs dev del 6 /mnt/ And I got success. But I am not sure if this is the right thing

Re: Can't remove device -> I/O error

Kernel: Linux game 4.13.3-1-ARCH #1 SMP PREEMPT Thu Sep 21 20:33:16 CEST 2017 x86_64 GNU/Linux -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Can't remove device -> I/O error

Hi, is there any chance to get my device removed? Scrub literally takes months to complete (SATA 2/3 mix, about 1 minute per gigabyte) and i'm not sure if that helps. I guess same with balance. Mabye there is a quicker way. I can do without some data if it's corrupted. I have a backup, but i want

[PATCH 05/21] btrfs: pass root to various extent ref mod functions

We need the actual root for the ref verifier tool to work, so change these functions to pass the root around instead. This will be used in a subsequent patch. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.c | 2 +- fs/btrfs/ctree.h | 7 ---

[PATCH 04/21] btrfs: add ref-verify mount option

This adds the infrastructure for turning ref verify on and off for a mount, to be used by a later patch. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 1 + fs/btrfs/super.c | 15 +++ 2 files changed, 16 insertions(+) diff --git a/fs/btrfs/ctree.h

[PATCH 10/21] btrfs: breakout empty head cleanup to a helper

Move this code out to a helper function to further simplivy __btrfs_run_delayed_refs. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 80 -- 1 file changed, 45 insertions(+), 35 deletions(-) diff --git

[PATCH 16/21] btrfs: add a comp_refs() helper

Instead of open-coding the delayed ref comparisons, add a helper to do the comparisons generically and use that everywhere. We compare sequence numbers last for following patches. Signed-off-by: Josef Bacik --- fs/btrfs/delayed-ref.c | 54

[PATCH 07/21] Btrfs: only check delayed ref usage in should_end_transaction

We were only doing btrfs_check_space_for_delayed_refs() if the metadata space was full, ie we couldn't allocate chunks. This assumes we'll be able to allocate chunks during transaction commit, but since nothing does a LIMIT flush during the transaction commit this won't actually happen unless we

[PATCH 06/21] Btrfs: add a extent ref verify tool

We were having corruption issues that were tied back to problems with the extent tree. In order to track them down I built this tool to try and find the culprit, which was pretty successful. If you compile with this tool on it will live verify every ref update that the fs makes and make sure it

[PATCH 03/21] btrfs: make the delalloc block rsv per inode

The way we handle delalloc metadata reservations has gotten progressively more complicated over the years. There is so much cruft and weirdness around keeping the reserved count and outstanding counters consistent and handling the error cases that it's impossible to understand. Fix this by

[PATCH 08/21] btrfs: add a helper to return a head ref

Simplify the error handling in __btrfs_run_delayed_refs by breaking out the code used to return a head back to the delayed_refs tree for processing into a helper function. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 30 +++--- 1 file changed,

[PATCH 12/21] btrfs: move all ref head cleanup to the helper function

We do a couple different cleanup operations on the ref head. We adjust counters, we'll free any reserved space if we didn't end up using the ref, and we clear the pending csum bytes. Move all these disparate things into cleanup_ref_head and clean up the logic in __btrfs_run_delayed_refs so that

[PATCH 13/21] btrfs: remove delayed_ref_node from ref_head

This is just excessive information in the ref_head, and makes the code complicated. It is a relic from when we had the heads and the refs in the same tree, which is no longer the case. With this removal I've cleaned up a bunch of the cruft around this old assumption as well. Signed-off-by:

[PATCH 19/21] btrfs: don't call btrfs_start_delalloc_roots in flushoncommit

We're holding the sb_start_intwrite lock at this point, and doing async filemap_flush of the inodes will result in a deadlock if we freeze the fs during this operation. This is because we could do a btrfs_join_transaction() in the thread we are waiting on which would block at sb_start_intwrite,

[PATCH 17/21] btrfs: track refs in a rb_tree instead of a list

If we get a significant amount of delayed refs for a single block (think modifying multiple snapshots) we can end up spending an ungodly amount of time looping through all of the entries trying to see if they can be merged. This is because we only add them to a list, so we have O(2n) for every

[PATCH 18/21] btrfs: fix send ioctl on 32bit with 64bit kernel

We pass in a pointer in our send arg struct, this means the struct size doesn't match with 32bit user space and 64bit kernel space. Fix this by adding a compat mode and doing the appropriate conversion. Signed-off-by: Josef Bacik --- fs/btrfs/ioctl.c | 53

[PATCH 21/21] btrfs: add assertions for releasing trans handle reservations

These are useful for debugging problems where we mess with trans->block_rsv to make sure we're not screwing something up. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/extent-tree.c

[PATCH 14/21] btrfs: remove type argument from comp_tree_refs

We can get this from the ref we've passed in. Signed-off-by: Josef Bacik --- fs/btrfs/delayed-ref.c | 10 -- 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/delayed-ref.c b/fs/btrfs/delayed-ref.c index b9b41c838da4..a2973340a94f 100644 ---

[PATCH 01/21] Btrfs: rework outstanding_extents

Right now we do a lot of weird hoops around outstanding_extents in order to keep the extent count consistent. This is because we logically transfer the outstanding_extent count from the initial reservation through the set_delalloc_bits. This makes it pretty difficult to get a handle on how and

[PATCH 00/21] My current btrfs patch queue

while running xfstests. One is for 32bit ioctls on a 64bit kernel, two are lockdep fixes, and the last one is just so I can catch myself doing stupid stuff in the future. Dave this is based on for-next-20170929, and is in my btrfs-next tree in the master branch if you want to just pull it. Thanks

[PATCH 11/21] btrfs: move ref_mod modification into the if (ref) logic

We only use this logic if our ref isn't a ref_head, so move it up into the if (ref) case since we know that this is a normal ref and not a delayed ref head. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) diff

[PATCH 20/21] btrfs: move btrfs_truncate_block out of trans handle

Since we do a delalloc reserve in btrfs_truncate_block we can deadlock with freeze. If somebody else is trying to allocate metadata for this inode and it gets stuck in start_delalloc_inodes because of freeze we will deadlock. Be safe and move this outside of a trans handle. This also has a

[PATCH 15/21] btrfs: switch args for comp_*_refs

Because seriously? ref2 and then ref1? Signed-off-by: Josef Bacik --- fs/btrfs/delayed-ref.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/delayed-ref.c b/fs/btrfs/delayed-ref.c index a2973340a94f..bc940bb374cf 100644 ---

[PATCH 02/21] btrfs: add tracepoints for outstanding extents mods

This is handy for tracing problems with modifying the outstanding extents counters. Signed-off-by: Josef Bacik --- fs/btrfs/btrfs_inode.h | 2 ++ include/trace/events/btrfs.h | 21 + 2 files changed, 23 insertions(+) diff --git a/fs/btrfs/btrfs_inode.h

[PATCH 09/21] btrfs: move extent_op cleanup to a helper

Move the extent_op cleanup for an empty head ref to a helper function to help simplify __btrfs_run_delayed_refs. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 77 ++ 1 file changed, 40 insertions(+), 37 deletions(-) diff

Re: [PATCH v3 2/2] btrfs: Remove received_uuid during received snapshot ro->rw switch

On 29.09.2017 20:56, David Sterba wrote: > On Thu, Sep 28, 2017 at 10:53:18AM +0300, Nikolay Borisov wrote: >> Currently when a read-only snapshot is received and subsequently its ro >> property >> is set to false i.e. switched to rw-mode the received_uuid of that subvol >> remains >> intact.

Re: delayed inodes implementation questions

On Fri, Sep 29, 2017 at 04:00:29PM +0800, Qu Wenruo wrote: > On 2017年09月29日 15:15, Nikolay Borisov wrote: > > I've been studying the implementation of the delayed inodes feature and > > have a couple of questions. > > Sorry I can't answer your question, but as you may already know by the >

Re: [PATCH] btrfs: Use bd_dev to generate index when dev_state_hashtable add items.

On Fri, Sep 29, 2017 at 04:16:35PM +0800, Gu Jinxiang wrote: > From: Gu JinXiang > > Fix bug of > (). Please explain what bug are you fixing. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org

Re: [PATCH v3 2/2] btrfs: Remove received_uuid during received snapshot ro->rw switch

On Thu, Sep 28, 2017 at 10:53:18AM +0300, Nikolay Borisov wrote: > Currently when a read-only snapshot is received and subsequently its ro > property > is set to false i.e. switched to rw-mode the received_uuid of that subvol > remains > intact. However, once the received volume is switched to

Re: [PATCH 1/2] btrfs: Explicitly handle btrfs_update_root failure

On Thu, Sep 28, 2017 at 10:53:17AM +0300, Nikolay Borisov wrote: > btrfs_udpate_root can fail and it aborts the transaction, the correct way to > handle an aborted transaction is to explicitly end with btrfs_end_transaction. > Even now the code is correct since btrfs_commit_transaction would

Re: [PATCH v4 1/3] btrfs: undo writable when sprouting fails

On Thu, Sep 28, 2017 at 02:51:09PM +0800, Anand Jain wrote: > When new device is being added to seed FS, seed FS is marked writable, > but when we fail to bring in the new device, we missed to undo the > writable part. This patch fixes it. > > Signed-off-by: Anand Jain >

Re: [PATCH] btrfs: take the error path out if btrfs_attach_transaction() fails

On Thu, Sep 28, 2017 at 03:13:06PM +0800, Anand Jain wrote: > > I'd suggest to first get rid of the in-place returns and add necessary > > labels or separate exit sequences and then address the new error > > handling. > >As it goes deeper there are quite a number of things which aren't >

Re: [PATCH 1/7] btrfs-progs: check: supplement extent backref list with rbtree

On Tue, Jul 25, 2017 at 04:51:32PM -0400, je...@suse.com wrote: > From: Jeff Mahoney > > For the pathlogical case, like xfstests generic/297 that creates a > large file consisting of one, repeating reflinked extent, fsck can > take hours. The root cause is that calling

Re: [PATCH v3 07/14] btrfs-progs: Doc/mkfs: Add extra condition for rootdir option

On 09/28/2017 02:00 AM, Qu Wenruo wrote: > > > On 2017年09月28日 00:20, David Sterba wrote: >> On Mon, Sep 25, 2017 at 07:15:30AM -0400, Austin S. Hemmelgarn wrote: >>> On 2017-09-24 10:08, Goffredo Baroncelli wrote: On 09/24/2017 12:10 PM, Anand Jain wrote: >> >> A lot of points in this

Re: [PATCH] btrfs-progs: fix invalid assert in backref.c

On Thu, Sep 28, 2017 at 04:55:00PM -0400, Josef Bacik wrote: > This should be verify'ing that we have an empty key, not that we have a > filled out key. > > Signed-off-by: Josef Bacik > --- > Dave this is on top of your ext/jeffm/extent-cache branch and fixes the > segfault > you

Re: [PATCH v8 0/6] Btrfs: populate heuristic with code

On Thu, Sep 28, 2017 at 05:33:35PM +0300, Timofey Titovets wrote: > Compile tested, hand tested on live system > > Change v7 -> v8 > - All code moved to compression.c (again) > - Heuristic workspaces inmplemented another way > i.e. only share logic with compression workspaces > - Some

[GIT PULL] Btrfs fixes for 4.14-rc3

Hi, we've collected a bunch of isolated fixes, for crashes, user-visible behaviour or missing bits from other subsystem cleanups from the past. The overall number is not small but I was not able to make it significantly smaller. Most of the patches are supposed to go to stable. There are no

Re: [PATCH 0/4 v2] add subvolume support to mkfs

On Wed, Aug 30, 2017 at 10:24:46AM -0700, Yingyi Luo wrote: > From: yingyil > > Hi all, > > Thanks for all the comments you gave to me. I really appriciate it. I've > updated the patch, split it into small ones so that it may be easier > for you to see the changes and

Re: [PATCH v3 07/14] btrfs-progs: Doc/mkfs: Add extra condition for rootdir option

On 2017-09-27 20:00, Qu Wenruo wrote: On 2017年09月28日 00:20, David Sterba wrote: On Mon, Sep 25, 2017 at 07:15:30AM -0400, Austin S. Hemmelgarn wrote: On 2017-09-24 10:08, Goffredo Baroncelli wrote: On 09/24/2017 12:10 PM, Anand Jain wrote: A lot of points in this thread, let me address

[PATCH] btrfs: Use bd_dev to generate index when dev_state_hashtable add items.

From: Gu JinXiang Fix bug of (). Signed-off-by: Gu JinXiang --- fs/btrfs/check-integrity.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/check-integrity.c b/fs/btrfs/check-integrity.c index 9d3854839038..86d79bc4cfb3

Re: delayed inodes implementation questions

On 29.09.2017 11:00, Qu Wenruo wrote: > > > On 2017年09月29日 15:15, Nikolay Borisov wrote: >> Hello, >> >> >> I've been studying the implementation of the delayed inodes feature and >> have a couple of questions. > > Sorry I can't answer your question, but as you may already know by the >

Re: delayed inodes implementation questions

On 2017年09月29日 15:15, Nikolay Borisov wrote: Hello, I've been studying the implementation of the delayed inodes feature and have a couple of questions. Sorry I can't answer your question, but as you may already know by the rejection mail, Miao Xie has left Fujitsu and has been for several

[PATCH] btrfs: fix max chunk size on dup

Balancing a fresh METADATA=dup btrfs file system (with size < 50G) generates a 128MB sized block group. While we set max_stripe_size = max_chunk_size = 256MB, we get this half sized block group: $ btrfs ins dump-t -t CHUNK_TREE btrfs.img|grep length length 8388608 owner 2

delayed inodes implementation questions

Hello, I've been studying the implementation of the delayed inodes feature and have a couple of questions. 1. Why are there 2 lists when each and every node is on both lists at the same time? So if we happen to be running the async worker we are processing nodes based on the prepared_list, at

[PATCH v3 3/5] btrfs: tree-checker: Enhance output for btrfs_check_leaf

Enhance the output to print: 1) Reason 2) Bad value If reason can't explain enough 3) Good value (range) Signed-off-by: Qu Wenruo --- fs/btrfs/tree-checker.c | 27 +-- 1 file changed, 21 insertions(+), 6 deletions(-) diff --git

[PATCH v3 5/5] btrfs: tree-checker: Enhance output for check_extent_data_item

Output the invalid member name and its bad value, along with its expected value range or alignment. Signed-off-by: Qu Wenruo --- fs/btrfs/tree-checker.c | 98 +++-- 1 file changed, 70 insertions(+), 28 deletions(-) diff --git

[PATCH v3 0/5] Enhance tree block validation checker

The patchset can be fetched from github: https://github.com/adam900710/linux/tree/checker_enhance It's based on David's misc-next branch, with following commit as base: a5e50b4b444c ("btrfs: Add checker for EXTENT_CSUM") According to David's suggestion, enhance the output format of tree block

[PATCH v3 4/5] btrfs: tree-checker: Enhance output for check_csum_item

Output the bad value and expected good value (or its alignment). Signed-off-by: Qu Wenruo --- fs/btrfs/tree-checker.c | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/tree-checker.c b/fs/btrfs/tree-checker.c index

[PATCH v3 1/5] btrfs: Move leaf and node validation checker to tree-checker.c

It's no doubt the comprehensive tree block checker will become larger and larger, so move them into their own file is quite reasonable. Signed-off-by: Qu Wenruo --- fs/btrfs/Makefile | 2 +- fs/btrfs/ctree.h| 4 + fs/btrfs/disk-io.c | 284

[PATCH v3 2/5] btrfs: tree-checker: Enhance btrfs_check_node output

Use inline function to replace macro since we don't need stringification. (Macro still exist until all caller get updated) And add more info about the error, and replace EIO with EUCLEAN. For nr_items error, report if it's too large or too small, and output valid value range. For blk pointer,

WARNING: CPU: 1 PID: 13825 at fs/btrfs/backref.c:1255 find_parent_nodes+0xb5c/0x1310

Hi, Just ran into this warning while running deduplication. There were 10's of thousands of them over a 24hr period. No other problems were reported. Filesystem is raid1, freshly converted from single. Zstd compression.  4.14.0-rc2 kernel Sep 28 14:57:06 home kernel: [ cut here

Re: [PATCH v2 2/5] btrfs: tree-checker: Enhance btrfs_check_node output

On 2017年09月29日 14:05, Nikolay Borisov wrote: On 29.09.2017 04:36, Qu Wenruo wrote: Use inline function to replace macro since we don't need stringification. (Macro still exist until all caller get updated) And add more info about the error. For nr_items error, report if it's too large or

Re: [PATCH v2 2/5] btrfs: tree-checker: Enhance btrfs_check_node output

On 29.09.2017 04:36, Qu Wenruo wrote: > Use inline function to replace macro since we don't need > stringification. > (Macro still exist until all caller get updated) > > And add more info about the error. > > For nr_items error, report if it's too large or too small, and output > valid value